big-model runs on the sim engine, whose whole vocabulary is the Greek
alphabet, cycled at a fixed rate until max_tokens. That is what it is for:
with no H100s it is the only thing exercising the TP8 two-node path, the
fabric invariant and rebuild-after-node-loss. The output is the point.
What was wrong is that nothing said so. /v1/models returned it with
owned_by "llmgw", identical to the real model, and the chat picker sorted
purely by name — so "big-model" led the list, was preselected by the
browser, and anyone who typed a question and pressed Send got Greek
letters back with no explanation. Reported as the model spitting out crap,
which is a fair reading of what it looks like from outside.
Simulated is derived from the profiles rather than configured: an operator
who adds a sim profile and forgets a flag would be shipping placeholder
text under a real model's name. A model counts as simulated only when
every profile is, because one real profile means the request can still
come back as inference.
The marker on /v1/models is owned_by "llmgw-sim", which is already what
the sim engine calls itself on its own /v1/models — the gateway was
flattening a distinction that existed one layer down.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replies are plain text, so a model asked about an integral answers with
\int_0^1 x^2 dx and the reader gets the source.
KaTeX is vendored into the binary rather than loaded from a CDN. This gateway
makes no outbound requests of its own and is expected to run where it could
not make one; a chat page that silently loses its maths on an air-gapped
cluster would be worse than one that never had it. 600 KB of JS, CSS and WOFF2
fonts, which the stripped binary absorbs without changing its rounded size.
The ordering is the safety argument, and there is a test for it rather than a
comment alone. The reply is put in place with textContent exactly as before,
and only then walked for maths: renderMathInElement replaces the spans it
recognises, so nothing the model wrote is ever parsed as HTML and the markup
that appears is KaTeX's. trust stays false — KaTeX's default, and what keeps
\href and \includegraphics out of a reply — and throwOnError is off so one
malformed expression does not take the rest with it.
Rendered once the reply is complete. Typesetting each chunk would try to set
half an expression on every token and flicker through the whole answer.
The static handler now serves a small embedded asset tree rather than one
file, so it moved to its own embed: a file server over the tree that also
carries the templates would serve those too. Directory listings are refused —
nothing links to one, so the only requests are from somebody looking around.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>