Add an in-browser chat, without putting a key in the browser

So someone can try a model without setting up a client, and — more often —
tell whether a problem is the gateway or their code.

The page carries a session, and the server turns that into the identity an API
key would have produced, resolved from the person's project. The request then
goes down the ordinary path by being handed to the same handler /v1 uses:
admission, model allowlist, rate limits, budget, accounting. There is exactly
one request path, so there is no second place for its rules to drift — and
without that, a playground is a hole in every limit the rest of the gateway
enforces. The allowlist test posts a forbidden model and expects 403 for that
reason.

The body is rebuilt rather than forwarded, so the page cannot set fields it has
no business setting: a stream:false that would hang the reader, a max_tokens
large enough to occupy a replica for an hour. One turn is capped at 512 tokens.

The picker offers what is actually served and allowed, not what is catalogued,
because offering a model with no replica produces a 503 the reader cannot
interpret.

Inline vanilla JavaScript, for the same reason the charts are hand-drawn SVG.
It streams, because on CPU inference ten seconds is normal and a chat that
hangs and then dumps a paragraph feels broken. Replies are written with
textContent throughout: model output and user input have no business being
parsed as markup. Its POSTs carry the CSRF token in a header, which a
cross-site form cannot set at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 12:05:30 +02:00
1 parent 859f08da9b
commit de21649ad6
10 files changed
+567 -8

No files matched your search

+21
View File
@@ -236,6 +236,27 @@ Tell them out of band and have them change it from their profile. Their sessions
are closed — if they lost the password, the reason might be that someone else has
it. Their API keys keep working; those are separate credentials.
## Trying a model in the browser
`/chat` is a playground for anyone signed in. It exists so somebody can see
whether a model works without setting up a client, and — more often — so they
can tell whether a problem is the gateway or their code.
**The browser never holds a key.** The page carries a session; the server turns
that into the same identity an API key would produce, resolved from the person's
project, and the request goes down the ordinary path: admission, model
allowlist, rate limits, budget, accounting. Anything else would make the page a
hole in every limit the rest of the gateway enforces.
Usage from the page is attributed to `session:<user-id>` rather than a key, so
it is billed to the right project and visibly did not come from a credential
somebody holds. One turn is capped at 512 tokens: a playground should not be the
thing that ties up a replica for ten minutes, and anyone who wants a longer
answer has an API key for that.
Nothing is stored. The conversation lives in the page and is gone when it is
closed.
## Seeing the cluster
The dashboard — nodes, replicas, models, why something did not get placed, usage