Commit Graph
3 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 2cae92d113 Offer a model that loads on demand in the chat picker
The picker listed what had an endpoint, and a model at min 0 has none until
somebody asks for it. On this page a message is the asking, so an on-demand
model could never be started from here: mistral-nemo-12b-instruct sat in the
catalog, served by nothing, and absent from the one place someone went to
try it.

Such a model is now offered, marked "not loaded", with a note saying the
first message fetches the weights and starts the engine, and that until it
is up the reply is "the model is loading". A model stopped outright — max 0 —
is still left out, because nothing would load it and offering it would only
produce an error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 10:13:42 +02:00
Ilker ManapandClaude Opus 5 f80eef9fa6 Stop the chat page asking for more tokens than the model allows
CI / test (push) Successful in 1m49s
The page imposes its own ceiling on a turn, so a playground request cannot
occupy a replica for ten minutes. It sent that ceiling regardless of the
model, and a model declaring a smaller max_output then answered every turn
with "max_tokens is 2048, but this model allows at most 1024" — the gateway
refusing its own page, over a number the person never chose and has no way
to change.

The ceiling is the page's to impose on itself and the model's limit is not
negotiable, so it sends the smaller of the two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 00:48:07 +02:00
Ilker ManapandClaude Opus 5 c848b2c73a Offer a model once in the chat picker, not once per name
CI / test (push) Successful in 1m46s
An endpoint advertises every name a client may address it by — the model's
own and all of its aliases — because that is what the router matches a
request against. The picker took those strings as they came, so one model
appeared as several: "coder", "coder-7b" and "qwen2.5-coder-7b-instruct"
were three choices, and choosing between them is a question with no answer.

Each name is now resolved through the catalog and the model listed once,
under its catalog name — the one that also appears in the model list, the
usage table and the logs.

/v1/models still lists the aliases separately, which is deliberate: there a
client is discovering which ids it may send, and every alias is one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-28 00:24:16 +02:00