The picker listed what had an endpoint, and a model at min 0 has none until
somebody asks for it. On this page a message is the asking, so an on-demand
model could never be started from here: mistral-nemo-12b-instruct sat in the
catalog, served by nothing, and absent from the one place someone went to
try it.
Such a model is now offered, marked "not loaded", with a note saying the
first message fetches the weights and starts the engine, and that until it
is up the reply is "the model is loading". A model stopped outright — max 0 —
is still left out, because nothing would load it and offering it would only
produce an error.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The page imposes its own ceiling on a turn, so a playground request cannot
occupy a replica for ten minutes. It sent that ceiling regardless of the
model, and a model declaring a smaller max_output then answered every turn
with "max_tokens is 2048, but this model allows at most 1024" — the gateway
refusing its own page, over a number the person never chose and has no way
to change.
The ceiling is the page's to impose on itself and the model's limit is not
negotiable, so it sends the smaller of the two.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An endpoint advertises every name a client may address it by — the model's
own and all of its aliases — because that is what the router matches a
request against. The picker took those strings as they came, so one model
appeared as several: "coder", "coder-7b" and "qwen2.5-coder-7b-instruct"
were three choices, and choosing between them is a question with no answer.
Each name is now resolved through the catalog and the model listed once,
under its catalog name — the one that also appears in the model list, the
usage table and the logs.
/v1/models still lists the aliases separately, which is deliberate: there a
client is discovering which ids it may send, and every alias is one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>