A served model's longest answer changed in place, and Strata's context
CI / test (push) Successful in 9m29s

max_output on POST /api/v1/models/{name}/context, --max-output on ctl model
context, and a field in the console's Context length… form. Alone it changes
only the catalog's limit, which the router holds max_tokens to, and starts no
replica again; with a context it is set with it, never past it.

Strata's context is now changed like the others', through its setup's
--context, which writes the engine's start script at each start. Neither
needs a model removed and added again any more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-01 00:15:38 +01:00
1 parent f4b34dbf11
commit 31f440ab6e
9 files changed
+200 -37

No files matched your search

+12 -2
View File
@@ -2730,7 +2730,7 @@
"/api/v1/models/{name}/context": {
"post": {
"operationId": "postModelsNameContext",
"summary": "Change a served model's context: the catalog, its engines' arguments and memory, then its replicas restarted",
"summary": "Change a served model's context and longest answer: the catalog, its engines' arguments and memory, then its replicas restarted; max_output alone changes only the catalog and restarts nothing",
"tags": [
"models"
],
@@ -11273,6 +11273,13 @@
"max_output": {
"type": "integer"
},
"output_only": {
"type": "boolean",
"description": "OutputOnly is a change of the longest answer alone, MaxOutputBefore what it was (0 no limit); nothing was started again."
},
"max_output_before": {
"type": "integer"
},
"trained": {
"type": "integer"
},
@@ -11317,13 +11324,16 @@
},
"modelserve.ContextRequest": {
"type": "object",
"description": "ContextRequest is the context to change to: tokens one request may use, and for llama.cpp how many at once (its pool is the two multiplied).",
"description": "ContextRequest is the context to change to: tokens one request may use, and for llama.cpp how many at once (its pool is the two multiplied). MaxOutput is the longest answer, which may be changed with it or alone: Tokens 0 with a MaxOutput changes only that, in the catalog, which the router reads it from — nothing is started again.",
"properties": {
"tokens": {
"type": "integer"
},
"slots": {
"type": "integer"
},
"max_output": {
"type": "integer"
}
}
},