A served model's longest answer changed in place, and Strata's context

max_output on POST /api/v1/models/{name}/context, --max-output on ctl model
context, and a field in the console's Context length… form. Alone it changes
only the catalog's limit, which the router holds max_tokens to, and starts no
replica again; with a context it is set with it, never past it.

Strata's context is now changed like the others', through its setup's
--context, which writes the engine's start script at each start. Neither
needs a model removed and added again any more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

CHANGELOG.md

@@ -30,6 +30,13 @@ APIs change between minors without a deprecation period.
 
 ### Added
 
+- **A served model's longest answer, changed in place.** `llmgw ctl model
+  context <model> --max-output N` (and `max_output` on `POST
+  /api/v1/models/{name}/context`, and the console's Context length… form)
+  sets the longest answer on its own: the catalog's limit, in force at once,
+  with no replica started again. With a context it is set with it. Strata's
+  context can now be changed too (its setup's `--context`), so neither
+  needs the model removed and added again.
 - **FreeToken, an engine for models larger than the card.** Mixture-of-experts
   models several times a card's size, from one card, with the experts in RAM
   and streamed over PCIe — read from Hugging Face safetensors checkpoints as

docs/admin-api.openapi.json

@@ -2730,7 +2730,7 @@
     "/api/v1/models/{name}/context": {
       "post": {
         "operationId": "postModelsNameContext",
-        "summary": "Change a served model's context: the catalog, its engines' arguments and memory, then its replicas restarted",
+        "summary": "Change a served model's context and longest answer: the catalog, its engines' arguments and memory, then its replicas restarted; max_output alone changes only the catalog and restarts nothing",
         "tags": [
           "models"
         ],
@@ -11273,6 +11273,13 @@
           "max_output": {
             "type": "integer"
           },
+          "output_only": {
+            "type": "boolean",
+            "description": "OutputOnly is a change of the longest answer alone, MaxOutputBefore what it was (0 no limit); nothing was started again."
+          },
+          "max_output_before": {
+            "type": "integer"
+          },
           "trained": {
             "type": "integer"
           },
@@ -11317,13 +11324,16 @@
       },
       "modelserve.ContextRequest": {
         "type": "object",
-        "description": "ContextRequest is the context to change to: tokens one request may use, and for llama.cpp how many at once (its pool is the two multiplied).",
+        "description": "ContextRequest is the context to change to: tokens one request may use, and for llama.cpp how many at once (its pool is the two multiplied). MaxOutput is the longest answer, which may be changed with it or alone: Tokens 0 with a MaxOutput changes only that, in the catalog, which the router reads it from — nothing is started again.",
         "properties": {
           "tokens": {
             "type": "integer"
           },
           "slots": {
             "type": "integer"
+          },
+          "max_output": {
+            "type": "integer"
           }
         }
       },

docs/install.md

@@ -1767,23 +1767,37 @@ llmgw ctl model remove qwen-7b --purge   # its files on the mirror and the nodes
 The console has the same in a model's menu: Stop… or Start, Remove…, and
 Remove with its files….
 
-### Changing a served model's context
+### Changing a served model's context and longest answer
 
-How much one request may use is set when a model is served. To change it
-afterwards:
+How much one request may use, and how long an answer may be, are set when a
+model is served. To change them afterwards, in place:
 
 ```
-llmgw ctl model context qwen-7b 32768             # tokens a request
-llmgw ctl model context qwen-7b 32768 --slots 4   # llama.cpp: four at once
+llmgw ctl model context qwen-7b 32768                     # tokens a request
+llmgw ctl model context qwen-7b 32768 --slots 4           # llama.cpp: four at once
+llmgw ctl model context qwen-7b 32768 --max-output 8192   # and the longest answer
+llmgw ctl model context qwen-7b --max-output 4096         # the longest answer alone
 ```
 
-The catalog, the engine's arguments (`--ctx-size` and `--parallel` for
-llama.cpp, `--max-model-len` for vLLM, `--context-length` for SGLang) and
-the memory its KV cache takes change together. A context longer than the
+The longest answer is the gateway's: a request asking for more `max_tokens`
+is refused, `/v1/models` lists it as `max_output_tokens` for clients to read,
+and the account page's chat asks for no more. Changed alone it is only
+the catalog's, in force at once, and no replica is started again. With a
+context it is set with it; left out, it grows and shrinks with the context
+as before, unless it was set to something else. It never exceeds the context.
+
+For a context, the catalog, the engine's arguments (`--ctx-size` and
+`--parallel` for llama.cpp, `--max-model-len` for vLLM, `--context-length` for
+SGLang, `--context` for Strata's setup, `--max-seq-len-override` for
+FreeToken) and the memory its KV cache takes change together. Strata and
+FreeToken size their caches to the card themselves, so their cards are not
+checked here. A context longer than the
 model was trained for is refused, and so is one its cards cannot hold — with
 the figures, so you know how far to go. Its replicas are then started again on
 it, one by one, a new one ready before the old one stops. The console has the
-same under a model's menu, Context length…. A model declared in the cluster's
+same under a model's menu, Context length…, with the longest answer beside
+it. The admin API is `POST /api/v1/models/{name}/context` with `tokens`,
+`slots` and `max_output`, any of them left out. A model declared in the cluster's
 settings is changed there (`llmgw ctl config edit`).
 
 ### Moving a served model to more cards

internal/cli/model.go

@@ -472,10 +472,10 @@ func newAdaptersCmd(client func() *ctlClient) *cobra.Command {
 }
 
 func newModelContextCmd(client func() *ctlClient) *cobra.Command {
-	var slots int
+	var slots, maxOut int
 	c := &cobra.Command{
-		Use:   "context <model> <tokens>",
-		Short: "Change how much one request to a served model may use",
+		Use:   "context <model> [tokens]",
+		Short: "Change how much one request to a served model may use, and how long its answers may be",
 		Long: "The context lives in three places: the catalog (what a request may use, and\n" +
 			"what it may answer), the engine's arguments (llama.cpp's --ctx-size and\n" +
 			"--parallel, vLLM's --max-model-len), and the memory the KV cache takes. This\n" +
@@ -483,21 +483,39 @@ func newModelContextCmd(client func() *ctlClient) *cobra.Command {
 			"and a context its cards would not hold, saying what each would need. Its\n" +
 			"replicas are then started again on it, each new one before the old stops.\n\n" +
 			"--slots is how many requests a llama.cpp replica answers at once; its pool is\n" +
-			"slots × tokens. Left out, it keeps what it has.",
+			"slots × tokens. Left out, it keeps what it has.\n\n" +
+			"--max-output is the longest answer the gateway lets a request ask for. With a\n" +
+			"context it is set with it; left out, it follows the context as it did. Alone,\n" +
+			"without tokens, it changes only that, and no replica is started again.\n\n" +
+			"Strata's context is its setup's --context, and FreeToken's\n" +
+			"--max-seq-len-override; for both the card is not checked, since each sizes its\n" +
+			"caches to it.",
 		Example: "  llmgw ctl model context qwen3-coder-30b 65536\n" +
-			"  llmgw ctl model context qwen2.5-7b 16384 --slots 4",
-		Args: cobra.ExactArgs(2),
+			"  llmgw ctl model context qwen2.5-7b 16384 --slots 4\n" +
+			"  llmgw ctl model context qwen3.8-flash-next-iq2_xs 32768 --max-output 8192\n" +
+			"  llmgw ctl model context qwen3.8-flash-next-iq2_xs --max-output 4096",
+		Args: cobra.RangeArgs(1, 2),
 		RunE: func(cmd *cobra.Command, args []string) error {
-			tokens, err := strconv.Atoi(args[1])
-			if err != nil {
-				return fmt.Errorf("the context is a whole number of tokens, not %q", args[1])
+			tokens := 0
+			if len(args) == 2 {
+				var err error
+				if tokens, err = strconv.Atoi(args[1]); err != nil {
+					return fmt.Errorf("the context is a whole number of tokens, not %q", args[1])
+				}
+			} else if maxOut == 0 {
+				return fmt.Errorf("say the context's tokens, or --max-output, or both")
 			}
-			body, _ := json.Marshal(modelserve.ContextRequest{Tokens: tokens, Slots: slots})
+			body, _ := json.Marshal(modelserve.ContextRequest{Tokens: tokens, Slots: slots, MaxOutput: maxOut})
 			var v modelserve.ContextChange
 			if err := client().patient(time.Minute).post(cmd.Context(), "/api/v1/models/"+args[0]+"/context", body, &v); err != nil {
 				return err
 			}
 			out := cmd.OutOrStdout()
+			if v.OutputOnly {
+				fmt.Fprintf(out, "%s: answers up to %d tokens (was %d); its context stays %d, and no replica was started again\n",
+					v.Model, v.MaxOutput, v.MaxOutputBefore, v.After)
+				return nil
+			}
 			fmt.Fprintf(out, "%s: %d → %d tokens a request, answers up to %d", v.Model, v.Before, v.After, v.MaxOutput)
 			if v.Trained > 0 {
 				fmt.Fprintf(out, " (trained for %d)", v.Trained)
@@ -523,6 +541,7 @@ func newModelContextCmd(client func() *ctlClient) *cobra.Command {
 		},
 	}
 	c.Flags().IntVar(&slots, "slots", 0, "llama.cpp: requests a replica answers at once (default: as it is)")
+	c.Flags().IntVar(&maxOut, "max-output", 0, "the longest answer, in tokens (default: follows the context)")
 	return c
 }
 

internal/cli/tui_actions.go

@@ -346,10 +346,12 @@ func (m consoleModel) modelMenu(md adminapi.ModelView) *menuDialog {
 		}},
 		{"Context length…", func(m *consoleModel) tea.Cmd {
 			m.openForm(&formDialog{title: "Context of " + md.Name,
-				note: "What one request may use. The engine's arguments and the memory change with it, checked against " +
-					"what the cards hold; its replicas are then started again on it, one by one.",
+				note: "What one request may use, and the longest answer. A changed context changes the engine's arguments " +
+					"and memory, checked against what the cards hold, and starts its replicas again one by one; the " +
+					"longest answer alone changes only the gateway's limit, at once.",
 				fields: []formField{
 					textF("tokens", strconv.Itoa(md.ContextLength), "a request's prompt and answer together"),
+					textF("longest answer", strconv.Itoa(md.MaxOutput), "tokens; empty follows the context"),
 					textF("slots", "", "llama.cpp: requests at once; empty keeps it"),
 				},
 				submit: func(m *consoleModel, v []string) (tea.Cmd, error) {
@@ -357,14 +359,25 @@ func (m consoleModel) modelMenu(md adminapi.ModelView) *menuDialog {
 					if err != nil || tokens < 256 {
 						return nil, fmt.Errorf("the context is a whole number of tokens, at least 256")
 					}
-					slots := 0
+					maxOut := 0
 					if v[1] != "" {
-						if slots, err = strconv.Atoi(v[1]); err != nil || slots < 1 {
+						if maxOut, err = strconv.Atoi(v[1]); err != nil || maxOut < 1 {
+							return nil, fmt.Errorf("the longest answer is a whole number of tokens")
+						}
+					}
+					slots := 0
+					if v[2] != "" {
+						if slots, err = strconv.Atoi(v[2]); err != nil || slots < 1 {
 							return nil, fmt.Errorf("slots is a whole number from 1")
 						}
 					}
-					return m.call("Context of "+md.Name, http.MethodPost, path+"/context",
-						modelserve.ContextRequest{Tokens: tokens, Slots: slots}, contextSays), nil
+					req := modelserve.ContextRequest{Tokens: tokens, Slots: slots, MaxOutput: maxOut}
+					if tokens == md.ContextLength && slots == 0 && maxOut > 0 {
+						// The context as it is: only the longest answer, and
+						// no replica started again for it.
+						req.Tokens = 0
+					}
+					return m.call("Context of "+md.Name, http.MethodPost, path+"/context", req, contextSays), nil
 				}})
 			return nil
 		}},
@@ -999,6 +1012,9 @@ func contextSays(raw json.RawMessage) string {
 	if json.Unmarshal(raw, &v) != nil {
 		return "Done."
 	}
+	if v.OutputOnly {
+		return fmt.Sprintf("%s: answers up to %d tokens (was %d); no replica was started again.", v.Model, v.MaxOutput, v.MaxOutputBefore)
+	}
 	s := fmt.Sprintf("%s: %d → %d tokens a request, answers up to %d.", v.Model, v.Before, v.After, v.MaxOutput)
 	for _, p := range v.Profiles {
 		if p.After != "" {

internal/control/adminapi/routes.go

@@ -221,7 +221,7 @@ var Routes = []Route{
 	{Method: "POST", Path: "/api/v1/models/{name}/update", Summary: "Move a model to its repository's newer weights",
 		Description: "With a quality suite, or the model's standing gate, the new weights are asked it first as a hidden canary.",
 		Request:     modelserve.Change{}, OptionalBody: true, Response: modelserve.Job{}, Status: http.StatusAccepted},
-	{Method: "POST", Path: "/api/v1/models/{name}/context", Summary: "Change a served model's context: the catalog, its engines' arguments and memory, then its replicas restarted",
+	{Method: "POST", Path: "/api/v1/models/{name}/context", Summary: "Change a served model's context and longest answer: the catalog, its engines' arguments and memory, then its replicas restarted; max_output alone changes only the catalog and restarts nothing",
 		Request: modelserve.ContextRequest{}, Response: modelserve.ContextChange{}},
 	{Method: "POST", Path: "/api/v1/models/{name}/gpus", Summary: "Spread a served model over more cards or fewer: its profile, checked against its KV heads, its cards' memory and the nodes, then its replicas restarted",
 		Description: "Tensor parallel for vLLM, SGLang and TensorRT-LLM, a layer split for llama.cpp, on one node. With no room beside the running replicas, in_place stops each first.",

internal/control/modelserve/context.go

@@ -21,9 +21,13 @@ import (
 
 // ContextRequest is the context to change to: tokens one request may use,
 // and for llama.cpp how many at once (its pool is the two multiplied).
+// MaxOutput is the longest answer, which may be changed with it or alone:
+// Tokens 0 with a MaxOutput changes only that, in the catalog, which the
+// router reads it from — nothing is started again.
 type ContextRequest struct {
-	Tokens int `json:"tokens"`
-	Slots  int `json:"slots,omitempty"`
+	Tokens    int `json:"tokens,omitempty"`
+	Slots     int `json:"slots,omitempty"`
+	MaxOutput int `json:"max_output,omitempty"`
 }
 
 // ContextProfile is what one profile needs before and after.
@@ -37,13 +41,17 @@ type ContextProfile struct {
 
 // ContextChange is what was changed, and the replicas started again on it.
 type ContextChange struct {
-	Model     string           `json:"model"`
-	Before    int              `json:"before"`
-	After     int              `json:"after"`
-	MaxOutput int              `json:"max_output"`
-	Trained   int              `json:"trained,omitempty"`
-	Profiles  []ContextProfile `json:"profiles"`
-	Restarted int              `json:"restarted"`
+	Model     string `json:"model"`
+	Before    int    `json:"before"`
+	After     int    `json:"after"`
+	MaxOutput int    `json:"max_output"`
+	// OutputOnly is a change of the longest answer alone, MaxOutputBefore
+	// what it was (0 no limit); nothing was started again.
+	OutputOnly      bool             `json:"output_only,omitempty"`
+	MaxOutputBefore int              `json:"max_output_before,omitempty"`
+	Trained         int              `json:"trained,omitempty"`
+	Profiles        []ContextProfile `json:"profiles"`
+	Restarted       int              `json:"restarted"`
 	// Note is what could not be checked, said once.
 	Note string `json:"note,omitempty"`
 }
@@ -75,8 +83,14 @@ func (s *Server) SetContext(ctx context.Context, name string, req ContextRequest
 		return ContextChange{}, fmt.Errorf("no model %q in the catalog", name)
 	case s.deps.Models.Declared(name):
 		return ContextChange{}, fmt.Errorf("model %q is declared in the cluster's settings: change its context there (llmgw ctl config edit)", name)
+	case req.MaxOutput < 0:
+		return ContextChange{}, errors.New("the longest answer is a number of tokens")
+	case req.Tokens == 0 && req.MaxOutput > 0:
+		return s.setMaxOutput(ctx, m, req.MaxOutput, by)
 	case req.Tokens < 256:
 		return ContextChange{}, errors.New("a context of at least 256 tokens")
+	case req.MaxOutput > req.Tokens:
+		return ContextChange{}, fmt.Errorf("an answer of %d tokens does not fit a context of %d", req.MaxOutput, req.Tokens)
 	case req.Slots < 0 || req.Slots > 64:
 		return ContextChange{}, errors.New("slots are 1 to 64")
 	}
@@ -100,6 +114,9 @@ func (s *Server) SetContext(ctx context.Context, name string, req ContextRequest
 		out.MaxOutput = maxOutput(req.Tokens)
 	}
 	out.MaxOutput = min(out.MaxOutput, req.Tokens)
+	if req.MaxOutput > 0 {
+		out.MaxOutput = req.MaxOutput
+	}
 	if _, err := s.deps.Models.SetContext(ctx, name, req.Tokens, out.MaxOutput, profiles, by); err != nil {
 		return ContextChange{}, err
 	}
@@ -121,6 +138,21 @@ func (s *Server) SetContext(ctx context.Context, name string, req ContextRequest
 	return out, nil
 }
 
+// setMaxOutput changes only the longest answer: the catalog's, which the
+// router holds a request's max_tokens to. No engine reads it, so no replica
+// is started again.
+func (s *Server) setMaxOutput(ctx context.Context, m catalog.Model, maxOut int, by string) (ContextChange, error) {
+	if m.ContextLength > 0 && maxOut > m.ContextLength {
+		return ContextChange{}, fmt.Errorf("an answer of %d tokens does not fit %s's context of %d: change the context with it",
+			maxOut, m.Name, m.ContextLength)
+	}
+	if _, err := s.deps.Models.SetContext(ctx, m.Name, m.ContextLength, maxOut, m.Profiles, by); err != nil {
+		return ContextChange{}, err
+	}
+	return ContextChange{Model: m.Name, Before: m.ContextLength, After: m.ContextLength, MaxOutput: maxOut,
+		OutputOnly: true, MaxOutputBefore: m.MaxOutput}, nil
+}
+
 // shapeOf reads what sizes a model's context: a GGUF file's header from the
 // mirror, else the original weights' config from their repository.
 func (s *Server) shapeOf(ctx context.Context, m catalog.Model) (shape, string) {
@@ -174,6 +206,13 @@ func resizeProfile(p catalog.Profile, req ContextRequest, sh shape) (catalog.Pro
 	case "sglang":
 		pool = req.Tokens
 		p.Args = setArg(p.Args, "--context-length", strconv.Itoa(pool))
+	case "strata":
+		// Strata's setup takes the context, and writes the engine's start
+		// script with it at every start; it sizes its KV cache and expert
+		// cache to the card itself, so there is no card check to make here.
+		p.Args = setArg(p.Args, "--context", strconv.Itoa(req.Tokens))
+		change.Args = p.Args
+		return p, change, nil
 	case "freetoken":
 		// Its experts live in RAM and stream to the card, so the weights
 		// are not what has to fit there, and the card check below would

internal/control/modelserve/context_test.go

@@ -88,3 +88,57 @@ func TestEngineArgumentsAreSetInEitherSpelling(t *testing.T) {
 		t.Error("a comfyui profile took a context")
 	}
 }
+
+func TestTheLongestAnswerAloneChangesOnlyTheCatalogAndStartsNothing(t *testing.T) {
+	s, _, models, _, fleet := updSetup(t)
+	m := models.models["qwen"]
+	m.ContextLength, m.MaxOutput = 4096, 1024
+	m.Profiles[0].Args = []string{"--ctx-size", "4096"}
+	models.models["qwen"] = m
+	before := fleet.Replicas()
+
+	got, err := s.SetContext(context.Background(), "qwen", ContextRequest{MaxOutput: 3000}, "admin")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !got.OutputOnly || got.MaxOutput != 3000 || got.MaxOutputBefore != 1024 || got.Restarted != 0 {
+		t.Fatalf("%+v", got)
+	}
+	after := models.models["qwen"]
+	if after.ContextLength != 4096 || after.MaxOutput != 3000 || !slices.Equal(after.Profiles[0].Args, []string{"--ctx-size", "4096"}) {
+		t.Errorf("catalog %d/%d, args %v", after.ContextLength, after.MaxOutput, after.Profiles[0].Args)
+	}
+	for i, r := range fleet.Replicas() {
+		if r.ID != before[i].ID {
+			t.Errorf("%s was started again for a limit no engine reads", before[i].ID)
+		}
+	}
+	// An answer longer than the context is refused, alone or with one.
+	if _, err := s.SetContext(context.Background(), "qwen", ContextRequest{MaxOutput: 5000}, "admin"); err == nil {
+		t.Error("an answer longer than the context was accepted")
+	}
+	if _, err := s.SetContext(context.Background(), "qwen", ContextRequest{Tokens: 8192, MaxOutput: 9000}, "admin"); err == nil {
+		t.Error("an answer longer than the new context was accepted")
+	}
+	// With a context, the one asked for wins over the one that follows it.
+	got, err = s.SetContext(context.Background(), "qwen", ContextRequest{Tokens: 8192, MaxOutput: 6000}, "admin")
+	if err != nil || got.MaxOutput != 6000 || models.models["qwen"].MaxOutput != 6000 {
+		t.Errorf("%+v %v", got, err)
+	}
+}
+
+func TestStrataTakesItsContextInItsSetupArguments(t *testing.T) {
+	p := catalog.Profile{Name: "one-card", Engine: "strata", GPUClass: "6000-24g",
+		Args: []string{"--family", "qwen", "--size", "IQ2_XS"}}
+	got, change, err := resizeProfile(p, ContextRequest{Tokens: 32768}, shape{kvPerToken: 1 << 20, weights: 40 << 30})
+	if err != nil {
+		t.Fatal(err)
+	}
+	// Not refused for weights larger than the card: its experts live in RAM.
+	if !slices.Equal(got.Args, []string{"--family", "qwen", "--size", "IQ2_XS", "--context", "32768"}) || !slices.Equal(change.Args, got.Args) {
+		t.Errorf("args %v", got.Args)
+	}
+	if n := contextOf(catalog.Model{ContextLength: 4096}, got); n != 32768 {
+		t.Errorf("contextOf %d", n)
+	}
+}

internal/control/modelserve/gpus.go

@@ -208,6 +208,10 @@ func contextOf(m catalog.Model, p catalog.Profile) int {
 		if n := argInt(p.Args, "--context-length"); n > 0 {
 			return n
 		}
+	case "strata":
+		if n := argInt(p.Args, "--context"); n > 0 {
+			return n
+		}
 	case "freetoken":
 		if n := argInt(p.Args, "--max-seq-len-override"); n > 0 {
 			return n