Change a model's replica count without editing a file on the control plane
The count is scaling.min, and until now the only way to change it was to edit server.yaml on the host and restart — or, for a model added at runtime, to remove it and add it back, which means having its whole definition to hand. `llmgw ctl scale <model> --min N` instead. Only the scaling block moves: everything else is read back from what was stored and written out untouched, so changing a number cannot quietly become a redefinition. The catalog is rebuilt and validated before anything is written, as in Add. A max below a min is refused with the stored definition untouched — persisting a rejected change would come back on the next restart as a catalog that does not build. The answer reports the scaling in force rather than the request. The catalog fills in a max that was left out, and a caller echoing its own request would print a number the cluster is not using. A model declared in configuration is refused, for the reason Remove already refuses one: the YAML is read on every start, so the new count would hold until the next restart and then silently revert. A cluster disagreeing with its own config file is worse than an error saying where to edit. The help says what --max does, because it is not what it looks like: a model with a minimum runs exactly that many and nothing scales it up under load, and an on-demand model loads one replica however high its max. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
10e323132c
commit
707d774c76
6 files changed
+388
No files matched your search
@@ -124,6 +124,7 @@ func newCtlCmd() *cobra.Command {
|
||||
newBenchCmd(&asJSON),
|
||||
newStatusCmd(client, &asJSON),
|
||||
newModelCmd(client),
|
||||
newScaleCmd(client),
|
||||
newEngineCheckCmd(),
|
||||
newToolCheckCmd(),
|
||||
newWeightsCmd(client, &asJSON),
|
||||
|
||||
Reference in new issue
Block a user