A model can be stopped and kept, or removed with its files from the mirror and the nodes
CI / test (push) Successful in 10m37s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-29 20:45:48 +01:00
1 parent 5726ceb84e
commit 59187cdae5
19 files changed
+760 -14

No files matched your search

+24
View File
@@ -1629,6 +1629,30 @@ Not yet: models declared in server.yaml cannot name a peer's model as a
fallback (the catalog in the file is checked before the peers are read), and
a peer behind a private CA needs that CA in the system's trust store.
### Stopping a model, and removing one with its files
```
llmgw ctl model stop qwen-7b # none of its replicas run; it is kept
llmgw ctl model start qwen-7b # runs again, scaled as it was
llmgw ctl model remove qwen-7b # leaves the catalog; its files stay
llmgw ctl model remove qwen-7b --purge # its files on the mirror and the nodes too
```
- **Stop** keeps the model in the catalog with its files and what it was
scaled to; a request for it is told an administrator stopped it. `ctl
models` shows it as stopped. A `ctl scale` while it is stopped starts it at
that number.
- **Remove** leaves its files on the mirror and the nodes, for adding it
again without fetching it. `ctl weights` lists such files on the nodes.
- **Remove --purge** deletes every version of it on the mirror (a second
mirror follows), its adapters and the weights its last update replaced,
and the nodes' copies once its replicas have stopped. What another model
uses — a canary's weights, a shared adapter — stays. It cannot be undone:
adding it again fetches it again.
The console has the same in a model's menu: Stop… or Start, Remove…, and
Remove with its files….
### Changing a served model's context
How much one request may use is set when a model is served. To change it