llmgw stop, start and restart act on the whole system
Issue #3 asked for commands to stop and start the system, and 05aa21c gave
only one machine's services. Now:
- llmgw stop sets serving.stopped in the cluster's settings, live: the
reconciler wants every model at no replicas, freeing every GPU, and the
gateway answers every POST under /v1/ with 503 system_stopped. The control
planes, agents and mirror keep running, and no model's definition changes.
- llmgw start clears it; every model is placed again as it asks.
- llmgw restart has every agent restart over its stream (RestartAgent), then
the other control planes over ssh, standbys before a leader, then this
one; engines keep serving.
- llmgw status is the cluster's summary, saying when it is stopped.
One machine's services moved to llmgw service status|start|stop|restart.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
05aa21c4d4
commit
7245dbe1dd
29 files changed
+891
-254
No files matched your search
+31
-9
@@ -2294,20 +2294,42 @@ A cluster set up online becomes offline the same way: `bundle import` on the
|
||||
mirror and control plane, then the `offline:` block in `server.yaml` and a
|
||||
restart.
|
||||
|
||||
## Stopping and starting a machine
|
||||
## Stopping and starting the system
|
||||
|
||||
From a control plane:
|
||||
|
||||
```
|
||||
sudo llmgw status # each llmgw service here, and whether it runs
|
||||
sudo llmgw stop # agent, control plane, mirror — whichever this machine has
|
||||
sudo llmgw start
|
||||
sudo llmgw restart agent # or name one: server, agent, mirror
|
||||
llmgw status # does it serve; its control planes, nodes, models, store
|
||||
llmgw stop --note "maintenance" # every model taken down, every request answered 503
|
||||
llmgw start # every model back as its catalog entry asks
|
||||
sudo llmgw restart # every agent and control plane started again, engines serving throughout
|
||||
```
|
||||
|
||||
`llmgw stop` frees every GPU and answers requests for a model with 503
|
||||
(`system_stopped`, with Retry-After) until `llmgw start`. The control planes,
|
||||
agents and mirror keep running, so the console, the admin pages and `llmgw
|
||||
start` keep working, and no model's definition changes. It is the cluster's
|
||||
setting `serving.stopped`, so a restart does not undo it, and it is in the
|
||||
audit with its note. `--wait` waits until every replica is down, or every
|
||||
model has its replicas again.
|
||||
|
||||
`llmgw restart` restarts every node's agent, then the other control planes
|
||||
over ssh (`--ssh-user`, the one sudo was run by), then this one — the leader
|
||||
last, so it hands over to a standby already restarted. Engines keep serving
|
||||
and are taken over again; nothing is placed anew.
|
||||
|
||||
One machine's own services are `llmgw service`:
|
||||
|
||||
```
|
||||
sudo llmgw service status
|
||||
sudo llmgw service stop # agent, control plane, mirror — whichever this machine has
|
||||
sudo llmgw service start
|
||||
sudo llmgw service restart agent # or name one: server, agent, mirror
|
||||
```
|
||||
|
||||
Stopping a node's agent leaves its engines serving — the agent takes them
|
||||
over again when it starts — unless `llmgw stop --engines` stops them too, and
|
||||
the control plane then places their replicas elsewhere. Stopping the leading
|
||||
control plane hands over to a standby within seconds. The whole cluster's
|
||||
state is `llmgw ctl status`.
|
||||
over again when it starts — unless `llmgw service stop --engines` stops them
|
||||
too, and the control plane then places their replicas elsewhere.
|
||||
|
||||
## Removing llmgw from a machine
|
||||
|
||||
|
||||
Reference in new issue
Block a user