llmgw stop, start and restart act on the whole system

Issue #3 asked for commands to stop and start the system, and 05aa21c gave
only one machine's services. Now:

- llmgw stop sets serving.stopped in the cluster's settings, live: the
  reconciler wants every model at no replicas, freeing every GPU, and the
  gateway answers every POST under /v1/ with 503 system_stopped. The control
  planes, agents and mirror keep running, and no model's definition changes.
- llmgw start clears it; every model is placed again as it asks.
- llmgw restart has every agent restart over its stream (RestartAgent), then
  the other control planes over ssh, standbys before a leader, then this
  one; engines keep serving.
- llmgw status is the cluster's summary, saying when it is stopped.

One machine's services moved to llmgw service status|start|stop|restart.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-07 18:01:45 +02:00
1 parent 05aa21c4d4
commit 7245dbe1dd
29 files changed
+891 -254

No files matched your search

+31 -9
View File
@@ -2294,20 +2294,42 @@ A cluster set up online becomes offline the same way: `bundle import` on the
mirror and control plane, then the `offline:` block in `server.yaml` and a
restart.
## Stopping and starting a machine
## Stopping and starting the system
From a control plane:
```
sudo llmgw status # each llmgw service here, and whether it runs
sudo llmgw stop # agent, control plane, mirror — whichever this machine has
sudo llmgw start
sudo llmgw restart agent # or name one: server, agent, mirror
llmgw status # does it serve; its control planes, nodes, models, store
llmgw stop --note "maintenance" # every model taken down, every request answered 503
llmgw start # every model back as its catalog entry asks
sudo llmgw restart # every agent and control plane started again, engines serving throughout
```
`llmgw stop` frees every GPU and answers requests for a model with 503
(`system_stopped`, with Retry-After) until `llmgw start`. The control planes,
agents and mirror keep running, so the console, the admin pages and `llmgw
start` keep working, and no model's definition changes. It is the cluster's
setting `serving.stopped`, so a restart does not undo it, and it is in the
audit with its note. `--wait` waits until every replica is down, or every
model has its replicas again.
`llmgw restart` restarts every node's agent, then the other control planes
over ssh (`--ssh-user`, the one sudo was run by), then this one — the leader
last, so it hands over to a standby already restarted. Engines keep serving
and are taken over again; nothing is placed anew.
One machine's own services are `llmgw service`:
```
sudo llmgw service status
sudo llmgw service stop # agent, control plane, mirror — whichever this machine has
sudo llmgw service start
sudo llmgw service restart agent # or name one: server, agent, mirror
```
Stopping a node's agent leaves its engines serving — the agent takes them
over again when it starts — unless `llmgw stop --engines` stops them too, and
the control plane then places their replicas elsewhere. Stopping the leading
control plane hands over to a standby within seconds. The whole cluster's
state is `llmgw ctl status`.
over again when it starts — unless `llmgw service stop --engines` stops them
too, and the control plane then places their replicas elsewhere.
## Removing llmgw from a machine