There was no way to restart a running replica. A running engine picks up an
engine key, an edited profile or a new engine version only when it starts,
so after v0.10.3 the nine replicas already running stayed without a key —
reachable from any node — with nothing to do about it short of draining
nodes, which on a small CPU fleet has nowhere to move a model to.
`llmgw ctl restart REPLICA... | --model M | --all` queues replicas to be
replaced. A replica being replaced keeps serving but stops counting toward
its model, so the reconciler places a replacement beside it; once that is
ready the old one is drained and stopped as a node drain would. Replicas of
one model go one at a time, so a model never needs the memory of every
replica twice. Where no second copy fits anywhere, the old replica is
stopped after the drain grace period and the new one starts in its place,
and the log says so, since that is the one case with an outage.
`ctl replicas` shows "(restart queued)" and "(restart replacing)" while it
happens.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
