Restart replicas without taking them out of service first
There was no way to restart a running replica. A running engine picks up an engine key, an edited profile or a new engine version only when it starts, so after v0.10.3 the nine replicas already running stayed without a key — reachable from any node — with nothing to do about it short of draining nodes, which on a small CPU fleet has nowhere to move a model to. `llmgw ctl restart REPLICA... | --model M | --all` queues replicas to be replaced. A replica being replaced keeps serving but stops counting toward its model, so the reconciler places a replacement beside it; once that is ready the old one is drained and stopped as a node drain would. Replicas of one model go one at a time, so a model never needs the memory of every replica twice. Where no second copy fits anywhere, the old replica is stopped after the drain grace period and the new one starts in its place, and the log says so, since that is the one case with an outage. `ctl replicas` shows "(restart queued)" and "(restart replacing)" while it happens. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
611fddd236
commit
ad6d2d348d
6 files changed
+325
-3
No files matched your search
@@ -135,6 +135,7 @@ func newCtlCmd() *cobra.Command {
|
||||
newStatusCmd(client, &asJSON),
|
||||
newModelCmd(client),
|
||||
newScaleCmd(client),
|
||||
newRestartCmd(client),
|
||||
newEngineCheckCmd(),
|
||||
newToolCheckCmd(),
|
||||
newWeightsCmd(client, &asJSON),
|
||||
@@ -370,6 +371,9 @@ func newReplicasCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
|
||||
if r.Stale {
|
||||
state += " (stale profile)"
|
||||
}
|
||||
if r.Restart != "" {
|
||||
state += " (restart " + r.Restart + ")"
|
||||
}
|
||||
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n",
|
||||
r.ID, r.Model, state, r.Pool, join(nodes),
|
||||
(time.Duration(r.AgeSec) * time.Second).String(),
|
||||
|
||||
Reference in new issue
Block a user