Files
llmgw/internal
Ilker ManapandClaude Opus 5 e05d5d48ee
CI / test (push) Has been cancelled
Do not place on a fleet that has not reported in yet
A control plane that has just started knows nothing, and its first
reconcile tick runs before any agent has registered. "needs 2 nodes on one
fabric, but no node reports a fabric" in the log is that, and it looked
harmless because adoption catches up.

What is not harmless is placing on the fraction of the fleet that happened
to report first. After one restart both replicas of qwen2.5-0.5b landed on
node04 — the only node registered at that instant — when four were coming,
and nothing ever moved them, because the desired count was met.

Placement now waits, and leaves by either of two doors, because either
alone is wrong. Waiting for the window to expire would cost that much
downtime on every restart even when the whole fleet reconnected in a
second; waiting only for the node count to stop changing would hang forever
on a cluster whose nodes never all come back. So: the count holds steady for
a quarter of the window, or the window expires.

Only placement waits. Node expiry, stuck-replica expiry, draining and
reaping all stay correct on a partial fleet, and adoption in particular must
not wait — it is how a restart recovers what is already running, and
delaying it would leave live engines unaccounted for exactly while the
reconciler decides what to build.

The window is paid once per process. A node joining a running cluster is an
ordinary event, not a reason to stop placing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 23:18:29 +02:00
..