A replica missing a shard is not ready
CI / test (push) Successful in 1m23s

Restarting the control plane over a running two-node replica adopted half of
it. Every shard it still had was healthy, so it was reported Ready — with one
shard of two and no endpoint at all, because the missing one was rank 0. The
router was handed a replica nothing could route to.

The cause is that a replica had no idea how many shards it should have. It
could tell "every shard I have is ready" and nothing else, which is the same
sentence whether or not half of it is gone. It now carries the count from its
profile, set at placement and again at adoption, and a replica short of shards
fails rather than pretending.

Onboarding also wrote no engine paths, so a node onboarded by the script could
not find llama-server and failed every shard placed on it with "executable
file not found". The script now takes them, and says so when none are given —
a node with no engine paths can run the sim engine and nothing else.

The cluster now runs both at once: two-node simulated replicas on each fabric,
and real llama.cpp inference on the CPU, which is a better test bed than
either alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 12:43:51 +02:00
1 parent c97b671ef9
commit 7b4acd9284
5 files changed
+133 -1

No files matched your search

+21
View File
@@ -49,6 +49,27 @@ what makes a node's identity something it proves rather than something it picks.
`authorized_keys` into the new machine, so a node built by the script above
already satisfies it; an older machine may need the key added.
## What onboarding does not do
It gives a machine an identity and an agent. It does **not** install inference
engines or model weights, and a node missing either looks healthy in
`llmgw ctl nodes` right up until a replica is placed on it and the shard dies
with "executable file not found".
So the node is not finished until:
- the engines its catalog profiles name are installed (`vllm`, `sglang`,
`llama-server`), at the paths the agent configuration gives
- the weights are on disk, at the `source` each model declares
Both are deliberately outside this script. Engines are a platform decision with
versions and drivers attached, and distributing weights is its own problem —
hundreds of gigabytes that want a shared filesystem or a cache, not an scp from
a laptop. llmgw has no answer for it yet; that is a known gap, not an oversight.
The `sim` engine needs neither, which is what makes it useful for exercising
everything else.
## Fabrics
`--fabric` is how a node says which InfiniBand fabric it is attached to, and