CI went red on Linux with two failures that never appear on macOS. Both were
worth the trip.
The first is a real bug. processAlive used kill(pid, 0), which succeeds on a
zombie — a process that has exited and is only waiting to be reaped. That
happens for real whenever an engine's parent dies before it does and nothing
reaps the orphan, which in a container is the normal case: PID 1 is usually the
workload rather than an init. The supervisor would then report a dead adopted
engine as running forever, and hold its GPUs against every future placement.
Zombies are now detected through procfs on Linux, where the answer exists, and
the check degrades to its old behaviour elsewhere rather than to something worse.
The second was a bad test. TestTenantFairnessWithinBand asserted on the order
goroutines sent to a channel after being admitted, which is the order the Go
scheduler woke them, not the order the dispatcher chose. Under load it measured
the runtime. The rotation itself lives in bandQueue.pop, which is pure, so it is
tested there — including what happens when a tenant runs out mid-rotation and
when a cancelled waiter is removed. The controller-level test now asserts the
thing that is actually stable: nobody is starved.
Verified with three consecutive `go test -race ./...` runs in a Linux container.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Everything here works without GPUs and is covered by tests. The control plane
connection is not built yet, so llmgw agent itself is still a stub and the phase
exit criterion is not met.
- discovery: IB from /sys/class/infiniband, parsing the formats sysfs actually
uses and telling native InfiniBand from RoCE; GPUs via nvidia-smi, the
cgo-free path, with health flags for uncorrected ECC and memory already held
by another process. A node without IB is not an error, it is a single-node
island.
- engines: vLLM and SGLang command and environment construction, kept free of
process handling so it stays testable. The NCCL environment is the part the
plan marks critical: NCCL_IB_HCA lists only the chosen fabric's adapters,
because leaving it open lets NCCL pick an adapter on a fabric where the peer
is unreachable, and the job then hangs instead of failing.
- validation before launch: no mixed GPU types in a tensor-parallel group, the
node/GPU/TP arithmetic has to add up, and fp8 is refused below compute
capability 8.9 rather than silently falling back to a slower path.
- supervisor: process groups, because engines fork workers that hold GPU memory
and outlive a signal sent only to the leader; a bounded log tail, since engine
output is the only diagnosis when loading fails; Starting separate from
Loading, because a 70B model legitimately takes minutes; and orphan adoption
through pid files fingerprinted with the command line, since pids get reused
and adopting an unrelated process is worse than leaking the original.
- readiness is two questions: /health says the process is up, /v1/models says
the weights loaded. Routing on /health alone sends traffic into a loading model.
- llmgw agent inventory reports what the scheduler would see on a node.
One plan correction: GPU class sizes come from the driver in GiB, which does not
always match the marketing name. An L40S is sold as 48GB and reports 44.99 GiB,
so its class is l40s-45g, not l40s-48g. Consistency matters more than matching
the box, because that number decides whether a model fits. Plan §2 and §5.1
updated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ilkerandClaude Opus 5