Ship binaries with releases, and ask engines whether our flags still exist
CI / test (push) Successful in 1m31s
CI / test (push) Successful in 1m31s
A release was a git tag. Installing llmgw on a node meant having a Go toolchain, and the onboarding script copied a binary from wherever it was run — which works and quietly makes whoever runs it the build server, with the fleet ending up on whatever was in their working tree. Tags now build static linux/amd64, linux/arm64 and darwin binaries and attach them. CGO_ENABLED=0 is the point: it has to run on a node whose libc is older than the builder's, and llmgw has no C dependency because modernc.org/sqlite is pure Go precisely so this stays true. Verified: "not a dynamic executable", runs on a node, reports its version. `ctl engine-check` closes the other half of the phase 4 gap. llmgw's launch arguments are read from each engine's documentation, and engines rename flags between versions; when that happens the failure arrives minutes into a model load, in the engine's own error message, on a node nobody was watching. It builds the widest command llmgw would ever produce — multi-node, since a flag that only appears there is the one nobody notices has gone — and checks each flag against the engine's own --help. No GPU: a CLI prints its help anywhere. Verified against llama.cpp b10991, all seven flags accepted. Flags are matched on a word boundary, so --port is not satisfied by --port-range. Without that, a renamed flag would sail through the check that exists to catch renamed flags. It also warns when the help text looks truncated, which is how the first run went: llama.cpp's help cut to 400 of its 725 lines reported five flags as missing that the running cluster uses. TestSidecarRunsBeforeEngine was flaky under a full run, and worse than flaky — its wait loop fell through on timeout, so a slow machine turned the ordering assertion into a no-op and the failure surfaced later as something else. It now asserts, with a deadline that suits a machine running every other package at the same time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
7b4acd9284
commit
5b425c32e5
8 files changed
+407
-13
No files matched your search
+8
-3
@@ -1407,9 +1407,12 @@ The launch path is now built too:
|
||||
that NCCL picks the right HCA, or that the generated environment makes a
|
||||
real collective run. Those need hardware. The GPUs here are an inventory,
|
||||
the fabric is a sysfs-shaped directory, and the engine speaks no NCCL.
|
||||
- [ ] **Confirm the engine flags on real hardware.** The launch *sequence* is
|
||||
now exercised; the launch *arguments* for vLLM and SGLang are still read
|
||||
from their documentation rather than from a running engine.
|
||||
- [x] `ctl engine-check`, which asks an engine whether it still accepts the
|
||||
flags llmgw builds. No GPU needed — a CLI prints its help anywhere.
|
||||
Verified against llama.cpp b10991: all seven flags accepted.
|
||||
- [ ] **Run `engine-check` against vLLM and SGLang.** The launch *sequence* is
|
||||
exercised on the simulated cluster and the *arguments* now have a way to
|
||||
be checked; what is missing is an installed engine to check them against.
|
||||
|
||||
- **Exit criterion (met on the simulated cluster):** a 2-node replica on each
|
||||
fabric; killing a node rebuilds the replica within one fabric and never
|
||||
@@ -1457,6 +1460,8 @@ The launch path is now built too:
|
||||
rather than printed
|
||||
- [ ] OTel tracing
|
||||
- [x] systemd units and `docs/onboarding.md`
|
||||
- [x] Release binaries: static linux/amd64, linux/arm64 and darwin builds
|
||||
attached to each tag, so installing on a node does not need a Go toolchain
|
||||
- [ ] Reference documentation beyond the plan itself
|
||||
|
||||
### Phase 6 — v2 ideas
|
||||
|
||||
Reference in new issue
Block a user