Ship binaries with releases, and ask engines whether our flags still exist

A release was a git tag. Installing llmgw on a node meant having a Go
toolchain, and the onboarding script copied a binary from wherever it was run —
which works and quietly makes whoever runs it the build server, with the fleet
ending up on whatever was in their working tree. Tags now build static
linux/amd64, linux/arm64 and darwin binaries and attach them. CGO_ENABLED=0 is
the point: it has to run on a node whose libc is older than the builder's, and
llmgw has no C dependency because modernc.org/sqlite is pure Go precisely so
this stays true. Verified: "not a dynamic executable", runs on a node, reports
its version.

`ctl engine-check` closes the other half of the phase 4 gap. llmgw's launch
arguments are read from each engine's documentation, and engines rename flags
between versions; when that happens the failure arrives minutes into a model
load, in the engine's own error message, on a node nobody was watching. It
builds the widest command llmgw would ever produce — multi-node, since a flag
that only appears there is the one nobody notices has gone — and checks each
flag against the engine's own --help. No GPU: a CLI prints its help anywhere.
Verified against llama.cpp b10991, all seven flags accepted.

Flags are matched on a word boundary, so --port is not satisfied by
--port-range. Without that, a renamed flag would sail through the check that
exists to catch renamed flags.

It also warns when the help text looks truncated, which is how the first run
went: llama.cpp's help cut to 400 of its 725 lines reported five flags as
missing that the running cluster uses.

TestSidecarRunsBeforeEngine was flaky under a full run, and worse than flaky —
its wait loop fell through on timeout, so a slow machine turned the ordering
assertion into a no-op and the failure surfaced later as something else. It now
asserts, with a deadline that suits a machine running every other package at
the same time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 16:44:29 +02:00
1 parent 7b4acd9284
commit 5b425c32e5
8 files changed
+407 -13

No files matched your search

+27
View File
@@ -21,6 +21,12 @@ It imports a cloud image and lets cloud-init do the rest, which is why a node
takes a minute rather than an afternoon. It refuses to touch an existing VMID:
adopting a machine is not a decision a provisioning script should make.
Releases carry static `linux/amd64` and `linux/arm64` binaries. Download one and
pass it with `--binary` rather than copying whatever happens to be on the
machine you are running onboarding from — otherwise whoever runs the script
quietly becomes the build server, and the fleet ends up running whatever was in
their working tree.
## 2. The identity
On the host that holds the certificate authority — this is the part that cannot
@@ -70,6 +76,27 @@ a laptop. llmgw has no answer for it yet; that is a known gap, not an oversight.
The `sim` engine needs neither, which is what makes it useful for exercising
everything else.
## Checking an engine after an upgrade
Engines rename and remove flags between versions, and when that happens the
failure arrives minutes into a model load, in the engine's own error message, on
a node nobody was watching.
```
llmgw ctl engine-check vllm
llmgw ctl engine-check llamacpp --help-output "$(llama-server --help 2>&1)"
```
It builds the widest command llmgw would ever produce — a multi-node one, since
a flag that only appears there is exactly the one nobody notices has gone — and
checks every flag against the engine's own `--help`. No GPU required: a CLI
prints its help on any machine.
Give it the **whole** help. Truncating llama.cpp's 725 lines to 400 reports five
flags as missing that the running cluster uses; the command warns when the text
it was given does not document `--help` itself, which is the cheapest sign it
was cut short.
## Fabrics
`--fabric` is how a node says which InfiniBand fabric it is attached to, and