Add a node with one command: llmgw node add, with a terminal screen

Adding a node was deploy/bootstrap/onboard-node.sh, which is not installed
anywhere and wanted the node's name, the control plane's address and a
dozen flags. It is now in the binary, run on the control plane:

    sudo llmgw node add ubuntu@10.0.0.21

The node id is the machine's hostname, and the address its agent dials is
this machine's on the route to it, checked against the server certificate.
It installs llmgw there (the release for its architecture, fetched and
checked by the node; uname rather than dpkg, so RHEL works), runs doctor
there and stops on a problem before changing anything, issues and installs
the certificate, writes agent.yaml and the unit, starts the agent, and waits
for it to join and install llama.cpp. The authority is read first, so a
missing one is said before the node is touched.

The steps are events. In a terminal they draw a checklist (bubbletea) that
asks what a script cannot: which fabric each group of active adapters is on,
grouped by the subnet manager or RoCE network they report, and whether to go
on past a problem. Elsewhere, one line per step.

Found by running it on an empty VM: ssh's "Permanently added" warning on
stderr was read as the machine's hostname, so standard error is now kept
apart; and a node added again while cordoned never counted as joined, so
joining now means a heartbeat since the restart.

llmgw node remove forgets a lost node, which otherwise stays listed until
the control plane restarts. ctl nodes no longer lists vLLM as skipped on
every CPU node. The quick install, install and onboarding docs use it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-29 19:33:47 +02:00
1 parent c8da06988f
commit 2a16c26575
17 files changed
+2056 -120

No files matched your search

+30 -26
View File
@@ -488,40 +488,44 @@ bf16 is 141 GB and the same model at q4 is 43 GB.
## 5. Adding the node
Two halves, because they need different access: building the machine needs the
hypervisor, and issuing its identity needs the certificate authority. Where one
person holds both — the common case — one command does it:
On the control plane, which holds the certificate authority:
```
llmgw-add-node \
--name llmgw-node01 --ip 10.0.0.141/24 \
--server https://10.0.0.140:7443 \
--cores 16 --memory 131072 --disk 200 \
--fabric ib-a:mlx5_0,mlx5_1 \
--llamacpp-bin /opt/llama.cpp/llama-server
sudo llmgw node add ubuntu@10.0.0.141 --fabric ib-a:mlx5_0,mlx5_1
```
**The VM name, the hostname, the node id and the certificate's common name are
all the same string.** They do not have to be, and every time they are not,
somebody spends an afternoon working out which of the four a message refers to.
It logs in to the machine over SSH as that user, who needs passwordless sudo
there, and:
For a machine that already exists — bare metal, or a VM you built yourself — skip
provisioning and issue the identity only:
1. installs llmgw: the release for the machine's architecture, downloaded by
the machine and checked against the release's checksums, or this machine's
copy when it cannot download;
2. runs `llmgw doctor` there and stops on any problem, printing the fix. Run it
again once fixed, or add `--ignore-checks`;
3. issues the node's certificate and installs it, with the cluster's root,
trusted system-wide;
4. writes `agent.yaml` — only what the node cannot find for itself — and the
systemd unit, and starts the agent;
5. waits for the node to join and to install llama.cpp (`--wait` also waits
for vLLM, `--no-wait` for neither).
```
llmgw-onboard-node \
--node llmgw-node01 --addr 10.0.0.141 \
--server https://10.0.0.140:7443 \
--fabric ib-a:mlx5_0,mlx5_1 \
--llamacpp-bin /opt/llama.cpp/llama-server
```
In a terminal it draws this as a checklist and asks what a script cannot: which
fabric each group of the node's active adapters is on, when `--fabric` was not
given, and whether to go on past a problem doctor found.
Onboarding needs SSH from the CA host to the node. It waits for SSH rather than
failing on it, and learns the host key of a machine it has not seen — a new
machine spends its first minute regenerating host keys and restarting sshd.
**The node id is the machine's hostname**, and the address its agent dials is
this machine's on the route to it; `--node` and `--server` override them. The
VM name, the hostname, the node id and the certificate's common name being the
same string saves an afternoon each time they would differ.
It also checks that a configured engine actually runs, and says so loudly when
it does not. That check exists because the failure is otherwise invisible.
**SSH runs as root under sudo**, with root's keys. To use yours, log in to the
control plane with `ssh -A` and run `sudo --preserve-env=SSH_AUTH_SOCK llmgw node
add …`. It waits for SSH rather than failing on it, and learns the host key of
a machine it has not seen — a new machine spends its first minute regenerating
host keys and restarting sshd.
Running it again on a node is safe: a fresh certificate and configuration, and
the engines it runs are adopted by the restarted agent.
Within a few seconds: