Add a node with one command: llmgw node add, with a terminal screen
Adding a node was deploy/bootstrap/onboard-node.sh, which is not installed
anywhere and wanted the node's name, the control plane's address and a
dozen flags. It is now in the binary, run on the control plane:
sudo llmgw node add ubuntu@10.0.0.21
The node id is the machine's hostname, and the address its agent dials is
this machine's on the route to it, checked against the server certificate.
It installs llmgw there (the release for its architecture, fetched and
checked by the node; uname rather than dpkg, so RHEL works), runs doctor
there and stops on a problem before changing anything, issues and installs
the certificate, writes agent.yaml and the unit, starts the agent, and waits
for it to join and install llama.cpp. The authority is read first, so a
missing one is said before the node is touched.
The steps are events. In a terminal they draw a checklist (bubbletea) that
asks what a script cannot: which fabric each group of active adapters is on,
grouped by the subnet manager or RoCE network they report, and whether to go
on past a problem. Elsewhere, one line per step.
Found by running it on an empty VM: ssh's "Permanently added" warning on
stderr was read as the machine's hostname, so standard error is now kept
apart; and a node added again while cordoned never counted as joined, so
joining now means a heartbeat since the restart.
llmgw node remove forgets a lost node, which otherwise stays listed until
the control plane restarts. ctl nodes no longer lists vLLM as skipped on
every CPU node. The quick install, install and onboarding docs use it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
c8da06988f
commit
2a16c26575
17 files changed
+2056
-120
No files matched your search
+30
-26
@@ -488,40 +488,44 @@ bf16 is 141 GB and the same model at q4 is 43 GB.
|
||||
|
||||
## 5. Adding the node
|
||||
|
||||
Two halves, because they need different access: building the machine needs the
|
||||
hypervisor, and issuing its identity needs the certificate authority. Where one
|
||||
person holds both — the common case — one command does it:
|
||||
On the control plane, which holds the certificate authority:
|
||||
|
||||
```
|
||||
llmgw-add-node \
|
||||
--name llmgw-node01 --ip 10.0.0.141/24 \
|
||||
--server https://10.0.0.140:7443 \
|
||||
--cores 16 --memory 131072 --disk 200 \
|
||||
--fabric ib-a:mlx5_0,mlx5_1 \
|
||||
--llamacpp-bin /opt/llama.cpp/llama-server
|
||||
sudo llmgw node add ubuntu@10.0.0.141 --fabric ib-a:mlx5_0,mlx5_1
|
||||
```
|
||||
|
||||
**The VM name, the hostname, the node id and the certificate's common name are
|
||||
all the same string.** They do not have to be, and every time they are not,
|
||||
somebody spends an afternoon working out which of the four a message refers to.
|
||||
It logs in to the machine over SSH as that user, who needs passwordless sudo
|
||||
there, and:
|
||||
|
||||
For a machine that already exists — bare metal, or a VM you built yourself — skip
|
||||
provisioning and issue the identity only:
|
||||
1. installs llmgw: the release for the machine's architecture, downloaded by
|
||||
the machine and checked against the release's checksums, or this machine's
|
||||
copy when it cannot download;
|
||||
2. runs `llmgw doctor` there and stops on any problem, printing the fix. Run it
|
||||
again once fixed, or add `--ignore-checks`;
|
||||
3. issues the node's certificate and installs it, with the cluster's root,
|
||||
trusted system-wide;
|
||||
4. writes `agent.yaml` — only what the node cannot find for itself — and the
|
||||
systemd unit, and starts the agent;
|
||||
5. waits for the node to join and to install llama.cpp (`--wait` also waits
|
||||
for vLLM, `--no-wait` for neither).
|
||||
|
||||
```
|
||||
llmgw-onboard-node \
|
||||
--node llmgw-node01 --addr 10.0.0.141 \
|
||||
--server https://10.0.0.140:7443 \
|
||||
--fabric ib-a:mlx5_0,mlx5_1 \
|
||||
--llamacpp-bin /opt/llama.cpp/llama-server
|
||||
```
|
||||
In a terminal it draws this as a checklist and asks what a script cannot: which
|
||||
fabric each group of the node's active adapters is on, when `--fabric` was not
|
||||
given, and whether to go on past a problem doctor found.
|
||||
|
||||
Onboarding needs SSH from the CA host to the node. It waits for SSH rather than
|
||||
failing on it, and learns the host key of a machine it has not seen — a new
|
||||
machine spends its first minute regenerating host keys and restarting sshd.
|
||||
**The node id is the machine's hostname**, and the address its agent dials is
|
||||
this machine's on the route to it; `--node` and `--server` override them. The
|
||||
VM name, the hostname, the node id and the certificate's common name being the
|
||||
same string saves an afternoon each time they would differ.
|
||||
|
||||
It also checks that a configured engine actually runs, and says so loudly when
|
||||
it does not. That check exists because the failure is otherwise invisible.
|
||||
**SSH runs as root under sudo**, with root's keys. To use yours, log in to the
|
||||
control plane with `ssh -A` and run `sudo --preserve-env=SSH_AUTH_SOCK llmgw node
|
||||
add …`. It waits for SSH rather than failing on it, and learns the host key of
|
||||
a machine it has not seen — a new machine spends its first minute regenerating
|
||||
host keys and restarting sshd.
|
||||
|
||||
Running it again on a node is safe: a fresh certificate and configuration, and
|
||||
the engines it runs are adopted by the restarted agent.
|
||||
|
||||
Within a few seconds:
|
||||
|
||||
|
||||
Reference in new issue
Block a user