Add a node with one command: llmgw node add, with a terminal screen
Adding a node was deploy/bootstrap/onboard-node.sh, which is not installed
anywhere and wanted the node's name, the control plane's address and a
dozen flags. It is now in the binary, run on the control plane:
sudo llmgw node add ubuntu@10.0.0.21
The node id is the machine's hostname, and the address its agent dials is
this machine's on the route to it, checked against the server certificate.
It installs llmgw there (the release for its architecture, fetched and
checked by the node; uname rather than dpkg, so RHEL works), runs doctor
there and stops on a problem before changing anything, issues and installs
the certificate, writes agent.yaml and the unit, starts the agent, and waits
for it to join and install llama.cpp. The authority is read first, so a
missing one is said before the node is touched.
The steps are events. In a terminal they draw a checklist (bubbletea) that
asks what a script cannot: which fabric each group of active adapters is on,
grouped by the subnet manager or RoCE network they report, and whether to go
on past a problem. Elsewhere, one line per step.
Found by running it on an empty VM: ssh's "Permanently added" warning on
stderr was read as the machine's hostname, so standard error is now kept
apart; and a node added again while cordoned never counted as joined, so
joining now means a heartbeat since the restart.
llmgw node remove forgets a lost node, which otherwise stays listed until
the control plane restarts. ctl nodes no longer lists vLLM as skipped on
every CPU node. The quick install, install and onboarding docs use it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
c8da06988f
commit
2a16c26575
17 files changed
+2056
-120
No files matched your search
+24
-42
@@ -4,24 +4,16 @@ Written for whoever runs the cluster.
|
||||
|
||||
## The short version
|
||||
|
||||
Where one person holds both the hypervisor and the certificate authority —
|
||||
which is the common case — one command does the whole thing in about
|
||||
45 seconds:
|
||||
On the control plane:
|
||||
|
||||
```
|
||||
llmgw-add-node \
|
||||
--name llmgw-node05 --ip 10.0.0.145/24 \
|
||||
--server https://10.0.0.140:7443 \
|
||||
--cores 8 --memory 32768 --disk 60 \
|
||||
--llamacpp-bin /opt/llama.cpp/llama-server
|
||||
sudo llmgw node add ubuntu@10.0.0.145
|
||||
```
|
||||
|
||||
The VMID defaults to the last octet of the address, so `qm list` and
|
||||
`llmgw ctl nodes` stay in the same order. `--provision-only` and
|
||||
`--onboard-only` split it back into the two halves below when the two kinds
|
||||
of access belong to different people.
|
||||
|
||||
The rest of this page is what that command does, and what it still does not.
|
||||
That is all a machine that already exists needs — bare metal, or a VM you
|
||||
built. [install.md](install.md#5-adding-the-node) says what it does. On
|
||||
Proxmox, the machine itself can be built first by the script below, on the
|
||||
hypervisor.
|
||||
|
||||
## The two halves
|
||||
|
||||
@@ -54,41 +46,31 @@ their working tree.
|
||||
|
||||
## 2. The identity
|
||||
|
||||
On the host that holds the certificate authority — this is the part that cannot
|
||||
be delegated, because it issues the credential by which the control plane
|
||||
believes the node is who it says it is:
|
||||
On the control plane, which holds the certificate authority — this is the
|
||||
part that cannot be delegated, because it issues the credential by which the
|
||||
control plane believes the node is who it says it is:
|
||||
|
||||
```
|
||||
llmgw-onboard-node \
|
||||
--node llmgw-node04 \
|
||||
--addr 10.0.0.144 \
|
||||
--server https://10.0.0.140:7443 \
|
||||
--fabric ib-b
|
||||
sudo llmgw node add ilker@10.0.0.144 --fabric ib-b:mlx5_0,mlx5_1
|
||||
```
|
||||
|
||||
It issues a client certificate whose common name is the node id, installs it
|
||||
along with the CA root, trusts that root system-wide so other tooling on the
|
||||
machine stops treating llmgw's certificates as strangers, installs the binary
|
||||
and the unit, and starts the agent. The node appears in `llmgw ctl nodes` within
|
||||
a few seconds.
|
||||
It checks the machine with `llmgw doctor`, issues a client certificate whose
|
||||
common name is the node id (the hostname), installs it along with the CA root,
|
||||
trusts that root system-wide, installs the binary and the unit, starts the
|
||||
agent, and waits for the node to join.
|
||||
|
||||
If it does not, the control plane's log says why. A certificate whose common
|
||||
name does not match the claimed node id is refused by design — that check is
|
||||
what makes a node's identity something it proves rather than something it picks.
|
||||
If it does not join, the agent's log on the node says why. A certificate whose
|
||||
common name does not match the claimed node id is refused by design — that
|
||||
check is what makes a node's identity something it proves rather than
|
||||
something it picks.
|
||||
|
||||
**It needs ssh from the CA host to the node.** Provisioning copies that host's
|
||||
`authorized_keys` into the new machine, so a node built by the script above
|
||||
already satisfies it; an older machine may need the key added.
|
||||
**It needs SSH from the control plane to the node.** Provisioning copies the
|
||||
hypervisor's `authorized_keys` into the new machine; with `ssh -A` to the
|
||||
control plane and `sudo --preserve-env=SSH_AUTH_SOCK`, the same key that
|
||||
reaches the hypervisor reaches the node.
|
||||
|
||||
It waits for ssh rather than failing on it, and learns the host key of a machine
|
||||
it has not seen (`accept-new`, which still refuses a *changed* key — the risk
|
||||
being guarded against is a substituted machine, not a new one). Both are needed
|
||||
to run the halves back to back: for the first minute of its life a new machine
|
||||
is still regenerating host keys and restarting sshd, so it refuses connections
|
||||
it accepted a moment earlier.
|
||||
|
||||
It also checks that a configured engine actually runs, which is cheap here and
|
||||
expensive later — see below.
|
||||
`deploy/bootstrap/onboard-node.sh` is the script this replaced, for a machine
|
||||
holding an authority but not a current llmgw.
|
||||
|
||||
## The CPU model matters, and its default is wrong
|
||||
|
||||
|
||||
Reference in new issue
Block a user