diff --git a/deploy/bootstrap/onboard-node.sh b/deploy/bootstrap/onboard-node.sh
index 2078e54..35dc38c 100755
--- a/deploy/bootstrap/onboard-node.sh
+++ b/deploy/bootstrap/onboard-node.sh
@@ -31,6 +31,9 @@ usage: $0 --node ID --addr HOST --server URL [options]
                     a multi-node one.
   --mgmt-addr ADDR  address engines are reachable on (default: --addr)
   --mgmt-if NAME    interface NCCL bootstraps over (default eth0)
+  --nvidia-smi PATH where nvidia-smi lives, when it is not on PATH. Containerized
+                    driver installs routinely put it somewhere else.
+  --ib-root DIR     the sysfs InfiniBand directory (default /sys/class/infiniband)
   --pki DIR         where the authority lives (default /etc/llmgw/pki)
   --binary PATH     llmgw binary to install (default: the one running this)
   --ssh-user NAME   (default ilker)
@@ -39,6 +42,7 @@ USAGE
 }
 
 NODE= ADDR= SERVER= MGMT= MGMT_IF=eth0
+NVIDIA_SMI= IB_ROOT=/sys/class/infiniband
 PKI=/etc/llmgw/pki BINARY= SSH_USER=ilker
 FABRIC_NAME= FABRIC_HCAS="mlx5_0,mlx5_1"
 
@@ -50,6 +54,8 @@ while [ $# -gt 0 ]; do
     --fabric) FABRIC_NAME="${2%%:*}"; [ "${2#*:}" != "$2" ] && FABRIC_HCAS="${2#*:}"; shift 2;;
     --mgmt-addr) MGMT="$2"; shift 2;;
     --mgmt-if) MGMT_IF="$2"; shift 2;;
+    --nvidia-smi) NVIDIA_SMI="$2"; shift 2;;
+    --ib-root) IB_ROOT="$2"; shift 2;;
     --pki) PKI="$2"; shift 2;;
     --binary) BINARY="$2"; shift 2;;
     --ssh-user) SSH_USER="$2"; shift 2;;
@@ -127,6 +133,10 @@ reconnect_max: 30s
 $FABRICS
 
 discovery:
+  # Empty means look on PATH. Worth setting where the driver lives outside the
+  # default paths, which containerized installs routinely do.
+  nvidia_smi: "$NVIDIA_SMI"
+  ib_root: $IB_ROOT
   prefer_nvml: false
   skip_gpus: false
 
diff --git a/docs/node-onboarding.md b/docs/node-onboarding.md
new file mode 100644
index 0000000..333539d
--- /dev/null
+++ b/docs/node-onboarding.md
@@ -0,0 +1,103 @@
+# Adding a node
+
+Written for whoever runs the cluster. Two scripts, because the two halves of
+onboarding are done by different people with different access: one needs the
+hypervisor, the other needs the certificate authority.
+
+## 1. The machine
+
+On the Proxmox host:
+
+```
+llmgw-provision-node --vmid 144 --name llmgw-node04 --ip 10.0.0.144/24
+```
+
+Name it after the node id. **The VM name, the hostname, the node id and the
+certificate's common name are all the same string.** They do not have to be, and
+every time they are not, somebody spends an afternoon working out which of the
+four a message is talking about.
+
+It imports a cloud image and lets cloud-init do the rest, which is why a node
+takes a minute rather than an afternoon. It refuses to touch an existing VMID:
+adopting a machine is not a decision a provisioning script should make.
+
+## 2. The identity
+
+On the host that holds the certificate authority — this is the part that cannot
+be delegated, because it issues the credential by which the control plane
+believes the node is who it says it is:
+
+```
+llmgw-onboard-node \
+  --node llmgw-node04 \
+  --addr 10.0.0.144 \
+  --server https://10.0.0.140:7443 \
+  --fabric ib-b
+```
+
+It issues a client certificate whose common name is the node id, installs it
+along with the CA root, trusts that root system-wide so other tooling on the
+machine stops treating llmgw's certificates as strangers, installs the binary
+and the unit, and starts the agent. The node appears in `llmgw ctl nodes` within
+a few seconds.
+
+If it does not, the control plane's log says why. A certificate whose common
+name does not match the claimed node id is refused by design — that check is
+what makes a node's identity something it proves rather than something it picks.
+
+**It needs ssh from the CA host to the node.** Provisioning copies that host's
+`authorized_keys` into the new machine, so a node built by the script above
+already satisfies it; an older machine may need the key added.
+
+## Fabrics
+
+`--fabric` is how a node says which InfiniBand fabric it is attached to, and
+there is no way around stating it. OpenSM's default subnet prefix is `fe80::` on
+every fabric, so a matching prefix proves nothing and a wrong guess produces a
+replica that starts, passes health checks, and then hangs on its first
+all-reduce.
+
+Omit it for a node with no fabric. Such a node is its own island: it can host
+single-node replicas and will never be given a multi-node one.
+
+Two adapters per fabric is the default; `--fabric ib-b:mlx5_2,mlx5_3` names them
+explicitly. An adapter must never appear under two fabrics — the agent refuses
+that configuration, because the consequence is the hang above.
+
+## Where the driver is
+
+```
+--nvidia-smi /opt/nvidia/bin/nvidia-smi
+--ib-root /sys/class/infiniband
+```
+
+Both default to the usual places. Containerized driver installs routinely put
+`nvidia-smi` somewhere else, and an agent that cannot find it registers with no
+GPUs — which looks exactly like a node that has none.
+
+## Removing a node
+
+```
+llmgw ctl drain llmgw-node04     # move its replicas off, one at a time
+llmgw ctl nodes                  # wait until it holds none
+```
+
+Then stop the agent. Draining brings each replacement up before stopping the
+original, which is the difference between a rolling move and an outage.
+
+The node's certificate stays valid until it expires. There is no revocation list
+— if a machine is decommissioned rather than retired, reissue the authority or
+accept that a certificate naming a node that no longer exists can still connect
+and claim to be it.
+
+## Testing without hardware
+
+Every part of this works on machines with no GPUs and no InfiniBand. Point
+`--nvidia-smi` at a script that prints the CSV discovery asks for, point
+`--ib-root` at a sysfs-shaped directory, and use `engine: sim` in the catalog.
+
+The simulated engine rendezvouses for real: rank 0 binds the dist-init port and
+does not serve until every worker has joined, and workers serve nothing. That
+exercises the multi-node launch sequence, the fabric constraint, and failure and
+rebuild — which is most of what goes wrong — without exercising NCCL, which is
+the rest.
diff --git a/llmgw-plan.md b/llmgw-plan.md
index 18e7575..c6e27cc 100644
--- a/llmgw-plan.md
+++ b/llmgw-plan.md
@@ -1391,12 +1391,32 @@ The launch path is now built too:
       bounded by `rendezvous_timeout` so a head that never appears fails the
       shard instead of hanging it.
 
-- [ ] **Not verified against real engines.** Everything above is tested with
-      stub processes. The Ray sequencing, and whether the deployed vLLM version
-      wants `ray` or `mp`, can only be confirmed on real hardware.
+- [x] **Verified against a simulated cluster**, which is not the same as real
+      hardware and covers more than stub processes did. Four nodes across two
+      fabrics, eight GPUs each; `sim` is a real engine adapter whose leader
+      binds a rendezvous port and does not serve until every worker has joined,
+      and whose workers serve nothing at all.
 
-- **Exit criterion:** a 2-node replica on each fabric; killing a node rebuilds the replica on the
-  same fabric and never spreads it across two.
+      What that proves: placement honours the fabric constraint, the leader
+      starts first and workers wait for its port, only rank 0 is health-checked,
+      a 2-node replica comes up on each fabric, and losing half of one fails the
+      replica and rebuilds it **within a single fabric** rather than spanning
+      two.
+
+      What it does not prove: that a given vLLM version wants `ray` or `mp`,
+      that NCCL picks the right HCA, or that the generated environment makes a
+      real collective run. Those need hardware. The GPUs here are an inventory,
+      the fabric is a sysfs-shaped directory, and the engine speaks no NCCL.
+- [ ] **Confirm the engine flags on real hardware.** The launch *sequence* is
+      now exercised; the launch *arguments* for vLLM and SGLang are still read
+      from their documentation rather than from a running engine.
+
+- **Exit criterion (met on the simulated cluster):** a 2-node replica on each
+  fabric; killing a node rebuilds the replica within one fabric and never
+  spreads it across two. Measured: node02 lost at t+60s, the ib-a replica failed
+  and was reaped, a replacement was ready at t+80s on ib-b — the fabric that
+  could still host it — while node01 sat idle rather than being paired across
+  fabrics with a node from ib-b.
 
 ### Phase 5 — Production features (3 weeks)
 - [x] **SQLite store and migrations**, behind an interface, with per-dialect