Document what changed since v0.9.2, and fabrics for the sites that have them
The changelog's unreleased section had stopped at refusing unloadable weights; it now covers the rest: telling independent fabrics apart, their subnet managers and maps, NVLink on every build and across nodes, the multi-node launch fixes and vLLM's new default, doctor, node load, editors and the Responses API, the chat and mirror pages, a person's own use, the admin pages, the new metrics, and how to upgrade. docs/fabrics.md is new, for a site with InfiniBand or NVLink: what llmgw does with them, what each node and fabric needs first, declaring fabrics, checking a node with doctor, what the pages and metrics show and which to alert on, what an engine is given, and where to look when a replica spanning nodes will not run. It says plainly that none of it has yet run on such hardware. install.md, node-onboarding.md, onboarding.md, the README and the example agent configuration no longer say the portable binary cannot see NVLink or that a fabric's name cannot be checked, and describe the account page and connecting an editor. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f989e67e1a
commit
80e0dd252f
10 files changed
+440
-67
No files matched your search
+12
-6
@@ -178,18 +178,24 @@ was cut short.
|
||||
|
||||
## Fabrics
|
||||
|
||||
`--fabric` is how a node says which InfiniBand fabric it is attached to, and
|
||||
there is no way around stating it. OpenSM's default subnet prefix is `fe80::` on
|
||||
every fabric, so a matching prefix proves nothing and a wrong guess produces a
|
||||
replica that starts, passes health checks, and then hangs on its first
|
||||
all-reduce.
|
||||
`--fabric` is how a node says which InfiniBand fabric it is attached to. The
|
||||
name is checked against what the adapters report — every port on one fabric
|
||||
sees the same subnet managers — so two independent fabrics given one name are
|
||||
placed as two, and the cluster page says so; give each fabric its own name.
|
||||
The agent installs `infiniband-diags`, which that check needs, if it is missing.
|
||||
|
||||
Before a node with a fabric joins, run `sudo llmgw doctor` on it: it checks the
|
||||
subnet manager, GPUDirect RDMA, the fabric manager and IMEX on NVLink machines,
|
||||
and the declared fabrics against the adapters. [fabrics.md](fabrics.md) has the
|
||||
rest.
|
||||
|
||||
Omit it for a node with no fabric. Such a node is its own island: it can host
|
||||
single-node replicas and will never be given a multi-node one.
|
||||
|
||||
Two adapters per fabric is the default; `--fabric ib-b:mlx5_2,mlx5_3` names them
|
||||
explicitly. An adapter must never appear under two fabrics — the agent refuses
|
||||
that configuration, because the consequence is the hang above.
|
||||
that configuration: a replica given adapters on two fabrics starts, passes its
|
||||
health checks, and hangs on its first all-reduce.
|
||||
|
||||
## Where the driver is
|
||||
|
||||
|
||||
Reference in new issue
Block a user