Document what changed since v0.9.2, and fabrics for the sites that have them
The changelog's unreleased section had stopped at refusing unloadable weights; it now covers the rest: telling independent fabrics apart, their subnet managers and maps, NVLink on every build and across nodes, the multi-node launch fixes and vLLM's new default, doctor, node load, editors and the Responses API, the chat and mirror pages, a person's own use, the admin pages, the new metrics, and how to upgrade. docs/fabrics.md is new, for a site with InfiniBand or NVLink: what llmgw does with them, what each node and fabric needs first, declaring fabrics, checking a node with doctor, what the pages and metrics show and which to alert on, what an engine is given, and where to look when a replica spanning nodes will not run. It says plainly that none of it has yet run on such hardware. install.md, node-onboarding.md, onboarding.md, the README and the example agent configuration no longer say the portable binary cannot see NVLink or that a fabric's name cannot be checked, and describe the account page and connecting an editor. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f989e67e1a
commit
80e0dd252f
10 files changed
+440
-67
No files matched your search
@@ -47,26 +47,24 @@ heartbeat_interval: 5s
|
||||
reconnect_base: 1s
|
||||
reconnect_max: 30s
|
||||
|
||||
# Fabric labelling is explicit and there is no way around it. OpenSM's default
|
||||
# subnet prefix is fe80:: on every fabric, so a matching prefix proves nothing
|
||||
# and a wrong guess produces a replica that starts, passes health checks and
|
||||
# then hangs on its first all-reduce (plan §5.2).
|
||||
# Each fabric this node's adapters are on, by name. The name is checked against
|
||||
# what the adapters report — the subnet managers every port on one fabric sees —
|
||||
# so two independent fabrics given one name are placed as two and the cluster
|
||||
# page says so; give each its own. The rate is measured, not read from here.
|
||||
#
|
||||
# A node with no fabrics is its own single-node island: it can host single-node
|
||||
# replicas and will never be given a multi-node one.
|
||||
# replicas and will never be given a multi-node one. docs/fabrics.md has more.
|
||||
fabrics:
|
||||
- name: ib-a
|
||||
hcas: [mlx5_0, mlx5_1, mlx5_2, mlx5_3]
|
||||
rate_gbps: 200
|
||||
|
||||
discovery:
|
||||
# Worth setting where the driver lives outside the default paths, which
|
||||
# containerized driver installs often do.
|
||||
nvidia_smi: ""
|
||||
ib_root: /sys/class/infiniband
|
||||
# NVML is the only source of NVLink topology, which is what keeps a
|
||||
# tensor-parallel group off the PCIe path inside a node. It needs a binary
|
||||
# built with `-tags nvml`; otherwise this silently falls back to nvidia-smi.
|
||||
# NVLink topology comes from `nvidia-smi topo -m` on every build; a binary
|
||||
# built with `-tags nvml` reads it from NVML instead, when this is set.
|
||||
prefer_nvml: true
|
||||
|
||||
# Where this node keeps model weights besides /var/lib/llmgw-weights, which is
|
||||
|
||||
Reference in new issue
Block a user