A card in the wrong PCIe slot is said so
CI / test (push) Successful in 9m17s

At the user's asking: we know what a card's link could be, so a card
linked slower is reported. Each agent reads, apart from the GPU query so
that a driver not knowing a field costs nothing else, the fastest PCIe
generation card and slot allow together, the card's own, and the lanes in
use and the card's most (nvidia-smi; NVML in the nvml build). The current
generation is not compared: it drops while a card idles. A card on fewer
lanes, or a slot of an older generation, is a node warning with the GB/s
it carries against what it could — on the dashboard, the console and in
llmgw doctor, which points at lspci's LnkCap and LnkSta. MIG instances are
said of their card. GPU proto fields 23-26.

The PoC's Quadro RTX 6000 reads Gen3 of Gen3, x16 of x16, idle at Gen1:
nothing to say.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-29 08:49:16 +01:00
1 parent 6a870a8c15
commit 721705b29d
13 files changed
+279 -3

No files matched your search

+21
View File
@@ -1053,6 +1053,27 @@ A node with no location is placed as before, as a rack of its own.
See [facility-aware-placement.md](facility-aware-placement.md) for power caps
and holds.
### A card in the wrong slot
An x16 card in a slot wired x8 or x4, behind a riser, or in a slot of an
older PCIe generation than the card's, links slower than it was built to and
works otherwise. A model on the card still decodes at the card's own memory
speed, but it loads slower, a replica split over several cards by PCIe is
slower, and so is anything that spills to the host. Each card's agent reads
how its link stands — the fastest generation card and slot allow together,
and the lanes in use — against what the card could do, and the node says so
among its warnings (the dashboard, the console's Nodes, `llmgw doctor`):
```
GPU 1 (NVIDIA H100 80GB HBM3) linked at x8 of its x16 lanes: 31.5 GB/s where it
could have 63.0. Check the slot (wired fewer lanes, or an older generation), a
riser, or the BIOS's PCIe setting
```
The generation a card reports now drops while it idles, to save power; what is
compared is the fastest it may reach, which does not. `lspci -vv -s BUS-ID`
shows the same: `LnkCap` is the card's, `LnkSta` the link's.
## 6. Setting the node's CPU model, if it is a VM
Proxmox defaults a VM to `kvm64`, which hides AVX, AVX2, FMA and even SSE4.2