Facility-aware placement: racks, PDUs, holds and power caps
Nodes say where they are with llmgw.io/site, room, row, rack and pdu labels, from agent.yaml or `llmgw ctl label` (kept in the store, over the agent's). Placement spreads a model's replicas across racks and PDUs, keeps a multi-node replica within one domain when its profile says keep_within, places nothing in a domain held with `llmgw ctl facility hold` (optionally draining it), and refuses a shard that would take a domain past its power cap from facility.budgets, counting declared, derived and measured draw. Explain-placement names the domain, and keeps the rejections of a placed replica; `ctl nodes` gains LOCATION and --domain; `ctl facility` lists the domains; the console's Nodes view shows and sets location and holds. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d41104a452
commit
0d3eb82b9b
42 files changed
+3510
-63
No files matched your search
+23
-2
@@ -857,8 +857,8 @@ llmgw ctl nodes
|
||||
```
|
||||
|
||||
```
|
||||
NODE STATE GPUS MEM CLASSES FABRICS REPLICAS LAST SEEN
|
||||
llmgw-node01 ready 8/8 free 503G h100-80g ib-a 0 2s
|
||||
NODE STATE LOCATION GPUS MEM CLASSES FABRICS REPLICAS LAST SEEN
|
||||
llmgw-node01 ready - 8/8 free 503G h100-80g ib-a 0 2s
|
||||
```
|
||||
|
||||
If it does not appear, the control plane's log says why. A certificate whose
|
||||
@@ -870,6 +870,27 @@ Repeat for each node. Nodes in the same fabric must be given the same `--fabric`
|
||||
name; nodes with no InfiniBand take no `--fabric` at all and are each their own
|
||||
island, which is fine for single-node models.
|
||||
|
||||
### Saying where the node is
|
||||
|
||||
On more than one rack, tell llmgw which rack, and which PDUs, each node is in.
|
||||
Placement then keeps a model's replicas in different racks and on different
|
||||
PDUs, places nothing in a rack held for maintenance, and keeps each rack under
|
||||
a power cap if you give it one:
|
||||
|
||||
```
|
||||
llmgw ctl label llmgw-node01 llmgw.io/rack=r12-07 llmgw.io/pdu=r12-07-a,r12-07-b
|
||||
llmgw ctl label llmgw-node01 llmgw.io/row=12 llmgw.io/room=hall-b llmgw.io/site=sto1
|
||||
```
|
||||
|
||||
Labels set this way are kept by the control plane and win over the node's
|
||||
`agent.yaml`, where the same keys can go under `labels:` instead; a node need
|
||||
not have joined yet. `llmgw ctl nodes` shows the LOCATION, `--domain
|
||||
rack:r12-07` lists one rack, and `llmgw ctl facility` lists every rack, row and
|
||||
PDU with what it draws. In the console it is the node's menu, **Location…**.
|
||||
A node with no location is placed as before, as a rack of its own.
|
||||
See [facility-aware-placement.md](facility-aware-placement.md) for power caps
|
||||
and holds.
|
||||
|
||||
## 6. Setting the node's CPU model, if it is a VM
|
||||
|
||||
Proxmox defaults a VM to `kvm64`, which hides AVX, AVX2, FMA and even SSE4.2
|
||||
|
||||
Reference in new issue
Block a user