Facility-aware placement: racks, PDUs, holds and power caps

Nodes say where they are with llmgw.io/site, room, row, rack and pdu labels,
from agent.yaml or `llmgw ctl label` (kept in the store, over the agent's).
Placement spreads a model's replicas across racks and PDUs, keeps a
multi-node replica within one domain when its profile says keep_within,
places nothing in a domain held with `llmgw ctl facility hold` (optionally
draining it), and refuses a shard that would take a domain past its power
cap from facility.budgets, counting declared, derived and measured draw.
Explain-placement names the domain, and keeps the rejections of a placed
replica; `ctl nodes` gains LOCATION and --domain; `ctl facility` lists the
domains; the console's Nodes view shows and sets location and holds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 01:17:49 +02:00
1 parent d41104a452
commit 0d3eb82b9b
42 files changed
+3510 -63

No files matched your search

+113
View File
@@ -86,6 +86,15 @@ rather than repeated.
| `url` | `string` | [External](#external) |
| `model` | `string` | [External](#external) |
| `api_key_file` | `string` | [External](#external) |
| `budgets` | `[]FacilityBudget` | [Facility](#facility) |
| `margin` | `float64` | [Facility](#facility) |
| `gpu_watts` | `float64` | [Facility](#facility) |
| `cpu_core_watts` | `float64` | [Facility](#facility) |
| `node_watts` | `float64` | [Facility](#facility) |
| `measured_fresh` | `time.Duration` | [Facility](#facility) |
| `domain` | `string` | [FacilityBudget](#facilitybudget) |
| `cap_watts` | `float64` | [FacilityBudget](#facilitybudget) |
| `other_watts` | `float64` | [FacilityBudget](#facilitybudget) |
| `format` | `string` | [GuardModel](#guardmodel) |
| `categories` | `[]string` | [GuardModel](#guardmodel) |
| `model` | `string` | [JudgePolicy](#judgepolicy) |
@@ -194,6 +203,8 @@ rather than repeated.
| `cpu_cores` | `int` | [Profile](#profile) |
| `min_disk_gb` | `int` | [Profile](#profile) |
| `node_selector` | `map[string]string` | [Profile](#profile) |
| `power_watts` | `float64` | [Profile](#profile) |
| `keep_within` | `string` | [Profile](#profile) |
| `weight` | `float64` | [Profile](#profile) |
| `priority` | `int` | [Profile](#profile) |
| `startup_timeout` | `time.Duration` | [Profile](#profile) |
@@ -271,6 +282,7 @@ rather than repeated.
| `router` | `Router` | [Server](#server) |
| `control` | `Control` | [Server](#server) |
| `models` | `[]catalog.Model` | [Server](#server) |
| `facility` | `Facility` | [Server](#server) |
| `endpoints` | `[]Endpoint` | [Server](#server) |
| `bin` | `string` | [SimConfig](#simconfig) |
| `python` | `string` | [TRTLLMConfig](#trtllmconfig) |
@@ -839,6 +851,75 @@ APIKeyFile holds the provider's key, on the control plane. A file
rather than the key itself, so the key is in neither the catalog, the
store nor the audit log; empty for a provider that asks for none.
## Facility
### `budgets`
`[]FacilityBudget`
Budgets are the domains with a power cap.
### `margin`
`float64`
Margin is the share of every cap kept free, 0.1 for ten percent. Zero
keeps none, for caps already given derated.
### `gpu_watts`
`float64`
GPUWatts is a card's draw when it does not report its power limit:
an agent without nvidia-smi's power fields, or a node that has not
sent a heartbeat yet. Zero counts such a card as drawing nothing.
### `cpu_core_watts`
`float64`
CPUCoreWatts is the draw of a core reserved by a replica with no GPU
(cpu_cores). Zero counts such replicas as drawing nothing.
### `node_watts`
`float64`
NodeWatts is what a node draws besides its cards and reserved cores —
idle CPUs, memory, fans, disks — counted for every node in a capped
domain, busy or not.
### `measured_fresh`
`time.Duration`
MeasuredFresh is how old a node's power reading may be and still be
counted against its domains. Zero takes two minutes.
## FacilityBudget
### `domain`
`string`
Domain is level:name, as the nodes' labels give it: rack:r12-07,
pdu:r12-07-a, row:12.
### `cap_watts`
`float64`
CapWatts is what the domain may draw. For a dual-fed rack it is what
one feed carries alone, derated as the facility derates it (usually
80%), since that is what the rack has to survive.
### `other_watts`
`float64`
OtherWatts is drawn in the domain by what llmgw does not place on:
switches, storage, machines outside the cluster.
## GuardModel
### `format`
@@ -1638,6 +1719,27 @@ in the inventory says how much memory a node has, so the operator says
which nodes are meant for a model instead of the scheduler inferring it.
Agents have reported labels since the beginning; this is what reads them.
### `power_watts`
`float64`
PowerWatts is what one replica of this profile draws, in watts, split
evenly over its nodes, counted against the power cap of every rack,
PDU or other domain its nodes are in (facility.budgets in server.yaml).
Zero derives it: each card at the power limit it reports, each
reserved core at facility.cpu_core_watts. Only matters where a domain
has a cap.
### `keep_within`
`string`
KeepWithin keeps a multi-node replica's nodes inside one rack, row,
room or site, read from their llmgw.io/rack (and so on) labels: for a
fabric whose leaves do not reach across racks, or cables that do not.
Nodes without the label are passed over for such a profile. Empty is
no constraint.
### `weight`
`float64`
@@ -2249,6 +2351,17 @@ Takes a [Control](#control) block.
`[]catalog.Model`
### `facility`
`Facility`
Facility is the power caps of racks, PDUs and other domains, and how
a replica's draw is estimated against them. Where a node is comes from
its llmgw.io/site, room, row, rack and pdu labels; holding a domain for
maintenance is `llmgw ctl facility hold`, not configuration.
Takes a [Facility](#facility) block.
### `endpoints`
`[]Endpoint`