Commit Graph
4 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 0d3eb82b9b Facility-aware placement: racks, PDUs, holds and power caps
Nodes say where they are with llmgw.io/site, room, row, rack and pdu labels,
from agent.yaml or `llmgw ctl label` (kept in the store, over the agent's).
Placement spreads a model's replicas across racks and PDUs, keeps a
multi-node replica within one domain when its profile says keep_within,
places nothing in a domain held with `llmgw ctl facility hold` (optionally
draining it), and refuses a shard that would take a domain past its power
cap from facility.budgets, counting declared, derived and measured draw.
Explain-placement names the domain, and keeps the rejections of a placed
replica; `ctl nodes` gains LOCATION and --domain; `ctl facility` lists the
domains; the console's Nodes view shows and sets location and holds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 01:17:49 +02:00
Ilker ManapandClaude Opus 5.5 d10fad064e Mark facility steps 1-5 deferred; node sensors collected
CI / test (push) Successful in 3m45s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 20:25:50 +02:00
Ilker ManapandClaude Opus 5.5 cd10e086c8 Read nodes' heat and power: GPUs, CPUs, board inlet, RAPL
Release / build (push) Successful in 1m10s
CI / test (push) Successful in 3m51s
Release / notes (push) Successful in 3s
Every agent samples, at most every ten seconds: nvidia-smi's temperature,
HBM temperature, power draw, enforced limit and throttle reasons per GPU;
hwmon's CPU package and board inlet temperatures; and CPU package power
from RAPL's energy counters. Sent with heartbeats, published as metrics
and shown on the node page; unmeasured sensors are absent, never zero.
Observe-only, step 0 of docs/facility-aware-placement.md, which now
describes the agent source and what these readings can and cannot say.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 19:49:08 +02:00
Ilker ManapandClaude Opus 5.5 8724dc027a Plan facility-aware placement: rack power, temperature and cooling
CI / test (push) Successful in 3m41s
A design document for step 5 of resource allocation, deferred: location
labels and rack spreading, power budgets with estimated then measured draw,
thermal and cooling conditions with hysteresis, opt-in shedding through
preemption, pluggable facility sources, and a rollout that is useful at
each step and safe when data is missing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 19:25:31 +02:00