Place on nodes that can actually hold the model

Placement had no idea how much memory a node has or a model needs. A CPU
profile has no card to be free, so eligibleNodes accepted every ready node:
Qwen2.5-Coder 14B, whose working set is about 24 GB, would have been placed
just as happily on one of the 8 GB nodes, which reports itself perfectly
healthy right up until the OOM killer arrives.

node_selector was yesterday's stopgap and it is the operator restating by
hand something the system can know. Agents now report MemTotal, profiles
declare min_memory_gb, and placement filters on it, naming what was short —
"has 8 GB of memory, needs 24" is actionable where "no candidates were
considered" is not. The label selector stays, because "this node is for
coding models" is a real thing to want to say that memory does not express.

Two deliberate non-exclusions. A profile with no requirement fits anywhere,
which is every profile that exists today. And a node that reports no memory
is still usable: an agent older than the field says nothing, and letting a
control plane upgrade quietly empty half a fleet is a worse failure than the
one this prevents.

ctl nodes grows a MEM column showing "-" rather than 0 for a node that did
not say, because an old agent is not a small machine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 23:12:44 +02:00
1 parent e9fe3ad379
commit 3081c6f69b
13 files changed
+233 -8

No files matched your search

+8 -3
View File
@@ -44,8 +44,12 @@ type NodeView struct {
State string `json:"state"`
Labels map[string]string `json:"labels,omitempty"`
GPUTotal int `json:"gpu_total"`
GPUFree int `json:"gpu_free"`
GPUTotal int `json:"gpu_total"`
GPUFree int `json:"gpu_free"`
// MemoryGB is the machine's RAM, 0 when the agent did not report it. It is
// what decides whether a CPU profile can be placed here, so it belongs
// where someone looks when a model will not schedule.
MemoryGB int `json:"memory_gb,omitempty"`
Classes []string `json:"gpu_classes,omitempty"`
Fabrics []string `json:"fabrics,omitempty"`
@@ -140,7 +144,8 @@ func (a *API) Nodes() []NodeView {
ID: n.ID, Hostname: n.Hostname, MgmtAddr: n.MgmtAddr,
State: n.State.String(), Labels: n.Labels,
GPUTotal: len(n.GPUs), GPUFree: len(inv.FreeGPUs(n.ID)),
Fabrics: n.FabricNames(), AgentVersion: n.AgentVersion,
MemoryGB: n.MemoryTotalMB / 1024,
Fabrics: n.FabricNames(), AgentVersion: n.AgentVersion,
Replicas: replicasOn[n.ID],
}
seen := map[string]struct{}{}