Place on nodes that can actually hold the model
CI / test (push) Successful in 1m23s

Placement had no idea how much memory a node has or a model needs. A CPU
profile has no card to be free, so eligibleNodes accepted every ready node:
Qwen2.5-Coder 14B, whose working set is about 24 GB, would have been placed
just as happily on one of the 8 GB nodes, which reports itself perfectly
healthy right up until the OOM killer arrives.

node_selector was yesterday's stopgap and it is the operator restating by
hand something the system can know. Agents now report MemTotal, profiles
declare min_memory_gb, and placement filters on it, naming what was short —
"has 8 GB of memory, needs 24" is actionable where "no candidates were
considered" is not. The label selector stays, because "this node is for
coding models" is a real thing to want to say that memory does not express.

Two deliberate non-exclusions. A profile with no requirement fits anywhere,
which is every profile that exists today. And a node that reports no memory
is still usable: an agent older than the field says nothing, and letting a
control plane upgrade quietly empty half a fleet is a worse failure than the
one this prevents.

ctl nodes grows a MEM column showing "-" rather than 0 for a node that did
not say, because an old agent is not a small machine.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 23:12:44 +02:00
1 parent e9fe3ad379
commit 3081c6f69b
13 files changed
+233 -8

No files matched your search

+12 -3
View File
@@ -133,10 +133,10 @@ func newNodesCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
}
w := tab()
fmt.Fprintln(w, "NODE\tSTATE\tGPUS\tCLASSES\tFABRICS\tREPLICAS\tLAST SEEN")
fmt.Fprintln(w, "NODE\tSTATE\tGPUS\tMEM\tCLASSES\tFABRICS\tREPLICAS\tLAST SEEN")
for _, n := range nodes {
fmt.Fprintf(w, "%s\t%s\t%d/%d free\t%s\t%s\t%d\t%s\n",
n.ID, n.State, n.GPUFree, n.GPUTotal,
fmt.Fprintf(w, "%s\t%s\t%d/%d free\t%s\t%s\t%s\t%d\t%s\n",
n.ID, n.State, n.GPUFree, n.GPUTotal, memColumn(n.MemoryGB),
join(n.Classes), join(n.Fabrics), n.Replicas, n.LastSeenAgo)
}
_ = w.Flush()
@@ -156,6 +156,15 @@ func newNodesCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
}
}
// memColumn shows a node's memory, distinguishing "did not say" from a number.
// An older agent reports nothing, and that is not the same as a small machine.
func memColumn(gb int) string {
if gb <= 0 {
return "-"
}
return fmt.Sprintf("%dG", gb)
}
func newModelsCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
return &cobra.Command{
Use: "models",