Files
llmgw/internal/control
Ilker ManapandClaude Opus 5 8fd4c1392d
CI / test (push) Successful in 1m46s
Stop handing the same memory to two models
Placement compared a profile's memory requirement against the node's total, so
a 31 GB machine accepted two models that each declared 15: each was asked about
on its own, and nothing recorded that the first had claimed anything. GPUs were
never handed out twice this way — they are tracked as claimed indices — and
memory was, because the allocation carried no figure for it.

An allocation records what its profile asked for, and eligibility subtracts
what a node already owes before comparing. The rejection names both numbers,
because "needs 15" beside a 31 GB machine reads as a bug in the scheduler
rather than a full node.

Declared, not measured. A profile that says 15 is treated as holding 15 whether
the process has touched that much yet or not — the alternative is deciding
against what a model has grown into so far, which is wrong every time one is
still loading.

Zero is nobody said, and claims nothing. Counting it would let one undeclared
model make a whole node look full.

Adoption after a restart re-reads the figure from the catalog rather than from
the agent: what a profile asked for is not something a running process reports,
and a control plane that forgot every claim on restart would put a second large
model onto a node already full of the first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-27 17:44:26 +02:00
..
2026-09-27 16:51:09 +02:00
2026-09-27 16:51:09 +02:00