Ilker ManapandClaude Opus 5
e0ea4f6be9
CI / test (push) Successful in 1m41s
Serving a model from the mirror page left "Needs RAM (GB)" optional and empty. Empty writes no constraint, and a CPU profile has no card to be short of, so every ready node became a candidate — and a 4.68 GB model went onto a 7.9 GB machine and left it 3.5 GB for a KV pool. Placement was not wrong; it was never told what the model needed. The mirror knows the size of the weights, so the field arrives filled in: weights plus half again for the KV pool, plus two for the engine. The half is there because the cache grows with the same things the weights do, layers and heads. It is an estimate and the field is still editable — what it is not is blank. A 1.12 GB model asks for 3 GB, so the small nodes stay usable. Pinning every model to the one large machine would be a different way of getting placement wrong. An unknown size fills in nothing rather than the floor of two, which would exclude no node while reading as though somebody had measured it. And when the estimate is larger than any node in the fleet, the page says so, with the largest node's memory. Better read there than discovered by a node being OOM-killed twenty minutes into a load. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>