Reserve CPU cores and require memory for CPU models
A profile's cpu_cores is reserved out of its node's CPUs by placement, as memory is, and becomes the engine's thread count. Agents report their CPU count in registration and heartbeats. New CPU models must declare memory; model views show reserved against measured memory with warnings, and ctl model resources sets both for a stored model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
26151ec25a
commit
990d82d801
31 files changed
+610
-53
No files matched your search
@@ -129,6 +129,7 @@ rather than repeated.
|
||||
| `args` | `[]string` | [Profile](#profile) |
|
||||
| `env` | `map[string]string` | [Profile](#profile) |
|
||||
| `min_memory_gb` | `int` | [Profile](#profile) |
|
||||
| `cpu_cores` | `int` | [Profile](#profile) |
|
||||
| `min_disk_gb` | `int` | [Profile](#profile) |
|
||||
| `node_selector` | `map[string]string` | [Profile](#profile) |
|
||||
| `weight` | `float64` | [Profile](#profile) |
|
||||
@@ -966,6 +967,20 @@ this. A node that does not report its memory is not excluded either —
|
||||
an older agent must not have its nodes quietly emptied by an upgrade of
|
||||
the control plane.
|
||||
|
||||
### `cpu_cores`
|
||||
|
||||
`int`
|
||||
|
||||
CPUCores is the cores a replica of a profile with no GPU is given:
|
||||
reserved out of its node's CPUs by placement, as memory is, and what
|
||||
its engine computes on (llama.cpp's --threads).
|
||||
|
||||
Reserved because an engine takes every core it is told about, and two
|
||||
on one node that each think they have all of them spin against each
|
||||
other: on an 8-vCPU node, two 7B replicas fell from 10 tokens a second
|
||||
to under one each. Zero reserves nothing and leaves the thread count to
|
||||
the node, which is every profile from before this.
|
||||
|
||||
### `min_disk_gb`
|
||||
|
||||
`int`
|
||||
|
||||
Reference in new issue
Block a user