Each agent measures one core's memory rate beside the whole machine's; the estimate takes a replica's threads at about one core's rate each, up to the machine's, calibrated by llama-bench on the PoC. The serve form shows it beside a card's, warns when slow, and asks for the CPU on purpose when a reply would take minutes; ctl model plan, the console and serve jobs say it too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
220110ed0d
commit
6403e61f4b
28 files changed
+657
-62
No files matched your search
@@ -8305,6 +8305,10 @@
|
||||
"type": "number",
|
||||
"description": "MemBandwidthGBs is how fast its memory reads, in GB/s, measured when its agent started; 0 when not reported. A model's speed on the CPU."
|
||||
},
|
||||
"mem_bandwidth_core_gbs": {
|
||||
"type": "number",
|
||||
"description": "MemBandwidthCoreGBs is what one core reads alone, in GB/s; 0 when not reported. A CPU replica's threads read at about this each."
|
||||
},
|
||||
"engines": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
@@ -11406,6 +11410,10 @@
|
||||
},
|
||||
"strata_size": {
|
||||
"type": "string"
|
||||
},
|
||||
"tokens_per_second": {
|
||||
"type": "number",
|
||||
"description": "TokensPerSecond is about how many tokens a second one reply gets this way, on the fastest host it fits; 0 when not known. On the CPU with the threads a replica is given by default: half the node's."
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
Reference in new issue
Block a user