A judge a policy names and the catalog lacked was served with 3 GB, no cores and llama.cpp's prompt cache; on a busy node its judgments missed their budget and fail: closed refused the requests. It now gets init's 2 GB and cores, and every judge modelserve serves runs with --cache-ram 0. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
99c34fd68f
commit
8b122aae09
8 files changed
+70
-3
No files matched your search
@@ -10769,6 +10769,10 @@
|
||||
"cpu_cores": {
|
||||
"type": "integer",
|
||||
"description": "CPUCores is what a replica computes on; zero takes half the smallest node it fits on."
|
||||
},
|
||||
"memory_gb": {
|
||||
"type": "integer",
|
||||
"description": "MemoryGB is what a replica on CPUs reserves, in place of the figure worked out from its weights and context: a judge's, whose own sizing is known."
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
Reference in new issue
Block a user