One machine runs one judge, on up to four cores
CI / test (push) Successful in 9m1s

A second judge replica on the same machine shares its cores and is lost
with it, so it bought nothing but half the speed of each. On a 4-core,
8 GB single-machine install, one judge on four cores passed 20 of 20
requests at about 3.5 s each, judged in and out. Two judges on two cores
each had passed 19 of 20.

server init now serves one judge when this machine is the only node, and
two, on two cores each and spread over the nodes, where there will be
others (HA setups, or a control plane that is not a node).
--judge-replicas and the setup's Copies still choose. install.md gives
the measured times in place of "half a second".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-16 19:35:42 +02:00
1 parent f8206c3540
commit 3e7dd20bad
7 files changed
+91 -29

No files matched your search

+16 -11
View File
@@ -719,13 +719,13 @@ type cpuSplit struct{ judge, starter int }
// the starter on one 4-core control plane all ran on all four: under load
// most judgments missed their budget and the requests were refused.
//
// A judge needs two cores: a judgment of about 280 tokens took 8.0 s on
// one, 4.2 s on two and 2.3 s on four, against a 5 s budget. When this
// machine is the node, each judge replica gets two (one each only below four
// CPUs, where two would not fit both), and the starter what is left, up to
// 8, or no reservation at all when nothing is, rather than a starter that
// fits nowhere. Otherwise two a judge, on whichever node takes it, and the
// starter, on a node not known yet, none.
// A judge needs cores: a judgment of about 280 tokens took 8.0 s on one,
// 4.2 s on two and 2.3 s on four, against a 5 s budget. On one machine its
// one replica gets up to four; where there are two, two each (one each only
// where two would not fit both). The starter has what is left, up to 8, or
// no reservation at all when nothing is, rather than a starter that fits
// nowhere. A machine that is not the node reserves two a judge on whichever
// node takes it, and the starter, on a node not known yet, none.
func (in *serverInit) cpuPlan() cpuSplit {
if in.noNode {
return cpuSplit{judge: judgeCores}
@@ -742,7 +742,10 @@ func (in *serverInit) cpuPlan() cpuSplit {
judges = in.judgeFor().Replicas
}
judge := judgeCores
if judges > 0 && judges*judge > n {
switch {
case judges == 1:
judge = min(n, judgeCoresAlone)
case judges > 0 && judges*judge > n:
judge = max(1, n/judges)
}
left := max(n-judges*judge, 0)
@@ -751,8 +754,10 @@ func (in *serverInit) cpuPlan() cpuSplit {
// judgeCores and judgeMemoryGB are what a judge replica needs: two cores
// for a judgment within its budget, and 2 GB for its 1.6 with no prompt
// cache.
// cache. judgeCoresAlone is what the one judge of a single machine takes,
// past which a model this small hardly gets faster.
const (
judgeCores = 2
judgeMemoryGB = 2
judgeCores = 2
judgeCoresAlone = 4
judgeMemoryGB = 2
)