A three-machine install can give the judges machines of their own
CI / test (push) Successful in 5m10s
CI / test (push) Successful in 5m10s
The setup screen asks for up to two judge machines after the store's two, and server init takes --judge-node for each. Each is added as a node labelled llmgw.role: judge, and the judge's profile selects that label, with cores by init's rule over those machines. A judge beside a busy control plane refused 38 of 60 requests under load; alone, 2. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
8b122aae09
commit
51ca09287f
12 files changed
+346
-44
No files matched your search
+13
-1
@@ -199,7 +199,19 @@ a busy one, or a VM without AVX2 (see
|
||||
[step 6](#6-setting-the-nodes-cpu-model-if-it-is-a-vm)), the judge is the
|
||||
first thing to fail. The choices there are a longer `latency_budget`,
|
||||
`fail: open` (serve the request, recorded as flagged), the judge on a GPU,
|
||||
or `--no-guard-model`. Defamation, specialized advice,
|
||||
or `--no-guard-model`.
|
||||
|
||||
In a three-machine install the judges are better on machines of their own.
|
||||
On a 4-core control plane that was also a node, 38 of 60 requests under load
|
||||
were refused for want of a judgment. With the judges on a node of their own
|
||||
(4 cores, two judges of two cores each), 2 were refused. The setup screen asks
|
||||
for up to two **judge machines** after the store's two, or give
|
||||
`--judge-node user@host` to `server init`, once for each. Each is added as a
|
||||
node labelled `llmgw.role: judge`, and the judge's profile selects that
|
||||
label, so the judges run only there and nothing else does. Four cores and
|
||||
8 GB are enough for two judges.
|
||||
|
||||
Defamation, specialized advice,
|
||||
intellectual property and elections are left out of what it refuses: they
|
||||
catch too much ordinary work. [guardrails.md](guardrails.md) has the rest.
|
||||
|
||||
|
||||
Reference in new issue
Block a user