The setup screen asks for up to two judge machines after the store's
two, and server init takes --judge-node for each. Each is added as a
node labelled llmgw.role: judge, and the judge's profile selects that
label, with cores by init's rule over those machines. A judge beside a
busy control plane refused 38 of 60 requests under load; alone, 2.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A second judge replica on the same machine shares its cores and is lost
with it, so it bought nothing but half the speed of each. On a 4-core,
8 GB single-machine install, one judge on four cores passed 20 of 20
requests at about 3.5 s each, judged in and out. Two judges on two cores
each had passed 19 of 20.
server init now serves one judge when this machine is the only node, and
two, on two cores each and spread over the nodes, where there will be
others (HA setups, or a control plane that is not a node).
--judge-replicas and the setup's Copies still choose. install.md gives
the measured times in place of "half a second".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Checked by a single-machine install of v0.14.6 on a 4-core, 8 GB VM. The
second judge fitted nowhere: v0.14.6 reserved 4 GB a judge, a figure from
one replica that grew to 3.2 GB through llama.cpp's prompt cache (up to
8 GB by default), which a judge never reuses. Once it fitted, every
judgment timed out. v0.14.6 gave a judge one core below 8 CPUs, and a
judgment of about 280 tokens takes 8.0 s on one core, 4.2 s on two and
2.3 s on four, against a 5 s budget: 0 of 20 requests went through.
A judge now runs with --cache-ram 0, uses about 1.6 GB and reserves 2,
and reserves two cores (one only below four CPUs). On the same VM, 19 of
20 went through.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Measured on the three-VM cluster with ha1's CPUs loaded: 38 of 60 requests
refused with both judges and the starter on ha1, each engine on all four
cores. With the judges moved to a 4-core, 32 GB node at 2 cores each, 2 of
60 were refused, both the first after the judges started.
init now reserves cores by this machine's count: one per judge replica
under 8 CPUs and two from 8, and the rest, up to 8, for the starter on CPU.
The starter reserves nothing when nothing is left, so it still fits. Judges
bound for nodes not yet known get two each. A judge reserves 4 GB; it uses
about 3.2.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Found by a three-machine install on fresh VMs. The second standby's run
asked for the floating address's interface with `ip route get <vip>`.
The machine already held the address, so that answered lo, and keepalived
would not start. The interface now comes from the route to the address's
network. Every cluster's VRRP router id was 51, so the PoC's two control
planes and the new three on one LAN each logged the other's adverts as an
invalid password; the id now comes from the address.
The Caddyfile, deploy/ha and install.md send clients to the control plane
whose /leader is 200, not /ready: /ready is 503 on a leader with no model
yet and would take the portal away with it. Setup no longer hides a key
file's path as if it were a password, and the starter's text no longer
says it runs elsewhere when this machine is a node.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The setup screens list the parts of an install: control plane, store, model
mirror, this machine as a node, starter model, pictures and 3D, the judge,
firewall, notifications, limits, on-demand turns and an offline bundle. Each
says what it is or what is missing without it. Space turns a part on or off;
Enter opens its settings in a dialog that explains it. A plan page lists what
is installed and what is left out, and the same install as a command. Every
setting is a server init flag too, written into server.yaml when init writes
it.
With three machines, the two database machines become control planes too,
standing by (server add's steps), with an optional floating address. The
mirror's management API answers all three control planes, so a standby that
comes to lead can still fetch models.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
