Commit Graph
5 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 314cc5e6a3 The judge init installs is no longer replaced by the leader's own
CI / test (push) Successful in 5m27s
Found on a fresh HA install with a judge machine: the leader saw the
policy's judge missing while init was still fetching it, served one
itself, and its entry replaced init's, so a second judge ran off the
judge machine. The keeper now waits ten minutes after it starts, and a
serve job it starts gives way to a model added meanwhile (AddNew).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 20:37:10 +02:00
Ilker ManapandClaude Opus 5.5 51ca09287f A three-machine install can give the judges machines of their own
CI / test (push) Successful in 5m10s
The setup screen asks for up to two judge machines after the store's
two, and server init takes --judge-node for each. Each is added as a
node labelled llmgw.role: judge, and the judge's profile selects that
label, with cores by init's rule over those machines. A judge beside a
busy control plane refused 38 of 60 requests under load; alone, 2.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 17:45:00 +02:00
Ilker ManapandClaude Opus 5.5 8b122aae09 A judge the leader serves by itself is sized as init's is
CI / test (push) Successful in 5m13s
A judge a policy names and the catalog lacked was served with 3 GB, no
cores and llama.cpp's prompt cache; on a busy node its judgments missed
their budget and fail: closed refused the requests. It now gets init's
2 GB and cores, and every judge modelserve serves runs with
--cache-ram 0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-19 16:32:04 +02:00
Ilker ManapandClaude Opus 5.5 2545fca174 An upgrade refits the judge an earlier server init installed
CI / test (push) Successful in 8m53s
A v0.14.6 install on fewer than eight cores refused every request until
its judge was given cores by hand, and upgrading did not change that. The
judge keeper, on the leader, now finds a judge whose entry is exactly what
init before v0.14.7 wrote: one llama.cpp CPU profile, an 8192 context and
nothing else, 2 or 4 GB, none to two cores. It gives the judge 2 GB,
--cache-ram 0 and cores by init's rule through modelsvc.SetProfiles, then
restarts its ready replicas. The replica count is left as it is. A judge
whose entry differs is left alone, and the log says so once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-16 22:06:56 +02:00
Ilker ManapandClaude Opus 5.5 b0273ffb5f Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn
Llama Guard 3 1B could not tell ordinary coding requests from harmful ones:
on the PoC it scored "continue" 0.51, "continue the code above" 0.89 and
"how do I kill all child processes in bash" 0.92 unsafe, against 0.97 to
1.00 for harmful requests, at Q4_K_M and Q8_0 alike, with its own template.
A site had "continue" refused. Qwen3Guard-Gen 0.6B gave the ordinary ones
0.15 at the most and the harmful ones 1.00, in 0.8 GB on CPUs.

- format qwen3guard: its prompt written out as its template writes it, for
  requests and answers, and its Safe / Unsafe / Controversial verdict read;
  controversial counts as the policy's judge.controversial says, safe by
  default.
- judge.models: judges asked in turn, the first that answers deciding; a
  failure is recorded against the judge that failed. guard judge use sets
  the order, replacing categories none of the judges names.
- known judges (Qwen3Guard 0.6B/4B/8B, Llama Guard 1B/8B): guard judge
  enable serves any of them, --gpu the 4B on the roomiest GPU class as a
  standard model; the leader fetches and serves one a policy names that the
  catalog lacks.
- server init installs qwen3guard-0.6b with the baseline's categories.
- a judge reads a long text by its start and end, which fit its context,
  rather than overflowing it and refusing the request under fail: closed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 10:22:47 +02:00