Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn

Llama Guard 3 1B could not tell ordinary coding requests from harmful ones:
on the PoC it scored "continue" 0.51, "continue the code above" 0.89 and
"how do I kill all child processes in bash" 0.92 unsafe, against 0.97 to
1.00 for harmful requests, at Q4_K_M and Q8_0 alike, with its own template.
A site had "continue" refused. Qwen3Guard-Gen 0.6B gave the ordinary ones
0.15 at the most and the harmful ones 1.00, in 0.8 GB on CPUs.

- format qwen3guard: its prompt written out as its template writes it, for
  requests and answers, and its Safe / Unsafe / Controversial verdict read;
  controversial counts as the policy's judge.controversial says, safe by
  default.
- judge.models: judges asked in turn, the first that answers deciding; a
  failure is recorded against the judge that failed. guard judge use sets
  the order, replacing categories none of the judges names.
- known judges (Qwen3Guard 0.6B/4B/8B, Llama Guard 1B/8B): guard judge
  enable serves any of them, --gpu the 4B on the roomiest GPU class as a
  standard model; the leader fetches and serves one a policy names that the
  catalog lacks.
- server init installs qwen3guard-0.6b with the baseline's categories.
- a judge reads a long text by its start and end, which fit its context,
  rather than overflowing it and refusing the request under fail: closed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 10:22:47 +02:00
1 parent 63f7c8cfd7
commit b0273ffb5f
24 files changed
+1313 -208

No files matched your search

+13 -3
View File
@@ -7,12 +7,14 @@ import (
"net/http/httptest"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"time"
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
"gitea.mynodes.xyz/ilker/llmgw/internal/guard"
)
func TestTheInitAgentUnitIsTheShippedUnit(t *testing.T) {
@@ -150,9 +152,17 @@ func TestTheInstalledConfigurationIsHardenedAndLoads(t *testing.T) {
if (p.Judge != nil) != judge {
t.Errorf("judge=%v: the policy's judge is %+v", judge, p.Judge)
}
if judge && (p.Judge.Model != guardModelName || p.Judge.Input != "block") {
if judge && (!slices.Equal(p.Judge.Judges(), []string{guardModelName}) || p.Judge.Input != "block") {
t.Errorf("judge %+v", p.Judge)
}
// A category the judge never names would never count.
if judge {
for _, c := range p.Judge.Categories {
if !slices.Contains(guard.DefaultCategories("qwen3guard"), c) {
t.Errorf("the baseline counts %q, which Qwen3Guard does not name", c)
}
}
}
if cfg.Auth.MaxKeyTTL != 90*24*time.Hour || cfg.Auth.SignupOpen || cfg.DebugPprof || cfg.AdminRemote {
t.Errorf("not hardened: ttl %v signup %v pprof %v remote %v",
cfg.Auth.MaxKeyTTL, cfg.Auth.SignupOpen, cfg.DebugPprof, cfg.AdminRemote)
@@ -161,13 +171,13 @@ func TestTheInstalledConfigurationIsHardenedAndLoads(t *testing.T) {
}
func TestTheGuardModelIsAValidJudge(t *testing.T) {
m := guardModel("http://10.0.0.5:8081/llama-guard-3-1b/guard/g.gguf")
m := guardModel("http://10.0.0.5:8081/qwen3guard-0.6b/guard/g.gguf")
c, err := catalog.New([]catalog.Model{m})
if err != nil {
t.Fatalf("not a valid catalog entry: %v", err)
}
got, _ := c.Resolve(guardModelName)
if !got.IsGuard() || got.Guard.Format != "llamaguard3" || len(got.Guard.Categories) != 13 || got.Scaling.Min != 2 {
if !got.IsGuard() || got.Guard.Format != "qwen3guard" || got.Scaling.Min != 2 {
t.Errorf("%+v", got)
}
for _, p := range got.Profiles {