The guardrails' judge has its own command, and its trouble is shown

llmgw ctl guard judge status lists each judge model the policies ask:
in the catalog, as a judge, replicas ready, its last failure to answer, and
what is wrong in words. enable serves the missing one through model serve -
Llama Guard 3 1B on CPUs, role guard, under the name the policies give it -
with --replace for a model of that name that is not a judge; the console has
it under Do. GET /api/v1/guard/judges, and the summary's judges, put the
problem on ctl status and the console's Overview.

A Llama Guard file served from the mirror page is added as a judge, under
the name the policies ask, rather than as a model clients use the policies
never find. ctl says plainly when the API answers with a page.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-06 10:47:37 +02:00
1 parent 2a942409ea
commit a8bb417618
24 files changed
+667 -13

No files matched your search

+23
View File
@@ -5,6 +5,7 @@ import (
"encoding/json"
"fmt"
"log/slog"
"maps"
"regexp"
"sort"
"strings"
@@ -43,6 +44,9 @@ type Gate struct {
m *gateMetrics
// alarm is told each time a judge gives no verdict; nil tells no one.
alarm func(model string, err error)
// lastErr is each judge's last failure to give a verdict.
errMu sync.Mutex
lastErr map[string]JudgeError
mu sync.Mutex
attached map[string][]string // "scope/id" → policy names
@@ -448,11 +452,30 @@ func (g *Gate) Input(ctx context.Context, s Subject, body []byte) (Verdict, erro
func (g *Gate) SetJudgeAlarm(f func(model string, err error)) { g.alarm = f }
func (g *Gate) judgeFailed(model string, err error) {
g.errMu.Lock()
if g.lastErr == nil {
g.lastErr = map[string]JudgeError{}
}
g.lastErr[model] = JudgeError{Err: err.Error(), At: g.now()}
g.errMu.Unlock()
if g.alarm != nil {
g.alarm(model, err)
}
}
// JudgeError is a judge's last failure to give a verdict.
type JudgeError struct {
Err string
At time.Time
}
// JudgeErrors are each judge's last failure to give a verdict, by model.
func (g *Gate) JudgeErrors() map[string]JudgeError {
g.errMu.Lock()
defer g.errMu.Unlock()
return maps.Clone(g.lastErr)
}
func judgeFinding(p Policy, action string, res judgment, err error) finding {
if err != nil {
// Unanswered: closed stops it, open lets it through, said.