The guardrails' judge has its own command, and its trouble is shown

llmgw ctl guard judge status lists each judge model the policies ask:
in the catalog, as a judge, replicas ready, its last failure to answer, and
what is wrong in words. enable serves the missing one through model serve -
Llama Guard 3 1B on CPUs, role guard, under the name the policies give it -
with --replace for a model of that name that is not a judge; the console has
it under Do. GET /api/v1/guard/judges, and the summary's judges, put the
problem on ctl status and the console's Overview.

A Llama Guard file served from the mirror page is added as a judge, under
the name the policies ask, rather than as a model clients use the policies
never find. ctl says plainly when the API answers with a page.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-06 10:47:37 +02:00
1 parent 2a942409ea
commit a8bb417618
24 files changed
+667 -13

No files matched your search

+6 -2
View File
@@ -59,6 +59,7 @@ const (
cAddControl
cReplicate
cServe
cJudge
cUpgrade
cRestartAll
cBackup
@@ -68,7 +69,7 @@ const (
)
var consoleLabels = [cCount]string{"Overview", "Monitor", "Nodes", "GPUs", "Models", "Replicas", "Usage", "Quotas", "Guardrails", "Federation", "Mirror", "Jobs", "Logs", "Accounts", "API keys", "Organizations", "Audit", "Manual",
"server.yaml", "Add a node", "Add a control plane", "Replicate the store", "Serve a model", "Upgrade the cluster",
"server.yaml", "Add a node", "Add a control plane", "Replicate the store", "Serve a model", "Serve the guardrails' judge", "Upgrade the cluster",
"Restart all replicas", "Back up now", "Wipe this machine", "Leave"}
// consoleAction is what is done once the screen closes: each runs its own
@@ -496,7 +497,7 @@ func (m consoleModel) key(k tea.KeyMsg) (tea.Model, tea.Cmd) {
return m.leave()
case cWipe:
return m.askWipe()
case cUpgrade, cRestartAll, cBackup:
case cUpgrade, cRestartAll, cBackup, cJudge:
return m, m.doEntry()
case cOverview:
return m, nil
@@ -1299,6 +1300,9 @@ func (m consoleModel) panel() (title, body, keys string) {
item = "\n Moves the store onto two machines under Patroni, replicated,\n so losing one loses nothing.\n\n " +
sDim.Render("On the command line:") +
"\n sudo --preserve-env=SSH_AUTH_SOCK llmgw server init \\\n --patroni <user>@<a> --patroni <user>@<b> --no-mirror --no-node --no-model"
case cJudge:
item = "\n Serves the model the guardrail policies ask as their judge — Llama\n Guard 3 1B on CPUs, under the name the policies give it — so what they\n judge is not refused (guard_unavailable). A model of that name that is\n not a judge is replaced. Overview says when a judge is missing.\n\n " +
sDim.Render("On the command line:") + "\n llmgw ctl guard judge status\n llmgw ctl guard judge enable --replace"
case cUpgrade:
item = "\n Moves the cluster to a release: this machine first, then each node\n in turn. Engines keep serving; a node that does not come back stops it.\n\n " +
sDim.Render("On the command line:") + "\n sudo llmgw ctl upgrade VERSION"