The guardrails' judge has its own command, and its trouble is shown

llmgw ctl guard judge status lists each judge model the policies ask:
in the catalog, as a judge, replicas ready, its last failure to answer, and
what is wrong in words. enable serves the missing one through model serve -
Llama Guard 3 1B on CPUs, role guard, under the name the policies give it -
with --replace for a model of that name that is not a judge; the console has
it under Do. GET /api/v1/guard/judges, and the summary's judges, put the
problem on ctl status and the console's Overview.

A Llama Guard file served from the mirror page is added as a judge, under
the name the policies ask, rather than as a model clients use the policies
never find. ctl says plainly when the API answers with a page.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-06 10:47:37 +02:00
1 parent 2a942409ea
commit a8bb417618
24 files changed
+667 -13

No files matched your search

+83
View File
@@ -894,6 +894,36 @@
}
}
},
"/api/v1/guard/judges": {
"get": {
"operationId": "getGuardJudges",
"summary": "The judge models the policies ask, and what is wrong with each",
"tags": [
"guard"
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/JudgeView"
}
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/jobs": {
"get": {
"operationId": "getJobs",
@@ -4359,6 +4389,52 @@
}
}
},
"JudgeView": {
"type": "object",
"description": "JudgeView is one judge model the guardrail policies ask, and whether it can answer them.",
"properties": {
"model": {
"type": "string"
},
"policies": {
"type": "array",
"items": {
"type": "string"
},
"description": "Policies are those that ask it; Closed those that refuse what it cannot read (fail: closed)."
},
"closed": {
"type": "array",
"items": {
"type": "string"
}
},
"in_catalog": {
"type": "boolean",
"description": "InCatalog is whether a model of that name is in the catalog, and Judge whether it is there as a judge (role: guard)."
},
"judge": {
"type": "boolean"
},
"ready_replicas": {
"type": "integer"
},
"wanted_replicas": {
"type": "integer"
},
"last_error": {
"type": "string",
"description": "LastError is its last failure to give a verdict, if any."
},
"last_error_at": {
"type": "string"
},
"problem": {
"type": "string",
"description": "Problem says, in words, what is wrong and what to do; empty when nothing is."
}
}
},
"Live": {
"type": "object",
"description": "Live is one of the cluster's adapters as its node last reported it.",
@@ -5696,6 +5772,13 @@
"control_planes": {
"$ref": "#/components/schemas/ControlPlanes",
"description": "ControlPlanes is the cluster's control planes when there is more than one: which leads, which stand by, and the floating address in front."
},
"judges": {
"type": "array",
"items": {
"$ref": "#/components/schemas/JudgeView"
},
"description": "Judges are the guardrails' judge models with something wrong."
}
}
},
+17 -1
View File
@@ -103,7 +103,23 @@ guard:
Patterns find what has a shape. For what does not — violence, self-harm,
abuse, whatever a moderation model was trained on — a policy can have a
**judge model on the cluster** read the text:
**judge model on the cluster** read the text.
The judge must be in the catalog under the name the policy gives, as a
judge (`role: guard`), with a replica ready. When it is not, what the
policy judges is refused with `guard_unavailable` (`fail: closed`) or served
unjudged (`fail: open`), and `llmgw ctl status` and the console's Overview
say which judge and why. To serve it:
```
llmgw ctl guard judge status # each judge the policies ask, and what is wrong with it
llmgw ctl guard judge enable # Llama Guard 3 1B on CPUs, under the name the policies ask
llmgw ctl guard judge enable --replace # when a model of that name is there, but not as a judge
```
or **Do → Serve the guardrails' judge** in the console. A Llama Guard file
served from the mirror page is served as a judge too, under the name the
policies ask. By hand, the catalog entry is:
```yaml
# a catalog model (server.yaml models:, or ctl model add)