The guardrails' judge has its own command, and its trouble is shown
llmgw ctl guard judge status lists each judge model the policies ask: in the catalog, as a judge, replicas ready, its last failure to answer, and what is wrong in words. enable serves the missing one through model serve - Llama Guard 3 1B on CPUs, role guard, under the name the policies give it - with --replace for a model of that name that is not a judge; the console has it under Do. GET /api/v1/guard/judges, and the summary's judges, put the problem on ctl status and the console's Overview. A Llama Guard file served from the mirror page is added as a judge, under the name the policies ask, rather than as a model clients use the policies never find. ctl says plainly when the API answers with a page. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
2a942409ea
commit
a8bb417618
24 files changed
+667
-13
No files matched your search
+17
-1
@@ -103,7 +103,23 @@ guard:
|
||||
|
||||
Patterns find what has a shape. For what does not — violence, self-harm,
|
||||
abuse, whatever a moderation model was trained on — a policy can have a
|
||||
**judge model on the cluster** read the text:
|
||||
**judge model on the cluster** read the text.
|
||||
|
||||
The judge must be in the catalog under the name the policy gives, as a
|
||||
judge (`role: guard`), with a replica ready. When it is not, what the
|
||||
policy judges is refused with `guard_unavailable` (`fail: closed`) or served
|
||||
unjudged (`fail: open`), and `llmgw ctl status` and the console's Overview
|
||||
say which judge and why. To serve it:
|
||||
|
||||
```
|
||||
llmgw ctl guard judge status # each judge the policies ask, and what is wrong with it
|
||||
llmgw ctl guard judge enable # Llama Guard 3 1B on CPUs, under the name the policies ask
|
||||
llmgw ctl guard judge enable --replace # when a model of that name is there, but not as a judge
|
||||
```
|
||||
|
||||
or **Do → Serve the guardrails' judge** in the console. A Llama Guard file
|
||||
served from the mirror page is served as a judge too, under the name the
|
||||
policies ask. By hand, the catalog entry is:
|
||||
|
||||
```yaml
|
||||
# a catalog model (server.yaml models:, or ctl model add)
|
||||
|
||||
Reference in new issue
Block a user