A judge model on the cluster: role guard in the catalog, judge in a guard policy
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
0b4c4f0698
commit
43f6a585a6
16 files changed
+803
-15
No files matched your search
@@ -92,6 +92,57 @@ guard:
|
||||
`buffer_all`, an answer that does not fit is stopped; streamed a sentence
|
||||
at a time it has gone out already, and is flagged.
|
||||
|
||||
## A judge model
|
||||
|
||||
Patterns find what has a shape. For what does not — violence, self-harm,
|
||||
abuse, whatever a moderation model was trained on — a policy can have a
|
||||
**judge model on the cluster** read the text:
|
||||
|
||||
```yaml
|
||||
# a catalog model (server.yaml models:, or ctl model add)
|
||||
- name: llama-guard-3-1b
|
||||
source: http://mirror:8081/llama-guard-3-1b/…/Llama-Guard-3-1B-Q4_K_M.gguf
|
||||
role: guard
|
||||
guard:
|
||||
format: llamaguard3 # llamaguard3, shieldgemma or json
|
||||
categories: [Violent Crimes, Non-Violent Crimes, Sex-Related Crimes, …] # names for S1, S2, …
|
||||
scaling: {min: 2, max: 2}
|
||||
profiles: [{name: cpu, engine: llamacpp}]
|
||||
|
||||
# in a guard policy
|
||||
judge:
|
||||
model: llama-guard-3-1b
|
||||
input: block # flag or block; empty does not ask about input
|
||||
output: flag
|
||||
categories: [S1, S10] # only these count; empty: any unsafe verdict
|
||||
latency_budget: 250ms # then the policy's fail: open serves (flagged), closed refuses
|
||||
```
|
||||
|
||||
A model with `role: guard` is the guardrails' alone:
|
||||
|
||||
- it is not in `/v1/models` or the chat page, and is refused (404) to
|
||||
every key, as a name and as a fallback;
|
||||
- it is `critical` by default and must always run — on demand is refused.
|
||||
Give it two replicas on different nodes, because every judged request
|
||||
waits on it.
|
||||
|
||||
The gateway's calls to it go past admission, accounting and the guardrails
|
||||
themselves.
|
||||
|
||||
- **Formats:**
|
||||
- `llamaguard3` reads Meta's "safe" / "unsafe
|
||||
S1,S10".
|
||||
- `shieldgemma` asks Google's model whether the text breaks the listed
|
||||
principles (`categories`, each a sentence) and reads Yes or No.
|
||||
- `json` expects `{"safe": false, "categories": [...]}`.
|
||||
- **What is read:**
|
||||
- **Input:** the request's last user turn, after the patterns.
|
||||
- **Output:** the answer, together with that user turn. A streamed answer
|
||||
is read every 600 characters and at the end, so text between two
|
||||
readings has gone out already; `stream: buffer_all` holds it until it
|
||||
is read.
|
||||
- **Cache:** the same text is not asked about twice within five minutes.
|
||||
|
||||
## Where a policy applies
|
||||
|
||||
`guard.default` applies everywhere. Others are attached to a part of the
|
||||
|
||||
Reference in new issue
Block a user