Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask
guard.enabled (unset is on) makes every policy check nothing while false, live: llmgw ctl guard off/on set it in the cluster's settings, and ctl status, ctl guard list and the guardrails page say when they are off. A refusal by a judge names the hazard it saw - judge: S6,Specialized Advice - and the verdict is kept with the request in a new usage column, guard_rule, shown by ctl request, the request log and the guardrails page. llmgw ctl guard judge ask has a judge read a text, past the cache, and shows its raw answer and how it is read; the endpoint is kept out of the audit so no text is recorded. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
3f5a809373
commit
249a91aeec
34 files changed
+505
-33
No files matched your search
@@ -924,6 +924,48 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/guard/judges/ask": {
|
||||
"post": {
|
||||
"operationId": "postGuardJudgesAsk",
|
||||
"summary": "Have a judge read a text and say what it answered; nothing is kept",
|
||||
"tags": [
|
||||
"guard"
|
||||
],
|
||||
"parameters": [
|
||||
{
|
||||
"$ref": "#/components/parameters/Actor"
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/JudgeAskRequest"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/JudgeAskView"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"401": {
|
||||
"$ref": "#/components/responses/Unauthorized"
|
||||
},
|
||||
"default": {
|
||||
"$ref": "#/components/responses/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/jobs": {
|
||||
"get": {
|
||||
"operationId": "getJobs",
|
||||
@@ -4129,6 +4171,10 @@
|
||||
"category": {
|
||||
"type": "string"
|
||||
},
|
||||
"rule": {
|
||||
"type": "string",
|
||||
"description": "Rule is what decided it within the category: a pattern's name, or a judge's verdict, \"S6,Specialized Advice\"."
|
||||
},
|
||||
"guard_ms": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
@@ -4191,6 +4237,10 @@
|
||||
"type": "object",
|
||||
"description": "GuardView is the library and where its policies are attached.",
|
||||
"properties": {
|
||||
"enabled": {
|
||||
"type": "boolean",
|
||||
"description": "Enabled is whether the guardrails check anything (guard.enabled)."
|
||||
},
|
||||
"policies": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
@@ -4389,6 +4439,50 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"JudgeAskRequest": {
|
||||
"type": "object",
|
||||
"description": "JudgeAskRequest is a text to have a judge read.",
|
||||
"properties": {
|
||||
"model": {
|
||||
"type": "string",
|
||||
"description": "Model is the judge; empty is the one the policies ask, when they ask one."
|
||||
},
|
||||
"text": {
|
||||
"type": "string"
|
||||
},
|
||||
"answer": {
|
||||
"type": "string",
|
||||
"description": "Answer, when given, is read as the assistant's reply to Text."
|
||||
}
|
||||
}
|
||||
},
|
||||
"JudgeAskView": {
|
||||
"type": "object",
|
||||
"description": "JudgeAskView is what the judge answered.",
|
||||
"properties": {
|
||||
"model": {
|
||||
"type": "string"
|
||||
},
|
||||
"verdict": {
|
||||
"type": "string",
|
||||
"description": "Verdict is safe or unsafe, as the gateway reads the answer."
|
||||
},
|
||||
"categories": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"raw": {
|
||||
"type": "string",
|
||||
"description": "Raw is the judge's answer as it gave it."
|
||||
},
|
||||
"took_ms": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
}
|
||||
}
|
||||
},
|
||||
"JudgeView": {
|
||||
"type": "object",
|
||||
"description": "JudgeView is one judge model the guardrail policies ask, and whether it can answer them.",
|
||||
@@ -5401,6 +5495,9 @@
|
||||
"guard_category": {
|
||||
"type": "string"
|
||||
},
|
||||
"guard_rule": {
|
||||
"type": "string"
|
||||
},
|
||||
"guard_ms": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
@@ -5779,6 +5876,10 @@
|
||||
"$ref": "#/components/schemas/JudgeView"
|
||||
},
|
||||
"description": "Judges are the guardrails' judge models with something wrong."
|
||||
},
|
||||
"guard_off": {
|
||||
"type": "boolean",
|
||||
"description": "GuardOff says the guardrails are turned off (guard.enabled: false): no request is checked, whatever the policies say."
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
+33
-1
@@ -15,6 +15,19 @@ checked and nothing costs anything. With one, every request is judged once it is
|
||||
within its limits, with its stored conversation spliced in, and before it is
|
||||
sent to a model; and the answer is judged before the client sees it.
|
||||
|
||||
## Turning them off and on
|
||||
|
||||
```
|
||||
llmgw ctl guard off --note "why" # no request is checked, whatever the policies say
|
||||
llmgw ctl guard on # every policy checks again, as it was
|
||||
```
|
||||
|
||||
That is `guard.enabled` in the cluster's settings, in force on every control
|
||||
plane within seconds without a restart. The policies and where they are
|
||||
attached are kept. While off, `llmgw ctl status`, `llmgw ctl guard list` and
|
||||
the guardrails page say so. To stop one policy rather than all, give it
|
||||
`mode: off`, or `mode: shadow` to have it record what it would do.
|
||||
|
||||
## A policy
|
||||
|
||||
```yaml
|
||||
@@ -117,7 +130,26 @@ llmgw ctl guard judge enable # Llama Guard 3 1B on CPUs, under the nam
|
||||
llmgw ctl guard judge enable --replace # when a model of that name is there, but not as a judge
|
||||
```
|
||||
|
||||
or **Do → Serve the guardrails' judge** in the console. A Llama Guard file
|
||||
or **Do → Serve the guardrails' judge** in the console.
|
||||
|
||||
**When it refuses ordinary requests.** A refusal by a judge names the hazard
|
||||
it saw — the client is told `refused by the gateway's policy (judge:
|
||||
S6,Specialized Advice)` — and the same is kept with the request: `llmgw ctl
|
||||
request <id>`, the server's log (`guard_rule`) and the guardrails page show
|
||||
it. To see what the judge says of any text, without anything being kept:
|
||||
|
||||
```
|
||||
llmgw ctl guard judge ask 'How do I sort a list in Python?'
|
||||
# llama-guard-3-1b: safe, 140 ms
|
||||
# it answered: "safe"
|
||||
```
|
||||
|
||||
A judge that says `unsafe` to ordinary text is usually the wrong weights
|
||||
(another repository's or quantisation's, without Llama Guard's template:
|
||||
`judge enable --replace` serves the right ones), or a policy counting every
|
||||
hazard: give its judge `categories`, leaving out S5, S6, S8 and S13 as the
|
||||
baseline does. While finding out, `judge.input: flag` records instead of
|
||||
refusing. A Llama Guard file
|
||||
served from the mirror page is served as a judge too, under the name the
|
||||
policies ask. By hand, the catalog entry is:
|
||||
|
||||
|
||||
@@ -372,6 +372,7 @@ rather than repeated.
|
||||
| `llamacpp` | `LlamaCppConfig` | [engines.Config](#enginesconfig) |
|
||||
| `trtllm` | `TRTLLMConfig` | [engines.Config](#enginesconfig) |
|
||||
| `sim` | `SimConfig` | [engines.Config](#enginesconfig) |
|
||||
| `enabled` | `*bool` | [guard.Config](#guardconfig) |
|
||||
| `default` | `[]string` | [guard.Config](#guardconfig) |
|
||||
| `policies` | `map[string]Policy` | [guard.Config](#guardconfig) |
|
||||
| `webhooks` | `[]Webhook` | [notify.Config](#notifyconfig) |
|
||||
@@ -3067,6 +3068,13 @@ Takes a [SimConfig](#simconfig) block.
|
||||
|
||||
## guard.Config
|
||||
|
||||
### `enabled`
|
||||
|
||||
`*bool`
|
||||
|
||||
Enabled turns every policy on or off at once: false checks nothing,
|
||||
whatever is defined or attached. Unset is on.
|
||||
|
||||
### `default`
|
||||
|
||||
`[]string`
|
||||
|
||||
Reference in new issue
Block a user