Regenerate the configuration reference with the judge and the guard role
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d55472c00a
commit
83e1444981
3 files changed
+98
No files matched your search
@@ -67,6 +67,13 @@ rather than repeated.
|
||||
| `url` | `string` | [External](#external) |
|
||||
| `model` | `string` | [External](#external) |
|
||||
| `api_key_file` | `string` | [External](#external) |
|
||||
| `format` | `string` | [GuardModel](#guardmodel) |
|
||||
| `categories` | `[]string` | [GuardModel](#guardmodel) |
|
||||
| `model` | `string` | [JudgePolicy](#judgepolicy) |
|
||||
| `input` | `string` | [JudgePolicy](#judgepolicy) |
|
||||
| `output` | `string` | [JudgePolicy](#judgepolicy) |
|
||||
| `categories` | `[]string` | [JudgePolicy](#judgepolicy) |
|
||||
| `latency_budget` | `time.Duration` | [JudgePolicy](#judgepolicy) |
|
||||
| `contact` | `string` | [Landing](#landing) |
|
||||
| `notice` | `string` | [Landing](#landing) |
|
||||
| `bin` | `string` | [LlamaCppConfig](#llamacppconfig) |
|
||||
@@ -93,6 +100,8 @@ rather than repeated.
|
||||
| `context_length` | `int` | [Model](#model) |
|
||||
| `max_output` | `int` | [Model](#model) |
|
||||
| `capabilities` | `Capabilities` | [Model](#model) |
|
||||
| `role` | `string` | [Model](#model) |
|
||||
| `guard` | `*GuardModel` | [Model](#model) |
|
||||
| `origin` | `*Origin` | [Model](#model) |
|
||||
| `prometheus` | `string` | [Monitoring](#monitoring) |
|
||||
| `window` | `time.Duration` | [Monitoring](#monitoring) |
|
||||
@@ -127,6 +136,7 @@ rather than repeated.
|
||||
| `drop_client_system` | `bool` | [Policy](#policy) |
|
||||
| `require_capabilities` | `bool` | [Policy](#policy) |
|
||||
| `validate_structured` | `bool` | [Policy](#policy) |
|
||||
| `judge` | `*JudgePolicy` | [Policy](#policy) |
|
||||
| `stream` | `string` | [Policy](#policy) |
|
||||
| `enabled` | `bool` | [Prefix](#prefix) |
|
||||
| `max_nodes` | `int` | [Prefix](#prefix) |
|
||||
@@ -625,6 +635,56 @@ APIKeyFile holds the provider's key, on the control plane. A file
|
||||
rather than the key itself, so the key is in neither the catalog, the
|
||||
store nor the audit log; empty for a provider that asks for none.
|
||||
|
||||
## GuardModel
|
||||
|
||||
### `format`
|
||||
|
||||
`string`
|
||||
|
||||
Format is llamaguard3 (Meta's Llama Guard 3: "safe", or "unsafe" and
|
||||
the categories), shieldgemma (Google's ShieldGemma: Yes or No), or
|
||||
json ({"safe": false, "categories": ["..."]}).
|
||||
|
||||
### `categories`
|
||||
|
||||
`[]string`
|
||||
|
||||
Categories the model is asked about, for shieldgemma — each a policy
|
||||
sentence — and the names reported for llamaguard3's S1, S2, ….
|
||||
|
||||
## JudgePolicy
|
||||
|
||||
### `model`
|
||||
|
||||
`string`
|
||||
|
||||
Model is a catalog model with role: guard.
|
||||
|
||||
### `input`
|
||||
|
||||
`string`
|
||||
|
||||
Input and Output are what an unsafe verdict does: flag or block.
|
||||
Empty does not ask the judge about that direction.
|
||||
|
||||
### `output`
|
||||
|
||||
`string`
|
||||
|
||||
### `categories`
|
||||
|
||||
`[]string`
|
||||
|
||||
Categories, when set, are the only ones that count: Llama Guard's
|
||||
codes (S1, S10) or the names the model reports.
|
||||
|
||||
### `latency_budget`
|
||||
|
||||
`time.Duration`
|
||||
|
||||
LatencyBudget is how long the judge is waited for before the policy's
|
||||
fail applies. Zero is 250ms.
|
||||
|
||||
## Landing
|
||||
|
||||
### `contact`
|
||||
@@ -848,6 +908,21 @@ Capabilities is what this model can do beyond plain completion.
|
||||
|
||||
Takes a [Capabilities](#capabilities) block.
|
||||
|
||||
### `role`
|
||||
|
||||
`string`
|
||||
|
||||
Role "guard" is a model that judges the gateway's traffic for the
|
||||
guardrails rather than serving clients: hidden from /v1/models and the
|
||||
chat page, refused to any key that names it, critical by default, and
|
||||
always running. Guard says how its answers are read.
|
||||
|
||||
### `guard`
|
||||
|
||||
`*GuardModel`
|
||||
|
||||
Takes a [GuardModel](#guardmodel) block.
|
||||
|
||||
### `origin`
|
||||
|
||||
`*Origin`
|
||||
@@ -1078,6 +1153,14 @@ request asked for in response_format: one that does not fit is
|
||||
stopped — or, streamed a sentence at a time, flagged, having been
|
||||
sent already.
|
||||
|
||||
### `judge`
|
||||
|
||||
`*JudgePolicy`
|
||||
|
||||
Judge has a judge model on the cluster read the text too.
|
||||
|
||||
Takes a [JudgePolicy](#judgepolicy) block.
|
||||
|
||||
### `stream`
|
||||
|
||||
`string`
|
||||
|
||||
Reference in new issue
Block a user