A judge model on the cluster: role guard in the catalog, judge in a guard policy
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
0b4c4f0698
commit
43f6a585a6
16 files changed
+803
-15
No files matched your search
+10
@@ -291,6 +291,16 @@ redacted values in answers.
|
||||
fallbacks for the scope (filter in `dispatch` and `budgetFallback`).
|
||||
|
||||
### B3. Judge model on the cluster (M)
|
||||
**Done (branch `guardrails-b`):** `role: guard` + `guard.format`; hidden and
|
||||
refused to clients and fallbacks; critical by default, never on demand;
|
||||
`Router.Judge` past admission/accounting/guardrails; llamaguard3,
|
||||
shieldgemma and json parsers; 5-minute cache; latency budget then
|
||||
fail open/closed; streamed answers read every 600 characters, or held with
|
||||
buffer_all. Simpler than written: the input judge runs after admission, not
|
||||
alongside it, and shadow mode judges synchronously within the budget rather
|
||||
than after the response. Not enforced: two replicas on different nodes —
|
||||
documented, left to `scaling.min`.
|
||||
|
||||
- `catalog.Model.Role: "guard"` plus `Guard{Format: llamaguard3 |
|
||||
shieldgemma | json, Categories}`. Guard models are hidden from
|
||||
`/v1/models`, the chat picker and allowlists; placed like any model,
|
||||
|
||||
Reference in new issue
Block a user