A judge model on the cluster: role guard in the catalog, judge in a guard policy

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 15:09:31 +02:00
1 parent 0b4c4f0698
commit 43f6a585a6
16 files changed
+803 -15

No files matched your search

+10
View File
@@ -291,6 +291,16 @@ redacted values in answers.
fallbacks for the scope (filter in `dispatch` and `budgetFallback`).
### B3. Judge model on the cluster (M)
**Done (branch `guardrails-b`):** `role: guard` + `guard.format`; hidden and
refused to clients and fallbacks; critical by default, never on demand;
`Router.Judge` past admission/accounting/guardrails; llamaguard3,
shieldgemma and json parsers; 5-minute cache; latency budget then
fail open/closed; streamed answers read every 600 characters, or held with
buffer_all. Simpler than written: the input judge runs after admission, not
alongside it, and shadow mode judges synchronously within the budget rather
than after the response. Not enforced: two replicas on different nodes —
documented, left to `scaling.min`.
- `catalog.Model.Role: "guard"` plus `Guard{Format: llamaguard3 |
shieldgemma | json, Categories}`. Guard models are hidden from
`/v1/models`, the chat picker and allowlists; placed like any model,