Compare commits

...
3 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 f285aa0548 v0.12.2
Release / build (push) Successful in 4m41s
CI / test (push) Successful in 4m43s
Release / notes (push) Successful in 3s
Guardrails off and on; a judge's refusal names the hazard; ctl guard judge ask.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 13:35:16 +02:00
Ilker ManapandClaude Opus 5.5 4721b8e353 Llama Guard 3's verdicts carry their hazards' names without a categories list
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 12:44:53 +02:00
Ilker ManapandClaude Opus 5.5 249a91aeec Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask
guard.enabled (unset is on) makes every policy check nothing while false,
live: llmgw ctl guard off/on set it in the cluster's settings, and ctl
status, ctl guard list and the guardrails page say when they are off.

A refusal by a judge names the hazard it saw - judge: S6,Specialized
Advice - and the verdict is kept with the request in a new usage column,
guard_rule, shown by ctl request, the request log and the guardrails page.

llmgw ctl guard judge ask has a judge read a text, past the cache, and shows
its raw answer and how it is read; the endpoint is kept out of the audit so
no text is recorded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-06 12:41:08 +02:00
34 changed files with 515 additions and 33 deletions

No files matched your search

+25
View File
@@ -22,6 +22,31 @@ is only released once its **exit criterion** is met, not when its code lands.
Until v1.0.0 the minor version is the breaking one: configuration formats and Go Until v1.0.0 the minor version is the breaking one: configuration formats and Go
APIs change between minors without a deprecation period. APIs change between minors without a deprecation period.
## v0.12.2 — 2026-10-06
The guardrails turned off and on as a whole, and a judge that refuses
ordinary requests explained: the hazard it saw is told to the client and
kept with the request, and `llmgw ctl guard judge ask` shows what it says of
any text. Nothing to do to upgrade; the store gains a column.
### Added
- **The guardrails turn off and on as a whole**: `llmgw ctl guard off` /
`on`, which is `guard.enabled` in the cluster's settings — in force within
seconds, the policies and their attachments kept. `ctl status`, `ctl guard
list` and the guardrails page say when they are off.
- **`llmgw ctl guard judge ask TEXT`** has the judge read a text and shows
its answer as given and as read — for finding out why ordinary requests
are refused. Nothing of the text is kept, not even in the audit.
### Changed
- **A judge's refusal names the hazard**: the client is told `(judge:
S6,Specialized Advice)` rather than `(judge)`, and the verdict is kept with
the request (`guard_rule`): `ctl request`, the server's log and the
guardrails page show it. Llama Guard 3's codes come with their names,
S9 Indiscriminate Weapons, without a categories list.
## v0.12.1 — 2026-10-06 ## v0.12.1 — 2026-10-06
The guardrails' judge, served and watched without server init: a judge The guardrails' judge, served and watched without server init: a judge
+101
View File
@@ -924,6 +924,48 @@
} }
} }
}, },
"/api/v1/guard/judges/ask": {
"post": {
"operationId": "postGuardJudgesAsk",
"summary": "Have a judge read a text and say what it answered; nothing is kept",
"tags": [
"guard"
],
"parameters": [
{
"$ref": "#/components/parameters/Actor"
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/JudgeAskRequest"
}
}
}
},
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/JudgeAskView"
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/jobs": { "/api/v1/jobs": {
"get": { "get": {
"operationId": "getJobs", "operationId": "getJobs",
@@ -4129,6 +4171,10 @@
"category": { "category": {
"type": "string" "type": "string"
}, },
"rule": {
"type": "string",
"description": "Rule is what decided it within the category: a pattern's name, or a judge's verdict, \"S6,Specialized Advice\"."
},
"guard_ms": { "guard_ms": {
"type": "integer", "type": "integer",
"format": "int64" "format": "int64"
@@ -4191,6 +4237,10 @@
"type": "object", "type": "object",
"description": "GuardView is the library and where its policies are attached.", "description": "GuardView is the library and where its policies are attached.",
"properties": { "properties": {
"enabled": {
"type": "boolean",
"description": "Enabled is whether the guardrails check anything (guard.enabled)."
},
"policies": { "policies": {
"type": "array", "type": "array",
"items": { "items": {
@@ -4389,6 +4439,50 @@
} }
} }
}, },
"JudgeAskRequest": {
"type": "object",
"description": "JudgeAskRequest is a text to have a judge read.",
"properties": {
"model": {
"type": "string",
"description": "Model is the judge; empty is the one the policies ask, when they ask one."
},
"text": {
"type": "string"
},
"answer": {
"type": "string",
"description": "Answer, when given, is read as the assistant's reply to Text."
}
}
},
"JudgeAskView": {
"type": "object",
"description": "JudgeAskView is what the judge answered.",
"properties": {
"model": {
"type": "string"
},
"verdict": {
"type": "string",
"description": "Verdict is safe or unsafe, as the gateway reads the answer."
},
"categories": {
"type": "array",
"items": {
"type": "string"
}
},
"raw": {
"type": "string",
"description": "Raw is the judge's answer as it gave it."
},
"took_ms": {
"type": "integer",
"format": "int64"
}
}
},
"JudgeView": { "JudgeView": {
"type": "object", "type": "object",
"description": "JudgeView is one judge model the guardrail policies ask, and whether it can answer them.", "description": "JudgeView is one judge model the guardrail policies ask, and whether it can answer them.",
@@ -5401,6 +5495,9 @@
"guard_category": { "guard_category": {
"type": "string" "type": "string"
}, },
"guard_rule": {
"type": "string"
},
"guard_ms": { "guard_ms": {
"type": "integer", "type": "integer",
"format": "int64" "format": "int64"
@@ -5779,6 +5876,10 @@
"$ref": "#/components/schemas/JudgeView" "$ref": "#/components/schemas/JudgeView"
}, },
"description": "Judges are the guardrails' judge models with something wrong." "description": "Judges are the guardrails' judge models with something wrong."
},
"guard_off": {
"type": "boolean",
"description": "GuardOff says the guardrails are turned off (guard.enabled: false): no request is checked, whatever the policies say."
} }
} }
}, },
+33 -1
View File
@@ -15,6 +15,19 @@ checked and nothing costs anything. With one, every request is judged once it is
within its limits, with its stored conversation spliced in, and before it is within its limits, with its stored conversation spliced in, and before it is
sent to a model; and the answer is judged before the client sees it. sent to a model; and the answer is judged before the client sees it.
## Turning them off and on
```
llmgw ctl guard off --note "why" # no request is checked, whatever the policies say
llmgw ctl guard on # every policy checks again, as it was
```
That is `guard.enabled` in the cluster's settings, in force on every control
plane within seconds without a restart. The policies and where they are
attached are kept. While off, `llmgw ctl status`, `llmgw ctl guard list` and
the guardrails page say so. To stop one policy rather than all, give it
`mode: off`, or `mode: shadow` to have it record what it would do.
## A policy ## A policy
```yaml ```yaml
@@ -117,7 +130,26 @@ llmgw ctl guard judge enable # Llama Guard 3 1B on CPUs, under the nam
llmgw ctl guard judge enable --replace # when a model of that name is there, but not as a judge llmgw ctl guard judge enable --replace # when a model of that name is there, but not as a judge
``` ```
or **Do → Serve the guardrails' judge** in the console. A Llama Guard file or **Do → Serve the guardrails' judge** in the console.
**When it refuses ordinary requests.** A refusal by a judge names the hazard
it saw — the client is told `refused by the gateway's policy (judge:
S6,Specialized Advice)` — and the same is kept with the request: `llmgw ctl
request <id>`, the server's log (`guard_rule`) and the guardrails page show
it. To see what the judge says of any text, without anything being kept:
```
llmgw ctl guard judge ask 'How do I sort a list in Python?'
# llama-guard-3-1b: safe, 140 ms
# it answered: "safe"
```
A judge that says `unsafe` to ordinary text is usually the wrong weights
(another repository's or quantisation's, without Llama Guard's template:
`judge enable --replace` serves the right ones), or a policy counting every
hazard: give its judge `categories`, leaving out S5, S6, S8 and S13 as the
baseline does. While finding out, `judge.input: flag` records instead of
refusing. A Llama Guard file
served from the mirror page is served as a judge too, under the name the served from the mirror page is served as a judge too, under the name the
policies ask. By hand, the catalog entry is: policies ask. By hand, the catalog entry is:
+8
View File
@@ -372,6 +372,7 @@ rather than repeated.
| `llamacpp` | `LlamaCppConfig` | [engines.Config](#enginesconfig) | | `llamacpp` | `LlamaCppConfig` | [engines.Config](#enginesconfig) |
| `trtllm` | `TRTLLMConfig` | [engines.Config](#enginesconfig) | | `trtllm` | `TRTLLMConfig` | [engines.Config](#enginesconfig) |
| `sim` | `SimConfig` | [engines.Config](#enginesconfig) | | `sim` | `SimConfig` | [engines.Config](#enginesconfig) |
| `enabled` | `*bool` | [guard.Config](#guardconfig) |
| `default` | `[]string` | [guard.Config](#guardconfig) | | `default` | `[]string` | [guard.Config](#guardconfig) |
| `policies` | `map[string]Policy` | [guard.Config](#guardconfig) | | `policies` | `map[string]Policy` | [guard.Config](#guardconfig) |
| `webhooks` | `[]Webhook` | [notify.Config](#notifyconfig) | | `webhooks` | `[]Webhook` | [notify.Config](#notifyconfig) |
@@ -3067,6 +3068,13 @@ Takes a [SimConfig](#simconfig) block.
## guard.Config ## guard.Config
### `enabled`
`*bool`
Enabled turns every policy on or off at once: false checks nothing,
whatever is defined or attached. Unset is on.
### `default` ### `default`
`[]string` `[]string`
+7 -1
View File
@@ -12,6 +12,7 @@ import (
"net/http" "net/http"
"net/url" "net/url"
"regexp" "regexp"
"slices"
"sort" "sort"
"strings" "strings"
"time" "time"
@@ -110,11 +111,16 @@ func (w *statusWriter) Flush() {
} }
} }
// notChanges are POSTs that change nothing and carry text that must not be
// kept: asking a judge about a sentence.
var notChanges = []string{"/api/v1/guard/judges/ask"}
// API records every change to the admin API: a request that is not a GET or // API records every change to the admin API: a request that is not a GET or
// HEAD, under /api/. actor says who made it. // HEAD, under /api/. actor says who made it.
func API(sink Sink, actor func(*http.Request) string, log *slog.Logger, next http.Handler) http.Handler { func API(sink Sink, actor func(*http.Request) string, log *slog.Logger, next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodGet || r.Method == http.MethodHead || !strings.HasPrefix(r.URL.Path, "/api/") { if r.Method == http.MethodGet || r.Method == http.MethodHead || !strings.HasPrefix(r.URL.Path, "/api/") ||
slices.Contains(notChanges, r.URL.Path) {
next.ServeHTTP(w, r) next.ServeHTTP(w, r)
return return
} }
+10
View File
@@ -72,3 +72,13 @@ func TestABodyLargerThanWhatIsRecordedReachesTheHandlerWhole(t *testing.T) {
t.Errorf("the handler was given %d of %d bytes", got, len(big)) t.Errorf("the handler was given %d of %d bytes", got, len(big))
} }
} }
func TestAskingAJudgeKeepsNoText(t *testing.T) {
sink := &memSink{}
h := API(sink, func(*http.Request) string { return "ilker" }, nil, http.HandlerFunc(func(http.ResponseWriter, *http.Request) {}))
h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("POST", "/api/v1/guard/judges/ask",
strings.NewReader(`{"text":"something a user wrote"}`)))
if len(sink.got) != 0 {
t.Errorf("a judge's question was recorded: %+v", sink.got)
}
}
+5 -2
View File
@@ -76,8 +76,11 @@ func newRequestCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
fmt.Fprintf(out, "tags %s\n", strings.Join(kv, ", ")) fmt.Fprintf(out, "tags %s\n", strings.Join(kv, ", "))
} }
if v.GuardAction != "" { if v.GuardAction != "" {
fmt.Fprintf(out, "guard %s by policy %s (%s), %d ms\n", v.GuardAction, v.GuardPolicy, what := or(v.GuardCategory, "-")
or(v.GuardCategory, "-"), v.GuardMs) if v.GuardRule != "" {
what += ": " + v.GuardRule
}
fmt.Fprintf(out, "guard %s by policy %s (%s), %d ms\n", v.GuardAction, v.GuardPolicy, what, v.GuardMs)
} else if v.GuardMs > 0 { } else if v.GuardMs > 0 {
fmt.Fprintf(out, "guard allowed, %d ms\n", v.GuardMs) fmt.Fprintf(out, "guard allowed, %d ms\n", v.GuardMs)
} }
+33
View File
@@ -3,11 +3,13 @@ package cli
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"strconv"
"strings" "strings"
"github.com/spf13/cobra" "github.com/spf13/cobra"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi" "gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
"gitea.mynodes.xyz/ilker/llmgw/internal/settings"
) )
func newGuardCmd(client func() *ctlClient, asJSON *bool) *cobra.Command { func newGuardCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
@@ -25,6 +27,34 @@ func newGuardCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
"with IDs), key:KEY-ID or user:USER-ID. An attachment applies at once.", "with IDs), key:KEY-ID or user:USER-ID. An attachment applies at once.",
} }
c.AddCommand(newGuardJudgeCmd(client, asJSON)) c.AddCommand(newGuardJudgeCmd(client, asJSON))
for _, sw := range []struct {
use, short string
on bool
}{
{"on", "Turn the guardrails on: every policy checks again", true},
{"off", "Turn the guardrails off: no request is checked, whatever the policies say", false},
} {
var note string
cmd := &cobra.Command{
Use: sw.use,
Short: sw.short,
Long: "guard.enabled in the cluster's settings: in force on every control plane\n" +
"within seconds, without a restart. The policies and where they are attached\n" +
"stay as they are, and come back as they were when turned on again. While\n" +
"off, llmgw ctl status says so.",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
if note == "" {
note = "guardrails " + sw.use
}
return changeSettings(cmd, client, note, func(doc *settings.Doc) error {
return doc.SetYAML([]string{"guard", "enabled"}, strconv.FormatBool(sw.on))
})
},
}
cmd.Flags().StringVar(&note, "note", "", "why, kept with the settings' version")
c.AddCommand(cmd)
}
c.AddCommand(&cobra.Command{ c.AddCommand(&cobra.Command{
Use: "list", Use: "list",
Short: "The policies, and where each is attached", Short: "The policies, and where each is attached",
@@ -38,6 +68,9 @@ func newGuardCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
return json.NewEncoder(cmd.OutOrStdout()).Encode(v) return json.NewEncoder(cmd.OutOrStdout()).Encode(v)
} }
out := cmd.OutOrStdout() out := cmd.OutOrStdout()
if !v.Enabled {
fmt.Fprintln(out, "The guardrails are OFF: no request is checked (llmgw ctl guard on).")
}
fmt.Fprintf(out, "Policies: %s\n", strings.Join(v.Policies, ", ")) fmt.Fprintf(out, "Policies: %s\n", strings.Join(v.Policies, ", "))
def := strings.Join(v.Default, ", ") def := strings.Join(v.Default, ", ")
if def == "" { if def == "" {
+37 -1
View File
@@ -29,7 +29,7 @@ func newGuardJudgeCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
"guard), with a replica ready; otherwise what the policy judges is refused\n" + "guard), with a replica ready; otherwise what the policy judges is refused\n" +
"(fail: closed, guard_unavailable) or served unjudged (fail: open).", "(fail: closed, guard_unavailable) or served unjudged (fail: open).",
} }
c.AddCommand(newGuardJudgeStatusCmd(client, asJSON), newGuardJudgeEnableCmd(client)) c.AddCommand(newGuardJudgeStatusCmd(client, asJSON), newGuardJudgeEnableCmd(client), newGuardJudgeAskCmd(client, asJSON))
return c return c
} }
@@ -187,3 +187,39 @@ func newGuardJudgeEnableCmd(client func() *ctlClient) *cobra.Command {
f.BoolVar(&detach, "detach", false, "start it and return; llmgw ctl jobs follows it") f.BoolVar(&detach, "detach", false, "start it and return; llmgw ctl jobs follows it")
return c return c
} }
func newGuardJudgeAskCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
var model, answer string
c := &cobra.Command{
Use: "ask TEXT",
Short: "Have a judge read a text and say what it answered",
Long: "For finding out why a request was refused: the judge reads TEXT as a user's\n" +
"message (with --answer, ANSWER as the reply to it), past the cache, and its\n" +
"answer is shown as it gave it and as the gateway reads it — \"unsafe\" and\n" +
"the hazards it named, S6 Specialized Advice and the like. Nothing is kept:\n" +
"not in the audit, the usage or the logs.",
Example: " llmgw ctl guard judge ask 'How do I sort a list in Python?'\n" +
" llmgw ctl guard judge ask 'Write a poem' --answer 'Roses are red'",
Args: cobra.ExactArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
body, _ := json.Marshal(adminapi.JudgeAskRequest{Model: model, Text: args[0], Answer: answer})
var v adminapi.JudgeAskView
if err := client().patient(45*time.Second).post(cmd.Context(), "/api/v1/guard/judges/ask", body, &v); err != nil {
return err
}
if *asJSON {
return emitJSON(v)
}
out := cmd.OutOrStdout()
fmt.Fprintf(out, "%s: %s", v.Model, v.Verdict)
if len(v.Categories) > 0 {
fmt.Fprintf(out, " (%s)", strings.Join(v.Categories, ", "))
}
fmt.Fprintf(out, ", %d ms\nit answered: %q\n", v.TookMs, v.Raw)
return nil
},
}
c.Flags().StringVar(&model, "model", "", "the judge (default: the one the policies ask)")
c.Flags().StringVar(&answer, "answer", "", "an assistant's reply to TEXT, judged as output is")
return c
}
+1 -1
View File
@@ -362,7 +362,7 @@ func newServerCmd() *cobra.Command {
api.SetEval(evals) api.SetEval(evals)
if st := rt.Store(); st != nil && gate != nil { if st := rt.Store(); st != nil && gate != nil {
api.SetGuard(&adminapi.GuardAdmin{Library: gate.Library, Store: st, Invalidate: gate.Invalidate, api.SetGuard(&adminapi.GuardAdmin{Library: gate.Library, Store: st, Invalidate: gate.Invalidate,
JudgeErrors: gate.JudgeErrors}) JudgeErrors: gate.JudgeErrors, Ask: gate.AskJudge})
} }
if st := rt.Store(); st != nil { if st := rt.Store(); st != nil {
// Removing a node revokes its certificate. // Removing a node revokes its certificate.
+3
View File
@@ -153,6 +153,9 @@ func printSummary(cmd *cobra.Command, s adminapi.Summary) {
fmt.Fprintf(out, "STORE %s\n", strings.Join(parts, ", ")) fmt.Fprintf(out, "STORE %s\n", strings.Join(parts, ", "))
} }
if s.GuardOff {
fmt.Fprintln(out, "GUARD off: no request is checked, whatever the policies say (llmgw ctl guard on)")
}
if len(s.Judges) > 0 { if len(s.Judges) > 0 {
fmt.Fprintln(out, "\nGUARDRAILS' JUDGE") fmt.Fprintln(out, "\nGUARDRAILS' JUDGE")
for _, j := range s.Judges { for _, j := range s.Judges {
+2 -1
View File
@@ -1545,6 +1545,7 @@ type RequestView struct {
GuardAction string `json:"guard_action,omitempty"` GuardAction string `json:"guard_action,omitempty"`
GuardPolicy string `json:"guard_policy,omitempty"` GuardPolicy string `json:"guard_policy,omitempty"`
GuardCategory string `json:"guard_category,omitempty"` GuardCategory string `json:"guard_category,omitempty"`
GuardRule string `json:"guard_rule,omitempty"`
GuardMs int64 `json:"guard_ms,omitempty"` GuardMs int64 `json:"guard_ms,omitempty"`
// What the client said about it: OpenAI's user field, and its tags. // What the client said about it: OpenAI's user field, and its tags.
EndUser string `json:"end_user,omitempty"` EndUser string `json:"end_user,omitempty"`
@@ -1573,7 +1574,7 @@ func (a *API) request(w http.ResponseWriter, r *http.Request) {
Model: u.Model, ServedModel: u.ServedModel, Replica: u.Replica, Status: u.Status, Model: u.Model, ServedModel: u.ServedModel, Replica: u.Replica, Status: u.Status,
LatencyMs: u.LatencyMs, QueueMs: u.QueueMs, TTFTMs: u.TTFTMs, LatencyMs: u.LatencyMs, QueueMs: u.QueueMs, TTFTMs: u.TTFTMs,
PromptTokens: u.PromptTokens, CompletionTokens: u.CompletionTokens, CostMicros: u.CostMicros, PromptTokens: u.PromptTokens, CompletionTokens: u.CompletionTokens, CostMicros: u.CostMicros,
GuardAction: u.GuardAction, GuardPolicy: u.GuardPolicy, GuardCategory: u.GuardCategory, GuardMs: u.GuardMs, GuardAction: u.GuardAction, GuardPolicy: u.GuardPolicy, GuardCategory: u.GuardCategory, GuardRule: u.GuardRule, GuardMs: u.GuardMs,
EndUser: u.EndUser, Tags: u.Tags, CacheHit: u.CacheHit, EndUser: u.EndUser, Tags: u.Tags, CacheHit: u.CacheHit,
}) })
} }
+11 -3
View File
@@ -30,6 +30,8 @@ type GuardAdmin struct {
Invalidate func() Invalidate func()
// JudgeErrors are each judge's last failure to give a verdict. // JudgeErrors are each judge's last failure to give a verdict.
JudgeErrors func() map[string]guard.JudgeError JudgeErrors func() map[string]guard.JudgeError
// Ask has a judge read a text, past the cache.
Ask func(ctx context.Context, model, text, answer string) (guard.JudgeAnswer, error)
} }
// SetGuard enables the guard endpoints. // SetGuard enables the guard endpoints.
@@ -37,6 +39,8 @@ func (a *API) SetGuard(g *GuardAdmin) { a.guard = g }
// GuardView is the library and where its policies are attached. // GuardView is the library and where its policies are attached.
type GuardView struct { type GuardView struct {
// Enabled is whether the guardrails check anything (guard.enabled).
Enabled bool `json:"enabled"`
Policies []string `json:"policies"` Policies []string `json:"policies"`
Default []string `json:"default"` Default []string `json:"default"`
Attachments []GuardAttachmentView `json:"attachments"` Attachments []GuardAttachmentView `json:"attachments"`
@@ -91,7 +95,10 @@ type GuardEventView struct {
Action string `json:"action"` Action string `json:"action"`
Policy string `json:"policy"` Policy string `json:"policy"`
Category string `json:"category,omitempty"` Category string `json:"category,omitempty"`
GuardMs int64 `json:"guard_ms"` // Rule is what decided it within the category: a pattern's name, or a
// judge's verdict, "S6,Specialized Advice".
Rule string `json:"rule,omitempty"`
GuardMs int64 `json:"guard_ms"`
} }
// policyView sums one policy up. // policyView sums one policy up.
@@ -201,6 +208,7 @@ type GuardAttachmentRequest struct {
func (a *API) registerGuard(mux Mux) { func (a *API) registerGuard(mux Mux) {
mux.HandleFunc("GET /api/v1/guard/judges", a.listJudges) mux.HandleFunc("GET /api/v1/guard/judges", a.listJudges)
mux.HandleFunc("POST /api/v1/guard/judges/ask", a.askJudge)
mux.HandleFunc("GET /api/v1/guard", func(w http.ResponseWriter, r *http.Request) { mux.HandleFunc("GET /api/v1/guard", func(w http.ResponseWriter, r *http.Request) {
if a.guard == nil { if a.guard == nil {
writeError(w, http.StatusNotImplemented, "no guard policies: define them under guard in the cluster's settings (llmgw ctl config edit)") writeError(w, http.StatusNotImplemented, "no guard policies: define them under guard in the cluster's settings (llmgw ctl config edit)")
@@ -211,7 +219,7 @@ func (a *API) registerGuard(mux Mux) {
writeError(w, http.StatusInternalServerError, err.Error()) writeError(w, http.StatusInternalServerError, err.Error())
return return
} }
v := GuardView{Default: a.guard.Library().Default, Attachments: []GuardAttachmentView{}} v := GuardView{Enabled: a.guard.Library().On(), Default: a.guard.Library().Default, Attachments: []GuardAttachmentView{}}
var names map[string]string var names map[string]string
if a.quotas != nil { if a.quotas != nil {
names, _ = a.scopeNames(r.Context()) names, _ = a.scopeNames(r.Context())
@@ -272,7 +280,7 @@ func (a *API) registerGuard(mux Mux) {
slices.SortStableFunc(v.Counts, func(a, b GuardCountView) int { return int(b.Count - a.Count) }) slices.SortStableFunc(v.Counts, func(a, b GuardCountView) int { return int(b.Count - a.Count) })
for _, e := range events { for _, e := range events {
v.Events = append(v.Events, GuardEventView{At: e.At, RequestID: e.RequestID, Project: project(e.ProjectID), v.Events = append(v.Events, GuardEventView{At: e.At, RequestID: e.RequestID, Project: project(e.ProjectID),
Model: e.Model, Status: e.Status, Action: e.Action, Policy: e.Policy, Category: e.Category, GuardMs: e.GuardMs}) Model: e.Model, Status: e.Status, Action: e.Action, Policy: e.Policy, Category: e.Category, Rule: e.Rule, GuardMs: e.GuardMs})
} }
writeJSON(w, http.StatusOK, v) writeJSON(w, http.StatusOK, v)
}) })
+57
View File
@@ -1,10 +1,12 @@
package adminapi package adminapi
import ( import (
"encoding/json"
"fmt" "fmt"
"net/http" "net/http"
"slices" "slices"
"sort" "sort"
"strings"
"time" "time"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler" "gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
@@ -44,6 +46,10 @@ func (a *API) Judges() []JudgeView {
return nil return nil
} }
lib := a.guard.Library() lib := a.guard.Library()
if !lib.On() {
// Off: no judge is asked, so none is missing.
return nil
}
byModel := map[string]*JudgeView{} byModel := map[string]*JudgeView{}
for name, p := range lib.Policies { for name, p := range lib.Policies {
if p.Judge == nil || p.Judge.Model == "" { if p.Judge == nil || p.Judge.Model == "" {
@@ -122,3 +128,54 @@ func (a *API) listJudges(w http.ResponseWriter, _ *http.Request) {
} }
writeJSON(w, http.StatusOK, a.Judges()) writeJSON(w, http.StatusOK, a.Judges())
} }
// JudgeAskRequest is a text to have a judge read.
type JudgeAskRequest struct {
// Model is the judge; empty is the one the policies ask, when they ask one.
Model string `json:"model,omitempty"`
Text string `json:"text"`
// Answer, when given, is read as the assistant's reply to Text.
Answer string `json:"answer,omitempty"`
}
// JudgeAskView is what the judge answered.
type JudgeAskView struct {
Model string `json:"model"`
// Verdict is safe or unsafe, as the gateway reads the answer.
Verdict string `json:"verdict"`
Categories []string `json:"categories,omitempty"`
// Raw is the judge's answer as it gave it.
Raw string `json:"raw"`
TookMs int64 `json:"took_ms"`
}
func (a *API) askJudge(w http.ResponseWriter, r *http.Request) {
if a.guard == nil || a.guard.Ask == nil {
writeError(w, http.StatusNotImplemented, "no guard policies: define them under guard in the cluster's settings (llmgw ctl config edit)")
return
}
var req JudgeAskRequest
if err := json.NewDecoder(r.Body).Decode(&req); err != nil || strings.TrimSpace(req.Text) == "" {
writeError(w, http.StatusBadRequest, "give the text for the judge to read")
return
}
if req.Model == "" {
judges := a.Judges()
if len(judges) == 0 {
writeError(w, http.StatusBadRequest, "no policy asks a judge: give the model")
return
}
req.Model = judges[0].Model
}
ans, err := a.guard.Ask(r.Context(), req.Model, req.Text, req.Answer)
v := JudgeAskView{Model: req.Model, Raw: ans.Raw, Categories: ans.Categories, TookMs: ans.Took.Milliseconds()}
if err != nil {
writeError(w, http.StatusBadGateway, fmt.Sprintf("the judge %s gave no verdict: %v (it answered %q)", req.Model, err, ans.Raw))
return
}
v.Verdict = "safe"
if ans.Unsafe {
v.Verdict = "unsafe"
}
writeJSON(w, http.StatusOK, v)
}
+25
View File
@@ -1,6 +1,7 @@
package adminapi package adminapi
import ( import (
"context"
"strings" "strings"
"testing" "testing"
"time" "time"
@@ -64,3 +65,27 @@ func TestEachJudgeSaysWhatIsWrongWithIt(t *testing.T) {
t.Errorf("an old failure still counts: %q", p) t.Errorf("an old failure still counts: %q", p)
} }
} }
func TestAJudgeIsAskedAndItsWordsShown(t *testing.T) {
a, _ := setup(t, testModels())
lib := guard.Config{Policies: map[string]guard.Policy{
"baseline": {Judge: &guard.JudgePolicy{Model: "llama-guard-3-1b", Input: "block"}}}}
var asked string
a.SetGuard(&GuardAdmin{Library: func() guard.Config { return lib },
Ask: func(_ context.Context, model, text, _ string) (guard.JudgeAnswer, error) {
asked = model + "|" + text
return guard.JudgeAnswer{Model: model, Raw: "unsafe\nS6", Unsafe: true,
Categories: []string{"S6", "Specialized Advice"}}, nil
}})
var v JudgeAskView
if code := call(t, a, "POST", "/api/v1/guard/judges/ask", `{"text":"how do I sort a list"}`, &v); code != 200 {
t.Fatalf("%d", code)
}
if asked != "llama-guard-3-1b|how do I sort a list" || v.Verdict != "unsafe" || v.Raw != "unsafe\nS6" ||
len(v.Categories) != 2 {
t.Errorf("asked %q, got %+v", asked, v)
}
if code := call(t, a, "POST", "/api/v1/guard/judges/ask", `{"text":" "}`, nil); code != 400 {
t.Errorf("no text: %d", code)
}
}
+2
View File
@@ -256,6 +256,8 @@ var Routes = []Route{
Response: GuardEventsView{}}, Response: GuardEventsView{}},
{Method: "GET", Path: "/api/v1/guard/judges", Summary: "The judge models the policies ask, and what is wrong with each", {Method: "GET", Path: "/api/v1/guard/judges", Summary: "The judge models the policies ask, and what is wrong with each",
Response: []JudgeView{}}, Response: []JudgeView{}},
{Method: "POST", Path: "/api/v1/guard/judges/ask", Summary: "Have a judge read a text and say what it answered; nothing is kept",
Request: JudgeAskRequest{}, Response: JudgeAskView{}},
{Method: "POST", Path: "/api/v1/guard/attachments", Summary: "Attach a guardrail policy", {Method: "POST", Path: "/api/v1/guard/attachments", Summary: "Attach a guardrail policy",
Request: GuardAttachmentRequest{}, Response: guard.Attachment{}}, Request: GuardAttachmentRequest{}, Response: guard.Attachment{}},
{Method: "DELETE", Path: "/api/v1/guard/attachments", Summary: "Detach a guardrail policy", {Method: "DELETE", Path: "/api/v1/guard/attachments", Summary: "Detach a guardrail policy",
+6
View File
@@ -44,6 +44,9 @@ type Summary struct {
ControlPlanes ControlPlanes `json:"control_planes,omitzero"` ControlPlanes ControlPlanes `json:"control_planes,omitzero"`
// Judges are the guardrails' judge models with something wrong. // Judges are the guardrails' judge models with something wrong.
Judges []JudgeView `json:"judges,omitempty"` Judges []JudgeView `json:"judges,omitempty"`
// GuardOff says the guardrails are turned off (guard.enabled: false):
// no request is checked, whatever the policies say.
GuardOff bool `json:"guard_off,omitempty"`
} }
// StoreMember is one member of a replicated store, as its Patroni reports it. // StoreMember is one member of a replicated store, as its Patroni reports it.
@@ -151,6 +154,9 @@ const idleThreshold = 10 * time.Minute
// Summary builds the fleet summary. // Summary builds the fleet summary.
func (a *API) Summary() Summary { func (a *API) Summary() Summary {
s := Summary{Version: version.Version} s := Summary{Version: version.Version}
if a.guard != nil && a.guard.Library != nil && !a.guard.Library().On() {
s.GuardOff = true
}
for _, j := range a.Judges() { for _, j := range a.Judges() {
if j.Problem != "" { if j.Problem != "" {
s.Judges = append(s.Judges, j) s.Judges = append(s.Judges, j)
+1
View File
@@ -484,6 +484,7 @@ var Reference = []Block{
{Key: "sim", Type: "SimConfig", Doc: "", Nested: "engines.SimConfig"}, {Key: "sim", Type: "SimConfig", Doc: "", Nested: "engines.SimConfig"},
}}, }},
{Package: "guard", Name: "Config", Title: "guard.Config", Doc: "", Fields: []Field{ {Package: "guard", Name: "Config", Title: "guard.Config", Doc: "", Fields: []Field{
{Key: "enabled", Type: "*bool", Doc: "Enabled turns every policy on or off at once: false checks nothing,\nwhatever is defined or attached. Unset is on.", Nested: ""},
{Key: "default", Type: "[]string", Doc: "", Nested: ""}, {Key: "default", Type: "[]string", Doc: "", Nested: ""},
{Key: "policies", Type: "map[string]Policy", Doc: "", Nested: ""}, {Key: "policies", Type: "map[string]Policy", Doc: "", Nested: ""},
}}, }},
+16 -1
View File
@@ -156,6 +156,10 @@ func (g *Gate) Invalidate() { g.reload(context.Background()) }
// from. // from.
func (g *Gate) Policy(s Subject) (Policy, string) { func (g *Gate) Policy(s Subject) (Policy, string) {
g.mu.Lock() g.mu.Lock()
if !g.cfg.On() {
g.mu.Unlock()
return Policy{}, ""
}
if !g.loading && g.store != nil && g.now().Sub(g.loadedAt) > attachmentsTTL { if !g.loading && g.store != nil && g.now().Sub(g.loadedAt) > attachmentsTTL {
g.loading = true g.loading = true
go g.reload(context.WithoutCancel(context.Background())) go g.reload(context.WithoutCancel(context.Background()))
@@ -328,7 +332,7 @@ func (g *Gate) Input(ctx context.Context, s Subject, body []byte) (Verdict, erro
done := func(f finding) Verdict { done := func(f finding) Verdict {
v.Action, v.Category, v.Rule = f.action, f.category, f.rule v.Action, v.Category, v.Rule = f.action, f.category, f.rule
if f.action == Block { if f.action == Block {
v.Message = fmt.Sprintf("this request was refused by the gateway's policy (%s)", f.category) v.Message = fmt.Sprintf("this request was refused by the gateway's policy (%s)", refusedFor(f))
v.Body = nil v.Body = nil
} }
v.Took = g.now().Sub(start) v.Took = g.now().Sub(start)
@@ -524,3 +528,14 @@ func toolAllowed(t Tools, name string) bool {
} }
return false return false
} }
// refusedFor is what a refusal names: the category, and for a judge the
// hazard it saw — "judge: S6,Specialized Advice" — so whoever is refused
// can say why, and a judge refusing ordinary questions can be told apart.
// Never the text.
func refusedFor(f finding) string {
if f.category == "judge" && f.rule != "" && f.rule != "unavailable" {
return "judge: " + f.rule
}
return f.category
}
+31
View File
@@ -278,3 +278,34 @@ func TestCapabilitiesAndStructuredAnswersWhenAskedFor(t *testing.T) {
} }
} }
} }
func TestTheGuardrailsTurnOffAndOnAsAWhole(t *testing.T) {
off := false
cfg := lib(map[string]Policy{"base": {Input: Rules{PII: map[string]string{"secret": "block"}}}}, "base")
cfg.Enabled = &off
g := gate(t, cfg)
secret := chat("user", "my key is sk-proj-abcdefghijklmnopqrstuvwx1234")
if v, err := g.Input(context.Background(), Subject{}, secret); err != nil || v.Action != "" {
t.Errorf("off, a secret was acted on: %+v %v", v, err)
}
on := true
cfg.Enabled = &on
if err := g.Replace(cfg); err != nil {
t.Fatal(err)
}
if v, _ := g.Input(context.Background(), Subject{}, secret); v.Action != Block {
t.Errorf("on again, a secret went through: %+v", v)
}
if !(Config{}).On() {
t.Error("unset is not on")
}
}
func TestAJudgesRefusalNamesTheHazard(t *testing.T) {
if got := refusedFor(finding{category: "judge", rule: "S6,Specialized Advice"}); got != "judge: S6,Specialized Advice" {
t.Errorf("%q", got)
}
if got := refusedFor(finding{category: "secret", rule: "pii:secret"}); got != "secret" {
t.Errorf("%q", got)
}
}
+54
View File
@@ -294,6 +294,10 @@ func parseJudgment(format string, names []string, body []byte, answer string) (j
} }
switch format { switch format {
case "llamaguard3": case "llamaguard3":
if len(names) == 0 {
// Its own: S1 is Violent Crimes, S9 Indiscriminate Weapons.
names = llamaGuard3Categories
}
lines := strings.Split(text, "\n") lines := strings.Split(text, "\n")
switch strings.ToLower(strings.TrimSpace(lines[0])) { switch strings.ToLower(strings.TrimSpace(lines[0])) {
case "safe": case "safe":
@@ -369,3 +373,53 @@ func DefaultCategories(format string) []string {
} }
return nil return nil
} }
// JudgeAnswer is what a judge said of a text: its words as it gave them,
// and as the gateway reads them.
type JudgeAnswer struct {
Model string
Raw string
Unsafe bool
Categories []string
Took time.Duration
}
// AskJudge has a judge read a text, past the cache, and says what it
// answered: for an operator finding out why ordinary requests are refused.
// With answer, the judge reads it as the assistant's reply to text.
func (g *Gate) AskJudge(ctx context.Context, model, text, answer string) (JudgeAnswer, error) {
out := JudgeAnswer{Model: model}
if g.judger == nil {
return out, errNoJudge
}
format, names, ok := g.judger.format(model)
if !ok {
return out, fmt.Errorf("%w: %q is not a model with role: guard", errNoJudge, model)
}
ctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
start := time.Now()
path, body := judgeRequest(model, format, names, text, answer)
reply, err := g.judger.call(ctx, model, path, body)
out.Took = time.Since(start)
if err != nil {
return out, err
}
var resp struct {
Choices []struct {
Message struct {
Content string `json:"content"`
} `json:"message"`
Text string `json:"text"`
} `json:"choices"`
}
if json.Unmarshal(reply, &resp) == nil && len(resp.Choices) > 0 {
out.Raw = strings.TrimSpace(resp.Choices[0].Message.Content + resp.Choices[0].Text)
}
res, err := parseJudgment(format, names, reply, answer)
if err != nil {
return out, err
}
out.Unsafe, out.Categories = res.unsafe, res.categories
return out, nil
}
+6
View File
@@ -25,10 +25,16 @@ import (
// Policies are attached to parts of the tenancy tree in the store; Default // Policies are attached to parts of the tenancy tree in the store; Default
// applies to every request. // applies to every request.
type Config struct { type Config struct {
// Enabled turns every policy on or off at once: false checks nothing,
// whatever is defined or attached. Unset is on.
Enabled *bool `koanf:"enabled"`
Default []string `koanf:"default"` Default []string `koanf:"default"`
Policies map[string]Policy `koanf:"policies"` Policies map[string]Policy `koanf:"policies"`
} }
// On is whether the guardrails check anything.
func (c Config) On() bool { return c.Enabled == nil || *c.Enabled }
// Policy is one named set of rules. // Policy is one named set of rules.
type Policy struct { type Policy struct {
// Mode: enforce acts, shadow only records what it would have done, off // Mode: enforce acts, shadow only records what it would have done, off
+1 -1
View File
@@ -191,7 +191,7 @@ func guardFields(u *store.Usage, in, out guard.Verdict) {
if out.Stricter(in) { if out.Stricter(in) {
v = out v = out
} }
u.GuardAction, u.GuardPolicy, u.GuardCategory = string(v.Action), v.Policy, v.Category u.GuardAction, u.GuardPolicy, u.GuardCategory, u.GuardRule = string(v.Action), v.Policy, v.Category, v.Rule
if v.Shadow && v.Action != guard.Allow { if v.Shadow && v.Action != guard.Allow {
u.GuardAction = "shadow-" + u.GuardAction u.GuardAction = "shadow-" + u.GuardAction
} }
+2 -1
View File
@@ -34,7 +34,7 @@ func (g *wordGuard) Input(_ context.Context, s guard.Subject, body []byte) (guar
case g.err != nil: case g.err != nil:
return guard.Verdict{}, g.err return guard.Verdict{}, g.err
case bytes.Contains(body, []byte("forbidden")): case bytes.Contains(body, []byte("forbidden")):
return guard.Verdict{Action: guard.Block, Policy: "words", Category: "test"}, nil return guard.Verdict{Action: guard.Block, Policy: "words", Category: "test", Rule: "S6,Specialized Advice"}, nil
case bytes.Contains(body, []byte("alice@example.com")): case bytes.Contains(body, []byte("alice@example.com")):
return guard.Verdict{Action: guard.Redact, Policy: "pii", Category: "email", return guard.Verdict{Action: guard.Redact, Policy: "pii", Category: "email",
Body: bytes.ReplaceAll(body, []byte("alice@example.com"), []byte("[EMAIL_1]"))}, nil Body: bytes.ReplaceAll(body, []byte("alice@example.com"), []byte("[EMAIL_1]"))}, nil
@@ -157,6 +157,7 @@ func TestABlockedRequestIsRecordedWithItsVerdictAndCostsNothing(t *testing.T) {
} }
if ok { if ok {
if u.GuardAction != "blocked" || u.GuardPolicy != "words" || u.GuardCategory != "test" || if u.GuardAction != "blocked" || u.GuardPolicy != "words" || u.GuardCategory != "test" ||
u.GuardRule != "S6,Specialized Advice" ||
u.CostMicros != 0 || u.PromptTokens != 0 || u.Status != http.StatusBadRequest { u.CostMicros != 0 || u.PromptTokens != 0 || u.Status != http.StatusBadRequest {
t.Errorf("recorded %+v", u) t.Errorf("recorded %+v", u)
} }
+4 -2
View File
@@ -1217,9 +1217,11 @@ func (r *Router) accessLog(req *parsedRequest, res dispatchResult, status int, t
} }
// What a guard did, by name: never what it matched. // What a guard did, by name: never what it matched.
if v := req.guardIn; res.guard.Stricter(v) { if v := req.guardIn; res.guard.Stricter(v) {
attrs = append(attrs, "guard", string(res.guard.Action), "guard_stage", "output", "guard_policy", res.guard.Policy) attrs = append(attrs, "guard", string(res.guard.Action), "guard_stage", "output", "guard_policy", res.guard.Policy,
"guard_category", res.guard.Category, "guard_rule", res.guard.Rule)
} else if v.Action != "" { } else if v.Action != "" {
attrs = append(attrs, "guard", string(v.Action), "guard_stage", "input", "guard_policy", v.Policy) attrs = append(attrs, "guard", string(v.Action), "guard_stage", "input", "guard_policy", v.Policy,
"guard_category", v.Category, "guard_rule", v.Rule)
} }
if res.outcome.Stopped { if res.outcome.Stopped {
attrs = append(attrs, "stopped_by_guard", true) attrs = append(attrs, "stopped_by_guard", true)
+3 -2
View File
@@ -50,6 +50,7 @@ type GuardEvent struct {
Action string Action string
Policy string Policy string
Category string Category string
Rule string
GuardMs int64 GuardMs int64
} }
@@ -57,14 +58,14 @@ type GuardEvent struct {
// flagged, redacted, blocked, or would have in shadow — since a moment. // flagged, redacted, blocked, or would have in shadow — since a moment.
func (s *DB) GuardEvents(ctx context.Context, since time.Time, limit int) ([]GuardEvent, error) { func (s *DB) GuardEvents(ctx context.Context, since time.Time, limit int) ([]GuardEvent, error) {
return queryAll(ctx, s.db, `SELECT ts, request_id, key_id, COALESCE(user_id, ''), project_id, model, status, return queryAll(ctx, s.db, `SELECT ts, request_id, key_id, COALESCE(user_id, ''), project_id, model, status,
guard_action, guard_policy, guard_category, guard_ms guard_action, guard_policy, guard_category, guard_rule, guard_ms
FROM usage WHERE ts >= ? AND guard_action <> '' ORDER BY ts DESC LIMIT ?`, FROM usage WHERE ts >= ? AND guard_action <> '' ORDER BY ts DESC LIMIT ?`,
[]any{millis(since), limit}, []any{millis(since), limit},
func(rows *sql.Rows) (GuardEvent, error) { func(rows *sql.Rows) (GuardEvent, error) {
var e GuardEvent var e GuardEvent
var ts int64 var ts int64
err := rows.Scan(&ts, &e.RequestID, &e.KeyID, &e.UserID, &e.ProjectID, &e.Model, &e.Status, err := rows.Scan(&ts, &e.RequestID, &e.KeyID, &e.UserID, &e.ProjectID, &e.Model, &e.Status,
&e.Action, &e.Policy, &e.Category, &e.GuardMs) &e.Action, &e.Policy, &e.Category, &e.Rule, &e.GuardMs)
e.At = fromMillis(ts) e.At = fromMillis(ts)
return e, err return e, err
}) })
@@ -0,0 +1,2 @@
-- Ported from the SQLite migration of the same number.
ALTER TABLE usage ADD COLUMN guard_rule TEXT NOT NULL DEFAULT '';
@@ -0,0 +1,5 @@
-- What in a guard policy decided a request, beside its category: the
-- pattern's name, or the judge's verdict — "S6,Specialized Advice" — so a
-- request a judge refused says which hazard it saw, and a judge that
-- refuses ordinary questions can be told from one that is right to.
ALTER TABLE usage ADD COLUMN guard_rule TEXT NOT NULL DEFAULT '';
+5 -5
View File
@@ -435,8 +435,8 @@ func (s *DB) WriteUsage(ctx context.Context, records []Usage) error {
`INSERT INTO usage (ts, key_id, project_id, model, served_model, `INSERT INTO usage (ts, key_id, project_id, model, served_model,
prompt_tokens, completion_tokens, cost_micros, status, latency_ms, user_id, prompt_tokens, completion_tokens, cost_micros, status, latency_ms, user_id,
request_id, replica, queue_ms, ttft_ms, request_id, replica, queue_ms, ttft_ms,
guard_action, guard_policy, guard_category, guard_ms, end_user, tags, cache_hit) guard_action, guard_policy, guard_category, guard_rule, guard_ms, end_user, tags, cache_hit)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
if err != nil { if err != nil {
return err return err
} }
@@ -450,7 +450,7 @@ func (s *DB) WriteUsage(ctx context.Context, records []Usage) error {
r.Model, r.ServedModel, r.PromptTokens, r.CompletionTokens, r.Model, r.ServedModel, r.PromptTokens, r.CompletionTokens,
r.CostMicros, r.Status, r.LatencyMs, r.UserID, r.CostMicros, r.Status, r.LatencyMs, r.UserID,
r.RequestID, r.Replica, r.QueueMs, r.TTFTMs, r.RequestID, r.Replica, r.QueueMs, r.TTFTMs,
r.GuardAction, r.GuardPolicy, r.GuardCategory, r.GuardMs, r.EndUser, tagsJSON(r.Tags), hit); err != nil { r.GuardAction, r.GuardPolicy, r.GuardCategory, r.GuardRule, r.GuardMs, r.EndUser, tagsJSON(r.Tags), hit); err != nil {
return err return err
} }
} }
@@ -864,11 +864,11 @@ func (s *DB) UsageByRequest(ctx context.Context, requestID string) (Usage, bool,
err := s.db.QueryRowContext(ctx, err := s.db.QueryRowContext(ctx,
`SELECT ts, key_id, project_id, COALESCE(user_id, ''), model, served_model, prompt_tokens, `SELECT ts, key_id, project_id, COALESCE(user_id, ''), model, served_model, prompt_tokens,
completion_tokens, cost_micros, status, latency_ms, request_id, replica, queue_ms, ttft_ms, completion_tokens, cost_micros, status, latency_ms, request_id, replica, queue_ms, ttft_ms,
guard_action, guard_policy, guard_category, guard_ms, end_user, tags, cache_hit guard_action, guard_policy, guard_category, guard_rule, guard_ms, end_user, tags, cache_hit
FROM usage WHERE request_id = ? ORDER BY ts DESC LIMIT 1`, requestID). FROM usage WHERE request_id = ? ORDER BY ts DESC LIMIT 1`, requestID).
Scan(&ts, &u.KeyID, &u.ProjectID, &u.UserID, &u.Model, &u.ServedModel, &u.PromptTokens, Scan(&ts, &u.KeyID, &u.ProjectID, &u.UserID, &u.Model, &u.ServedModel, &u.PromptTokens,
&u.CompletionTokens, &u.CostMicros, &u.Status, &u.LatencyMs, &u.RequestID, &u.Replica, &u.CompletionTokens, &u.CostMicros, &u.Status, &u.LatencyMs, &u.RequestID, &u.Replica,
&u.QueueMs, &u.TTFTMs, &u.GuardAction, &u.GuardPolicy, &u.GuardCategory, &u.GuardMs, &u.QueueMs, &u.TTFTMs, &u.GuardAction, &u.GuardPolicy, &u.GuardCategory, &u.GuardRule, &u.GuardMs,
&u.EndUser, &tags, &hit) &u.EndUser, &tags, &hit)
if errors.Is(err, sql.ErrNoRows) { if errors.Is(err, sql.ErrNoRows) {
return Usage{}, false, nil return Usage{}, false, nil
+3 -1
View File
@@ -257,11 +257,13 @@ type Usage struct {
// GuardAction is what a guard did — flagged, redacted, blocked — empty // GuardAction is what a guard did — flagged, redacted, blocked — empty
// when it let the request through untouched or there is none; // when it let the request through untouched or there is none;
// GuardPolicy and GuardCategory what decided it, GuardMs how long // GuardPolicy, GuardCategory and GuardRule what decided it — a judge's
// rule is its verdict, "S6,Specialized Advice" — GuardMs how long
// judging took. Never the text. // judging took. Never the text.
GuardAction string GuardAction string
GuardPolicy string GuardPolicy string
GuardCategory string GuardCategory string
GuardRule string
GuardMs int64 GuardMs int64
// EndUser is OpenAI's user field as the client sent it: its own name // EndUser is OpenAI's user field as the client sent it: its own name
+10 -7
View File
@@ -24,8 +24,10 @@ type guardPageData struct {
Window string Window string
Windows []string Windows []string
// Policies are the library, Default those applied to every request. // Policies are the library, Default those applied to every request.
Policies []string Policies []string
Default []string Default []string
// Off is the guardrails turned off (guard.enabled: false).
Off bool
Attachments []guardAttachmentRow Attachments []guardAttachmentRow
Counts []guardCountRow Counts []guardCountRow
Events []guardEventRow Events []guardEventRow
@@ -40,9 +42,9 @@ type guardCountRow struct {
} }
type guardEventRow struct { type guardEventRow struct {
At, RequestID, Where, Model, Action, Policy, Category string At, RequestID, Where, Model, Action, Policy, Category, Rule string
Status int Status int
Ms int64 Ms int64
} }
var guardWindows = map[string]time.Duration{"24h": 24 * time.Hour, "7d": 7 * 24 * time.Hour, "30d": 30 * 24 * time.Hour} var guardWindows = map[string]time.Duration{"24h": 24 * time.Hour, "7d": 7 * 24 * time.Hour, "30d": 30 * 24 * time.Hour}
@@ -54,8 +56,9 @@ func (p *AccountPages) RegisterAdminGuard(mux *http.ServeMux, s GuardAdminSource
mux.HandleFunc("GET /admin/guard", p.adminOnly(func(w http.ResponseWriter, r *http.Request, admin store.User) { mux.HandleFunc("GET /admin/guard", p.adminOnly(func(w http.ResponseWriter, r *http.Request, admin store.User) {
ctx, l := r.Context(), langOf(r) ctx, l := r.Context(), langOf(r)
lib := library() lib := library()
off := !lib.On()
d := guardPageData{Nav: p.navFor(r, admin, "guard"), Version: p.version, Window: r.URL.Query().Get("window"), d := guardPageData{Nav: p.navFor(r, admin, "guard"), Version: p.version, Window: r.URL.Query().Get("window"),
Windows: []string{"24h", "7d", "30d"}, Default: lib.Default} Windows: []string{"24h", "7d", "30d"}, Default: lib.Default, Off: off}
since, ok := guardWindows[d.Window] since, ok := guardWindows[d.Window]
if !ok { if !ok {
d.Window, since = "7d", guardWindows["7d"] d.Window, since = "7d", guardWindows["7d"]
@@ -104,7 +107,7 @@ func (p *AccountPages) RegisterAdminGuard(mux *http.ServeMux, s GuardAdminSource
at := e.At.UTC() at := e.At.UTC()
d.Events = append(d.Events, guardEventRow{At: l.Date(at) + " " + at.Format("15:04:05"), d.Events = append(d.Events, guardEventRow{At: l.Date(at) + " " + at.Format("15:04:05"),
RequestID: e.RequestID, Where: name(e.ProjectID), Model: e.Model, Action: guardActionWord(l, e.Action), RequestID: e.RequestID, Where: name(e.ProjectID), Model: e.Model, Action: guardActionWord(l, e.Action),
Policy: e.Policy, Category: e.Category, Status: e.Status, Ms: e.GuardMs}) Policy: e.Policy, Category: e.Category, Rule: e.Rule, Status: e.Status, Ms: e.GuardMs})
} }
noStore(w) noStore(w)
render(w, r, "admin_guard", d) render(w, r, "admin_guard", d)
+2 -1
View File
@@ -32,5 +32,6 @@
"guard.col.request": "Request", "guard.col.request": "Request",
"guard.col.model": "Model", "guard.col.model": "Model",
"guard.col.status": "Status", "guard.col.status": "Status",
"guard.request_hint": "<code>llmgw ctl request ID</code> says what else a request went through." "guard.request_hint": "<code>llmgw ctl request ID</code> says what else a request went through.",
"guard.off": "The guardrails are <strong>off</strong>: no request is checked, whatever the policies below say. <code>llmgw ctl guard on</code> turns them back on."
} }
+2 -1
View File
@@ -32,5 +32,6 @@
"guard.col.request": "İstek", "guard.col.request": "İstek",
"guard.col.model": "Model", "guard.col.model": "Model",
"guard.col.status": "Durum", "guard.col.status": "Durum",
"guard.request_hint": "Bir isteğin başka nelerden geçtiğini <code>llmgw ctl request ID</code> söyler." "guard.request_hint": "Bir isteğin başka nelerden geçtiğini <code>llmgw ctl request ID</code> söyler.",
"guard.off": "Guardrail'ler <strong>kapalı</strong>: aşağıdaki politikalar ne derse desin hiçbir istek kontrol edilmiyor. <code>llmgw ctl guard on</code> yeniden açar."
} }
+2 -1
View File
@@ -9,6 +9,7 @@
{{template "nav" .Nav}} {{template "nav" .Nav}}
</header> </header>
{{if .Error}}<p class="error" role="alert">{{.Error}}</p>{{end}} {{if .Error}}<p class="error" role="alert">{{.Error}}</p>{{end}}
{{if .Off}}<p class="error" role="alert">{{th "guard.off"}}</p>{{end}}
<h2>{{t "guard.policies"}}</h2> <h2>{{t "guard.policies"}}</h2>
{{if .Policies}} {{if .Policies}}
@@ -39,7 +40,7 @@
<table class="rows"> <table class="rows">
<tr><th>{{t "guard.col.when"}}</th><th>{{t "guard.col.request"}}</th><th>{{t "guard.col.where"}}</th><th>{{t "guard.col.model"}}</th><th>{{t "guard.col.action"}}</th><th>{{t "guard.col.policy"}}</th><th>{{t "guard.col.category"}}</th><th class="num">{{t "guard.col.status"}}</th><th class="num">ms</th></tr> <tr><th>{{t "guard.col.when"}}</th><th>{{t "guard.col.request"}}</th><th>{{t "guard.col.where"}}</th><th>{{t "guard.col.model"}}</th><th>{{t "guard.col.action"}}</th><th>{{t "guard.col.policy"}}</th><th>{{t "guard.col.category"}}</th><th class="num">{{t "guard.col.status"}}</th><th class="num">ms</th></tr>
{{range .Events}}<tr><td>{{.At}}</td><td><code>{{.RequestID}}</code></td><td>{{.Where}}</td><td>{{.Model}}</td> {{range .Events}}<tr><td>{{.At}}</td><td><code>{{.RequestID}}</code></td><td>{{.Where}}</td><td>{{.Model}}</td>
<td>{{.Action}}</td><td><code>{{.Policy}}</code></td><td>{{.Category}}</td><td class="num">{{.Status}}</td><td class="num">{{count .Ms}}</td></tr>{{end}} <td>{{.Action}}</td><td><code>{{.Policy}}</code></td><td>{{.Category}}{{if .Rule}} · {{.Rule}}{{end}}</td><td class="num">{{.Status}}</td><td class="num">{{count .Ms}}</td></tr>{{end}}
</table> </table>
<p class="sub">{{th "guard.request_hint"}}</p> <p class="sub">{{th "guard.request_hint"}}</p>
{{end}} {{end}}