Publish per-tenant request and budget series, and alert on them
Requests by outcome, tokens and first-token time per tenant, at the level router.tenant_label names (team by default), and every budget's share spent this period. Rules alert on budgets at 80% and 100%, tenants hitting limits, failing, spiking past their weekly average, or waiting on first tokens. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
9f479b9a70
commit
ce006a6e6f
10 files changed
+376
-2
No files matched your search
@@ -162,6 +162,7 @@ rather than repeated.
|
||||
| `prefix` | `Prefix` | [Router](#router) |
|
||||
| `session_header` | `string` | [Router](#router) |
|
||||
| `inject_usage` | `bool` | [Router](#router) |
|
||||
| `tenant_label` | `string` | [Router](#router) |
|
||||
| `max_body_bytes` | `int64` | [Router](#router) |
|
||||
| `timeouts` | `Timeouts` | [Router](#router) |
|
||||
| `scoring` | `Scoring` | [Router](#router) |
|
||||
@@ -1254,6 +1255,16 @@ SessionHeader carries the session id for affinity (plan §7.5 tier 0).
|
||||
InjectUsage asks engines for usage even when the client did not, so token
|
||||
accounting works regardless of client behaviour.
|
||||
|
||||
### `tenant_label`
|
||||
|
||||
`string`
|
||||
|
||||
TenantLabel is the level of the tree the per-tenant request series are
|
||||
labelled by — llmgw_tenant_requests_total and its siblings: "org",
|
||||
"team" or "project", or "none" for no such series. A label per project
|
||||
on a cluster of thousands is more series than Prometheus wants, and a
|
||||
team is what an alert is usually about; "team" is the default.
|
||||
|
||||
### `max_body_bytes`
|
||||
|
||||
`int64`
|
||||
|
||||
Reference in new issue
Block a user