Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates

notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 16:58:11 +02:00
1 parent 646ff996c7
commit 1b71c38f89
24 files changed
+907 -4

No files matched your search

+56
View File
@@ -230,6 +230,7 @@ rather than repeated.
| `engine_overrides` | `[]EngineOverride` | [Server](#server) |
| `backup_dir` | `string` | [Server](#server) |
| `guard` | `guard.Config` | [Server](#server) |
| `notify` | `notify.Config` | [Server](#server) |
| `firewall_networks` | `[]string` | [Server](#server) |
| `backup_recipients` | `[]string` | [Server](#server) |
| `releases` | `string` | [Server](#server) |
@@ -256,6 +257,10 @@ rather than repeated.
| `bin` | `string` | [VLLMConfig](#vllmconfig) |
| `launch_mode` | `string` | [VLLMConfig](#vllmconfig) |
| `ray_bin` | `string` | [VLLMConfig](#vllmconfig) |
| `url` | `string` | [Webhook](#webhook) |
| `format` | `string` | [Webhook](#webhook) |
| `events` | `[]string` | [Webhook](#webhook) |
| `secret_file` | `string` | [Webhook](#webhook) |
| `prompt_per_1k` | `int64` | [accounting.Price](#accountingprice) |
| `completion_per_1k` | `int64` | [accounting.Price](#accountingprice) |
| `name` | `string` | [admission.Band](#admissionband) |
@@ -298,6 +303,8 @@ rather than repeated.
| `sim` | `SimConfig` | [engines.Config](#enginesconfig) |
| `default` | `[]string` | [guard.Config](#guardconfig) |
| `policies` | `map[string]Policy` | [guard.Config](#guardconfig) |
| `webhooks` | `[]Webhook` | [notify.Config](#notifyconfig) |
| `host` | `string` | [notify.Config](#notifyconfig) |
| `work_dir` | `string` | [supervisor.Config](#supervisorconfig) |
| `log_lines` | `int` | [supervisor.Config](#supervisorconfig) |
| `stop_timeout` | `time.Duration` | [supervisor.Config](#supervisorconfig) |
@@ -1868,6 +1875,16 @@ guard.default, and to parts of the tenancy tree by attaching them
Takes a [guard.Config](#guardconfig) block.
### `notify`
`notify.Config`
Notify sends what needs a person — a model parked, a node lost, a
budget spent, new weights published — to webhooks, signed, without a
Prometheus and Alertmanager in between. Empty sends nothing.
Takes a [notify.Config](#notifyconfig) block.
### `firewall_networks`
`[]string`
@@ -2056,6 +2073,33 @@ node and has not been checked against a real cluster.
RayBin is used when LaunchMode is "ray".
## Webhook
### `url`
`string`
### `format`
`string`
Format is json (the event, with X-Llmgw-Signature when SecretFile is
set) or slack ({"text": …}, which Slack, Mattermost and Rocket.Chat
incoming webhooks take).
### `events`
`[]string`
Events are the kinds sent here; empty is all.
### `secret_file`
`string`
SecretFile holds the key the body is signed with: X-Llmgw-Signature
is sha256=<hex HMAC of the body>.
## accounting.Price
### `prompt_per_1k`
@@ -2340,6 +2384,18 @@ Takes a [SimConfig](#simconfig) block.
`map[string]Policy`
## notify.Config
### `webhooks`
`[]Webhook`
### `host`
`string`
Host names the gateway in messages; empty is this machine's name.
## supervisor.Config
### `work_dir`