Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 23:21:44 +02:00
1 parent bce54f0941
commit 35657c1417
20 files changed
+837 -14

No files matched your search

+4
View File
@@ -44,6 +44,10 @@ never stored — so a leaked database hands over no working credential, and
neither llmgw nor an administrator can recover a key somebody lost. A prefix is
kept so a key can be identified in a listing without revealing it.
**The store's own secrets**, like the one engine keys are derived from, plain
unless `secrets.master_key` seals them — with Vault's transit engine, so that
neither the store nor its backups hold them.
**People and tenancy**: accounts, the organisation/team/project tree, and the
limits attached to it. Passwords are hashed.
+37
View File
@@ -1394,6 +1394,43 @@ and point it at `https://<gateway>/scim/v2` with that token as the bearer.
join and leave them. Membership in a group is removed when someone leaves
it.
## Secrets in Vault
Every setting that names a file holding a secret — the store's `@address`,
`api_key_file`, `auth.oidc.client_secret_file`, `auth.scim.token_file`, a
webhook's `secret_file` — also takes `env:VARIABLE` or a field of a secret in
HashiCorp Vault or OpenBao:
```yaml
auth:
store: "@vault:secret/llmgw/store#dsn"
secrets:
vault:
addr: https://vault.internal:8200
ca_file: /etc/llmgw/vault-ca.pem
role_id_file: /etc/llmgw/secrets/vault-role-id # AppRole; or token_file,
secret_id_file: /etc/llmgw/secrets/vault-secret-id # or VAULT_TOKEN
master_key: vault-transit:llmgw
```
These are read when the gateway starts (an external model's key, each time
the catalog changes): a secret rotated in Vault is taken up at the next
restart.
**The store's own secrets** — what every engine's key is derived from — are
kept in the store, and so in its backups. `secrets.master_key` seals them:
- `vault-transit:KEY` seals with Vault's transit engine. The key never leaves
Vault, so neither the store nor any backup of it opens without Vault.
- Anything else is a reference to 32 random bytes (`openssl rand -hex 32`),
used here: better than nothing, but a key in `/etc/llmgw` is in every backup
beside what it seals. Keep it in Vault or the environment.
Values kept plain from before are sealed the first time they are read after
the key is set. The key cannot be changed once set without losing them: a new
one cannot open what the old one sealed (the engines are then restarted with
new keys: `llmgw ctl restart --all`).
## The firewall
`server init`, `node add`, `server add` and `mirror init` each keep that
+75
View File
@@ -230,6 +230,8 @@ rather than repeated.
| `target_inflight` | `float64` | [Scoring](#scoring) |
| `kv_ceiling` | `float64` | [Scoring](#scoring) |
| `session_load_factor` | `float64` | [Scoring](#scoring) |
| `vault` | `secrets.Vault` | [Secrets](#secrets) |
| `master_key` | `string` | [Secrets](#secrets) |
| `listen` | `string` | [Server](#server) |
| `admin_listen` | `string` | [Server](#server) |
| `metrics_listen` | `string` | [Server](#server) |
@@ -250,6 +252,7 @@ rather than repeated.
| `engine_overrides` | `[]EngineOverride` | [Server](#server) |
| `backup_dir` | `string` | [Server](#server) |
| `guard` | `guard.Config` | [Server](#server) |
| `secrets` | `Secrets` | [Server](#server) |
| `notify` | `notify.Config` | [Server](#server) |
| `firewall_networks` | `[]string` | [Server](#server) |
| `backup_recipients` | `[]string` | [Server](#server) |
@@ -279,6 +282,12 @@ rather than repeated.
| `bin` | `string` | [VLLMConfig](#vllmconfig) |
| `launch_mode` | `string` | [VLLMConfig](#vllmconfig) |
| `ray_bin` | `string` | [VLLMConfig](#vllmconfig) |
| `addr` | `string` | [Vault](#vault) |
| `token_file` | `string` | [Vault](#vault) |
| `role_id_file` | `string` | [Vault](#vault) |
| `secret_id_file` | `string` | [Vault](#vault) |
| `ca_file` | `string` | [Vault](#vault) |
| `namespace` | `string` | [Vault](#vault) |
| `url` | `string` | [Webhook](#webhook) |
| `format` | `string` | [Webhook](#webhook) |
| `events` | `[]string` | [Webhook](#webhook) |
@@ -1859,6 +1868,27 @@ arrive at nine are the reason for the window.
`float64`
## Secrets
### `vault`
`secrets.Vault`
Vault is how vault: references and vault-transit: are reached.
Takes a [Vault](#vault) block.
### `master_key`
`string`
MasterKey seals the secrets the store keeps for itself — what every
engine's key is derived from — so a copy of the store, or of a backup,
does not hold them: vault-transit:KEY seals with Vault's transit engine,
which never lets the key out; otherwise a reference to 32 random bytes
(openssl rand -hex 32), which then must be kept as safely as the store.
Values from before it was set are sealed the first time they are read.
## Server
### `listen`
@@ -2048,6 +2078,15 @@ guard.default, and to parts of the tenancy tree by attaching them
Takes a [guard.Config](#guardconfig) block.
### `secrets`
`Secrets`
Secrets is where secrets kept outside files are read from, and the key
the store's own secrets are sealed under.
Takes a [Secrets](#secrets) block.
### `notify`
`notify.Config`
@@ -2263,6 +2302,42 @@ node and has not been checked against a real cluster.
RayBin is used when LaunchMode is "ray".
## Vault
### `addr`
`string`
Addr is Vault's address, like https://vault.internal:8200; empty
takes VAULT_ADDR.
### `token_file`
`string`
TokenFile holds a token; empty takes VAULT_TOKEN, or logs in with
AppRole when RoleIDFile and SecretIDFile are set.
### `role_id_file`
`string`
### `secret_id_file`
`string`
### `ca_file`
`string`
CAFile verifies Vault's certificate; empty uses the system's.
### `namespace`
`string`
Namespace is Vault Enterprise's or OpenBao's namespace.
## Webhook
### `url`