The cluster's settings live in the store, and every change takes effect

Everything in server.yaml that is not one machine's own is now a versioned
document in the store (cluster_settings), the same on every control plane.
The first start against a store imports the file's cluster keys, with the
baseline guard policy when there is none; after that those keys in a file
are ignored, with a warning.

Each control plane polls the document's version and its own server.yaml's
hash. A change is validated and applied: guard live (Gate.Replace), anything
else by a graceful restart that systemd brings back, standbys first and the
leader 30s later. A version that does not load on a machine is not applied;
at start the newest one that loads is used, so a bad edit cannot keep every
control plane down.

llmgw ctl config show/get/set/unset/edit/history/rollback, the admin API
under /api/v1/settings, and the console's Settings screen, which layers the
store's document over the machine's file, write it. Commands beside the
control plane that read cluster keys (init, node add, server add, backup,
upgrade) read the layered configuration; init --bundle writes offline to
the store once it holds the settings.

A standby that becomes the leader re-reads models, engine choices and
facility holds from the store.

Retires appendGuardIfMissing: the import does it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 15:25:23 +02:00
1 parent cbbfce235e
commit 6289b01ed6
57 files changed
+2579 -224

No files matched your search

+221 -1
View File
@@ -83,6 +83,9 @@
{
"name": "restart"
},
{
"name": "settings"
},
{
"name": "summary"
},
@@ -1156,7 +1159,7 @@
"delete": {
"operationId": "deleteModelsName",
"summary": "Remove a model added at runtime",
"description": "A model declared in configuration is refused: it would come back on the next restart.",
"description": "A model declared in the cluster's settings is refused: it is changed there.",
"tags": [
"models"
],
@@ -2963,6 +2966,165 @@
}
}
},
"/api/v1/settings": {
"get": {
"operationId": "getSettings",
"summary": "The cluster's settings in force, or one earlier version",
"tags": [
"settings"
],
"parameters": [
{
"name": "version",
"in": "query",
"description": "An earlier version; the newest when absent.",
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SettingsView"
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
},
"put": {
"operationId": "putSettings",
"summary": "Change the cluster's settings: in force on every control plane within seconds",
"tags": [
"settings"
],
"parameters": [
{
"$ref": "#/components/parameters/Actor"
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SettingsRequest"
}
}
}
},
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SettingsView"
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/settings/history": {
"get": {
"operationId": "getSettingsHistory",
"summary": "Versions of the cluster's settings, newest first",
"tags": [
"settings"
],
"parameters": [
{
"name": "limit",
"in": "query",
"description": "How many; 20 when absent.",
"schema": {
"type": "string"
}
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/SettingsView"
}
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/settings/rollback": {
"post": {
"operationId": "postSettingsRollback",
"summary": "Put an earlier version of the cluster's settings back in force",
"tags": [
"settings"
],
"parameters": [
{
"$ref": "#/components/parameters/Actor"
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SettingsRollbackRequest"
}
}
}
},
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/SettingsView"
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/summary": {
"get": {
"operationId": "getSummary",
@@ -5350,6 +5512,64 @@
}
}
},
"SettingsRequest": {
"type": "object",
"description": "SettingsRequest is a new version of the cluster's settings, made from the version Base, which must still be the newest.",
"properties": {
"base": {
"type": "integer",
"format": "int64"
},
"body": {
"type": "string"
},
"note": {
"type": "string"
}
}
},
"SettingsRollbackRequest": {
"type": "object",
"description": "SettingsRollbackRequest puts an earlier version's settings back in force, as a new version.",
"properties": {
"version": {
"type": "integer",
"format": "int64"
},
"note": {
"type": "string"
}
}
},
"SettingsView": {
"type": "object",
"description": "SettingsView is one version of the cluster's settings.",
"properties": {
"version": {
"type": "integer",
"format": "int64"
},
"body": {
"type": "string"
},
"note": {
"type": "string"
},
"at": {
"type": "string",
"format": "date-time"
},
"by": {
"type": "string"
},
"local_keys": {
"type": "array",
"items": {
"type": "string"
}
}
}
},
"ShardView": {
"type": "object",
"description": "ShardView is one shard of a replica.",
+14 -8
View File
@@ -270,14 +270,20 @@ page's text.
## Settings
**server.yaml** (on a node, **agent.yaml**) is every key of the file, as a
tree of menus built from the configuration reference: the value set, or the
one that applies when nothing is, its type, and what it is for. Enter changes
a value or opens a block; on a list or map of blocks, Enter opens an entry and
**D** removes one. **U** puts a key back to its default. **S** saves: the file
is checked the way the service reads it before anything is written, and its
comments are kept. The service reads it when it starts, so the console offers
to restart it. Another control plane keeps its own copy.
**server.yaml** (on a node, **agent.yaml**) is every key, as a tree of menus
built from the configuration reference: the value set, or the one that
applies when nothing is, its type, and what it is for. Enter changes a value
or opens a block; on a list or map of blocks, Enter opens an entry and **D**
removes one. **U** puts a key back to its default. **S** saves.
On a control plane, the cluster's keys are read from and saved to the
cluster's settings in the store — the same on every control plane, in force
on all of them within seconds (`llmgw ctl config` does the same from a
shell) — and this machine's own keys (listeners, certificates, the store's
address) to its server.yaml, which the server notices and applies by itself.
Either is checked before anything is kept, and the file's comments survive.
On a node, agent.yaml is read when the agent starts, so the console offers
to restart it.
## Do
+7 -3
View File
@@ -5,9 +5,13 @@ policy can check, what it can do, and how one is attached to a team.
`llmgw server init` writes a `baseline` policy that applies to every
request, with Llama Guard 3 1B as its judge on a CPU; see
[install.md](install.md#2-the-control-plane) for what it checks. Without a
`guard:` section in `server.yaml` nothing is checked and nothing costs
anything. With one, every request is judged once it is authorised and
[install.md](install.md#2-the-control-plane) for what it checks. The
policies are part of the cluster's settings, kept in the store: `llmgw ctl
config edit` (or the console's Settings) changes them, and the change is in
force on every control plane within seconds, without a restart. A cluster
whose settings were brought in from a server.yaml without a `guard:` section
gets the `baseline` policy, without a judge. With no policies, nothing is
checked and nothing costs anything. With one, every request is judged once it is authorised and
within its limits, with its stored conversation spliced in, and before it is
sent to a model; and the answer is judged before the client sees it.
+32 -2
View File
@@ -241,8 +241,38 @@ password, change it, and issue yourself a key from the profile page.
### The configuration
`/etc/llmgw/server.yaml` is yours from here on. Three things in it are worth
knowing before you edit it:
There are two halves, and each change to either is in force within seconds,
on every machine it concerns, without anyone restarting anything.
**The cluster's settings** — guardrails, limits and prices, routing, the
reconciler, engines, notifications, the mirror's policy, models declared by
hand, offline, federation, facility budgets, backups' recipients — are kept in
the store, so every control plane runs on the same ones. Change them with
```
llmgw ctl config # what is in force, and its version
llmgw ctl config set router.retries 3 --note "flaky upstream"
llmgw ctl config edit # the whole document in $EDITOR
llmgw ctl config history # who changed what, when, and why
llmgw ctl config rollback 4 # back to version 4, as a new version
```
or on the console's **Settings** screen. A change is checked before it is
kept — one that would not load is refused — and recorded in the audit log.
Guardrails change live; anything else has every control plane start again on
the new settings, the standbys first and the leader half a minute later, so
one of them is always serving. Engines keep running through it.
The first control plane to start against a store brings in its server.yaml's
cluster keys — with a `baseline` guard policy, if the file had none — and
from then on those keys in any server.yaml are not read (the server says so
when it starts).
**This machine's own keys** stay in `/etc/llmgw/server.yaml`: `listen`,
`admin_listen`, `metrics_listen`, `agent_listen`, `agent_tls`, `auth.store`,
`secrets`, `tracing`, `monitoring`, `mirror.api` and `mirror.tls`, the backup
directory and key. The server notices the file change and starts again on it
by itself. Three things are worth knowing before you edit it:
- `admin_listen` is on loopback on purpose, and the server refuses any other
address unless `admin_remote: true`. Its API under `/api` adds models and