The cluster's settings live in the store, and every change takes effect
Everything in server.yaml that is not one machine's own is now a versioned document in the store (cluster_settings), the same on every control plane. The first start against a store imports the file's cluster keys, with the baseline guard policy when there is none; after that those keys in a file are ignored, with a warning. Each control plane polls the document's version and its own server.yaml's hash. A change is validated and applied: guard live (Gate.Replace), anything else by a graceful restart that systemd brings back, standbys first and the leader 30s later. A version that does not load on a machine is not applied; at start the newest one that loads is used, so a bad edit cannot keep every control plane down. llmgw ctl config show/get/set/unset/edit/history/rollback, the admin API under /api/v1/settings, and the console's Settings screen, which layers the store's document over the machine's file, write it. Commands beside the control plane that read cluster keys (init, node add, server add, backup, upgrade) read the layered configuration; init --bundle writes offline to the store once it holds the settings. A standby that becomes the leader re-reads models, engine choices and facility holds from the store. Retires appendGuardIfMissing: the import does it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
cbbfce235e
commit
6289b01ed6
57 files changed
+2579
-224
No files matched your search
+221
-1
@@ -83,6 +83,9 @@
|
||||
{
|
||||
"name": "restart"
|
||||
},
|
||||
{
|
||||
"name": "settings"
|
||||
},
|
||||
{
|
||||
"name": "summary"
|
||||
},
|
||||
@@ -1156,7 +1159,7 @@
|
||||
"delete": {
|
||||
"operationId": "deleteModelsName",
|
||||
"summary": "Remove a model added at runtime",
|
||||
"description": "A model declared in configuration is refused: it would come back on the next restart.",
|
||||
"description": "A model declared in the cluster's settings is refused: it is changed there.",
|
||||
"tags": [
|
||||
"models"
|
||||
],
|
||||
@@ -2963,6 +2966,165 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/settings": {
|
||||
"get": {
|
||||
"operationId": "getSettings",
|
||||
"summary": "The cluster's settings in force, or one earlier version",
|
||||
"tags": [
|
||||
"settings"
|
||||
],
|
||||
"parameters": [
|
||||
{
|
||||
"name": "version",
|
||||
"in": "query",
|
||||
"description": "An earlier version; the newest when absent.",
|
||||
"schema": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/SettingsView"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"401": {
|
||||
"$ref": "#/components/responses/Unauthorized"
|
||||
},
|
||||
"default": {
|
||||
"$ref": "#/components/responses/Error"
|
||||
}
|
||||
}
|
||||
},
|
||||
"put": {
|
||||
"operationId": "putSettings",
|
||||
"summary": "Change the cluster's settings: in force on every control plane within seconds",
|
||||
"tags": [
|
||||
"settings"
|
||||
],
|
||||
"parameters": [
|
||||
{
|
||||
"$ref": "#/components/parameters/Actor"
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/SettingsRequest"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/SettingsView"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"401": {
|
||||
"$ref": "#/components/responses/Unauthorized"
|
||||
},
|
||||
"default": {
|
||||
"$ref": "#/components/responses/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/settings/history": {
|
||||
"get": {
|
||||
"operationId": "getSettingsHistory",
|
||||
"summary": "Versions of the cluster's settings, newest first",
|
||||
"tags": [
|
||||
"settings"
|
||||
],
|
||||
"parameters": [
|
||||
{
|
||||
"name": "limit",
|
||||
"in": "query",
|
||||
"description": "How many; 20 when absent.",
|
||||
"schema": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/SettingsView"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"401": {
|
||||
"$ref": "#/components/responses/Unauthorized"
|
||||
},
|
||||
"default": {
|
||||
"$ref": "#/components/responses/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/settings/rollback": {
|
||||
"post": {
|
||||
"operationId": "postSettingsRollback",
|
||||
"summary": "Put an earlier version of the cluster's settings back in force",
|
||||
"tags": [
|
||||
"settings"
|
||||
],
|
||||
"parameters": [
|
||||
{
|
||||
"$ref": "#/components/parameters/Actor"
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"required": true,
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/SettingsRollbackRequest"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/SettingsView"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"401": {
|
||||
"$ref": "#/components/responses/Unauthorized"
|
||||
},
|
||||
"default": {
|
||||
"$ref": "#/components/responses/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/summary": {
|
||||
"get": {
|
||||
"operationId": "getSummary",
|
||||
@@ -5350,6 +5512,64 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"SettingsRequest": {
|
||||
"type": "object",
|
||||
"description": "SettingsRequest is a new version of the cluster's settings, made from the version Base, which must still be the newest.",
|
||||
"properties": {
|
||||
"base": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"body": {
|
||||
"type": "string"
|
||||
},
|
||||
"note": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"SettingsRollbackRequest": {
|
||||
"type": "object",
|
||||
"description": "SettingsRollbackRequest puts an earlier version's settings back in force, as a new version.",
|
||||
"properties": {
|
||||
"version": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"note": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"SettingsView": {
|
||||
"type": "object",
|
||||
"description": "SettingsView is one version of the cluster's settings.",
|
||||
"properties": {
|
||||
"version": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"body": {
|
||||
"type": "string"
|
||||
},
|
||||
"note": {
|
||||
"type": "string"
|
||||
},
|
||||
"at": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"by": {
|
||||
"type": "string"
|
||||
},
|
||||
"local_keys": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"ShardView": {
|
||||
"type": "object",
|
||||
"description": "ShardView is one shard of a replica.",
|
||||
|
||||
+14
-8
@@ -270,14 +270,20 @@ page's text.
|
||||
|
||||
## Settings
|
||||
|
||||
**server.yaml** (on a node, **agent.yaml**) is every key of the file, as a
|
||||
tree of menus built from the configuration reference: the value set, or the
|
||||
one that applies when nothing is, its type, and what it is for. Enter changes
|
||||
a value or opens a block; on a list or map of blocks, Enter opens an entry and
|
||||
**D** removes one. **U** puts a key back to its default. **S** saves: the file
|
||||
is checked the way the service reads it before anything is written, and its
|
||||
comments are kept. The service reads it when it starts, so the console offers
|
||||
to restart it. Another control plane keeps its own copy.
|
||||
**server.yaml** (on a node, **agent.yaml**) is every key, as a tree of menus
|
||||
built from the configuration reference: the value set, or the one that
|
||||
applies when nothing is, its type, and what it is for. Enter changes a value
|
||||
or opens a block; on a list or map of blocks, Enter opens an entry and **D**
|
||||
removes one. **U** puts a key back to its default. **S** saves.
|
||||
|
||||
On a control plane, the cluster's keys are read from and saved to the
|
||||
cluster's settings in the store — the same on every control plane, in force
|
||||
on all of them within seconds (`llmgw ctl config` does the same from a
|
||||
shell) — and this machine's own keys (listeners, certificates, the store's
|
||||
address) to its server.yaml, which the server notices and applies by itself.
|
||||
Either is checked before anything is kept, and the file's comments survive.
|
||||
On a node, agent.yaml is read when the agent starts, so the console offers
|
||||
to restart it.
|
||||
|
||||
## Do
|
||||
|
||||
|
||||
+7
-3
@@ -5,9 +5,13 @@ policy can check, what it can do, and how one is attached to a team.
|
||||
|
||||
`llmgw server init` writes a `baseline` policy that applies to every
|
||||
request, with Llama Guard 3 1B as its judge on a CPU; see
|
||||
[install.md](install.md#2-the-control-plane) for what it checks. Without a
|
||||
`guard:` section in `server.yaml` nothing is checked and nothing costs
|
||||
anything. With one, every request is judged once it is authorised and
|
||||
[install.md](install.md#2-the-control-plane) for what it checks. The
|
||||
policies are part of the cluster's settings, kept in the store: `llmgw ctl
|
||||
config edit` (or the console's Settings) changes them, and the change is in
|
||||
force on every control plane within seconds, without a restart. A cluster
|
||||
whose settings were brought in from a server.yaml without a `guard:` section
|
||||
gets the `baseline` policy, without a judge. With no policies, nothing is
|
||||
checked and nothing costs anything. With one, every request is judged once it is authorised and
|
||||
within its limits, with its stored conversation spliced in, and before it is
|
||||
sent to a model; and the answer is judged before the client sees it.
|
||||
|
||||
|
||||
+32
-2
@@ -241,8 +241,38 @@ password, change it, and issue yourself a key from the profile page.
|
||||
|
||||
### The configuration
|
||||
|
||||
`/etc/llmgw/server.yaml` is yours from here on. Three things in it are worth
|
||||
knowing before you edit it:
|
||||
There are two halves, and each change to either is in force within seconds,
|
||||
on every machine it concerns, without anyone restarting anything.
|
||||
|
||||
**The cluster's settings** — guardrails, limits and prices, routing, the
|
||||
reconciler, engines, notifications, the mirror's policy, models declared by
|
||||
hand, offline, federation, facility budgets, backups' recipients — are kept in
|
||||
the store, so every control plane runs on the same ones. Change them with
|
||||
|
||||
```
|
||||
llmgw ctl config # what is in force, and its version
|
||||
llmgw ctl config set router.retries 3 --note "flaky upstream"
|
||||
llmgw ctl config edit # the whole document in $EDITOR
|
||||
llmgw ctl config history # who changed what, when, and why
|
||||
llmgw ctl config rollback 4 # back to version 4, as a new version
|
||||
```
|
||||
|
||||
or on the console's **Settings** screen. A change is checked before it is
|
||||
kept — one that would not load is refused — and recorded in the audit log.
|
||||
Guardrails change live; anything else has every control plane start again on
|
||||
the new settings, the standbys first and the leader half a minute later, so
|
||||
one of them is always serving. Engines keep running through it.
|
||||
|
||||
The first control plane to start against a store brings in its server.yaml's
|
||||
cluster keys — with a `baseline` guard policy, if the file had none — and
|
||||
from then on those keys in any server.yaml are not read (the server says so
|
||||
when it starts).
|
||||
|
||||
**This machine's own keys** stay in `/etc/llmgw/server.yaml`: `listen`,
|
||||
`admin_listen`, `metrics_listen`, `agent_listen`, `agent_tls`, `auth.store`,
|
||||
`secrets`, `tracing`, `monitoring`, `mirror.api` and `mirror.tls`, the backup
|
||||
directory and key. The server notices the file change and starts again on it
|
||||
by itself. Three things are worth knowing before you edit it:
|
||||
|
||||
- `admin_listen` is on loopback on purpose, and the server refuses any other
|
||||
address unless `admin_remote: true`. Its API under `/api` adds models and
|
||||
|
||||
Reference in new issue
Block a user