Compare commits

..
4 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 c52b73feec Reference: the agents section of the cluster's settings
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 18:07:00 +02:00
Ilker ManapandClaude Opus 5.5 dc243f8840 The nodes' agent settings kept with the cluster's, agent.yaml down to what an agent needs to start
agents.default and agents.nodes.<id> in the cluster's settings hold
everything agent.yaml can say but the agent's id, control planes, address and
certificate, which stay in the file and are refused there. The control plane
sends each node its settings on register and, a second apart, when they
change; an agent whose settings changed checks they load beside its file,
keeps them and starts again on them, engines serving throughout. An edited
agent.yaml that loads restarts the agent too. History, audit, ctl config, the
console and the portal's settings page cover them as the rest of the
settings.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 18:06:36 +02:00
Ilker ManapandClaude Opus 5.5 659f20dc9a The cluster's settings edited on the portal, by administrators
/admin/settings edits the document llmgw ctl config edit opens: checked as a
control plane would load it, refused when made from a version since replaced
(the edit kept), its history viewed and rolled back as a new version, every
change audited by size and never by contents, since it can hold webhook
secrets. Administrators only; in English and Turkish.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 17:23:13 +02:00
Ilker ManapandClaude Opus 5.5 6bc6a67fb0 README: the releases since v0.11.0 as released, and knowledge bases
CI / test (push) Successful in 7m51s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 17:17:40 +02:00
36 changed files with 1498 additions and 275 deletions

No files matched your search

+22 -2
View File
@@ -807,14 +807,34 @@ Patroni configuration. The firewall's allowed set is the cluster's membership
removed changes every machine's rules. Retires `firewall.Admit` and
`firewall_networks` in server.yaml (imported into the store).
### E3. agent.yaml down to bootstrap (M)
### E3. agent.yaml down to bootstrap (M) — done 2026-10-08 (branch `settings-e`)
agent.yaml keeps the control planes' addresses, its certificate and its id.
Labels, fabrics, engines, heartbeat and supervisor settings move to the
store per node and reach the agent over its stream; a change to the file is
noticed and the agent restarts itself.
Built as a section of the cluster's settings rather than a store of its own,
so history, audit, `ctl config`, the console and the portal page cover it:
`agents.default` and `agents.nodes.<id>` (config.AgentSettings; bootstrap
keys node_id/server(s)/hostname/mgmt_addr/tls refused; each node's merge
loaded as an agent would, so a bad fabric is refused on save). A live key:
the control plane sends ControlMessage.AgentSettings on register and, a
second apart per node, on change; the agent compares with what it started on,
checks it loads beside its agent.yaml (else logs and keeps running), keeps it
in /var/lib/llmgw-agent/agent-settings.yaml and restarts (engines adopted).
At start: agent.yaml < kept settings < LLMGW_ env; kept settings that no
longer load are set aside, never a reason not to start. agent.yaml is
watched every 10 s and restarted on when it changes and loads. Left: file
keys are not imported into the settings (they keep working under them);
`node add` still writes a full agent.yaml; labels have two homes now
(`ctl label`, which wins, and agents.*.labels).
### E4. Web settings page (S)
### E4. Web settings page (S) — done 2026-10-08 (branch `settings-e`)
The same document, edited on /admin/settings, for whoever has no terminal.
Built: administrators only (staff roles get 404: webhook secrets), checked by
the same function as PUT /api/v1/settings, made from a base version or
refused on a newer one (the edit kept in the editor), history with view and
roll back as a new version, audit with the size not the body, English and
Turkish, a Settings link in the menu.
---
+18
View File
@@ -24,6 +24,24 @@ is only released once its **exit criterion** is met, not when its code lands.
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
APIs change between minors without a deprecation period.
## Unreleased
### Added
- **The cluster's settings on the portal** (BACKLOG E4): `/admin/settings`
for administrators. It edits the document `llmgw ctl config edit` opens:
checked as a control plane would load it, refused when made from a
version since replaced, with the history to view and roll back. Changes
are audited by size, never by contents.
- **The nodes' agent settings in the cluster's settings** (BACKLOG E3):
`agents.default` and `agents.nodes.<id>` hold everything agent.yaml can
say but the agent's id, control planes, address and certificate. They are
sent to the agents as they change; an agent starts again on new settings,
or on an edited agent.yaml, with its engines serving throughout. Settings
that would not load are refused on save, and set aside on a node rather
than keep its agent from starting. Nodes need this release to follow
them.
## v0.14.0 — 2026-10-08
Knowledge bases: a company's documents, put in by people from their own
+63 -33
View File
@@ -3,8 +3,9 @@
A single-binary LLM gateway for clusters with multiple InfiniBand fabrics,
NVLink domains and heterogeneous GPUs.
It serves LLMs running on your own GPU servers (vLLM / SGLang) behind one
OpenAI-compatible endpoint. Design document: [llmgw-plan.md](llmgw-plan.md).
It serves LLMs running on your own GPU and CPU servers (vLLM, SGLang,
llama.cpp, TensorRT-LLM) behind one OpenAI- and Anthropic-compatible
endpoint, and answers from your own documents. Design document: [llmgw-plan.md](llmgw-plan.md).
## Getting started
@@ -37,6 +38,9 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md).
| v0.9.0 | — | Managing people and the tenancy tree | released |
| v0.10.0 | — | InfiniBand fabrics and NVLink, doctor, node add, engines installed by the nodes | released |
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released |
| v1.0.0 | 5 | Tenancy, prefix affinity, benchmarking | planned |
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
@@ -44,38 +48,60 @@ control plane places replicas — including ones larger than a single machine
models load when someone asks for them and stop when nobody does, and keys,
budgets and rate limits decide who may use any of it.
Since v0.11.0, not yet released (see [CHANGELOG.md](CHANGELOG.md),
"Unreleased"):
Since v0.11.0 (each release's detail is in [CHANGELOG.md](CHANGELOG.md)):
- **Security hardening** — signed releases only, a token on the admin API,
engines not run as root, TLS for the agent port, etcd and Patroni, a
firewall for llmgw's internal ports, encrypted backups, bounded
connections, sign-in throttling, a Content-Security-Policy.
- **Guardrails** — policies that check requests and answers (PII, secrets,
prompt injection, deny patterns, a judge model such as Llama Guard), in
observe or enforce mode, attached to any part of the tenancy tree:
- **Knowledge bases** (v0.14.0). People put a company's documents in from
their own machines with `llmgw-kb`, a separate download for Windows,
macOS and Linux that reads PDF, Office, OpenDocument, mail and many more
formats and sends text and vectors, never the files. Collections are kept
apart per tenant, down to row-level security in PostgreSQL. A chat
request, or a part of the tree by default, is answered from them with
its sources: [docs/knowledge.md](docs/knowledge.md).
- **One command for the whole system** (v0.13.0): `llmgw stop`, `start` and
`restart`. Qwen3Guard is the guardrails' judge, on a GPU and on CPUs
together.
- **Settings that apply when changed** (v0.12.0). The cluster's
configuration lives in the store, the same on every control plane and in
force within seconds (`llmgw ctl config`, the console's Settings). The
firewall can be switched on and off for the whole cluster.
- **Security hardening** (v0.11.2): signed releases only, a token on the
admin API, engines not run as root, TLS for the agent port, etcd and
Patroni, a firewall for llmgw's internal ports, encrypted backups,
bounded connections, sign-in throttling, a Content-Security-Policy.
- **Guardrails** (v0.11.2). Policies check requests and answers for PII,
secrets, prompt injection, deny patterns, and with a judge model; they
observe or enforce, and attach to any part of the tenancy tree:
[docs/guardrails.md](docs/guardrails.md).
- **Administration** — an audit log of every change (`ctl audit`), staff
roles (viewer, operator, billing, admin), single sign-on through OpenID
Connect and provisioning through SCIM 2.0, key lifetime limits and
rotation with overlap (`keys ttl`, `keys rotate`), usage export as CSV
(`ctl usage export`).
- **Notifications** — webhooks, signed or Slack-shaped, for a model parked,
a node lost, a budget spent and new weights published (`notify.webhooks`,
`ctl notify test`), and Alertmanager receiver templates.
- **Traffic and models** — request tags, canaries of new weights with
automatic rollback, quality suites gating updates, a response cache,
kept and replayed requests, batches, Anthropic's Messages API, more of
OpenAI's (structured output, images, rerank, audio, moderations),
scaling by the clock, SLO-aware admission, TensorRT-LLM, MIG, models
imported from Ollama, routing by what each vLLM replica's cache holds,
LoRA adapters loaded on demand, other llmgw clusters as providers, and
placement that knows racks, PDUs and power caps.
- **Operations** — `server restore` and off-machine backup copies, secrets
in Vault, forgetting a person and keeping a project's requests in the
cluster, offline installs from a bundle, a Helm chart, tracing from the
control plane to the engines, the admin API in OpenAPI with a Terraform
provider, and the portal in English and Turkish with its docs at /docs.
- **Administration** (v0.11.2):
- an audit log of every change (`ctl audit`);
- staff roles (viewer, operator, billing, admin);
- single sign-on through OpenID Connect and provisioning through SCIM
2.0;
- key lifetime limits, and rotation with overlap (`keys ttl`, `keys
rotate`);
- usage export as CSV (`ctl usage export`).
- **Notifications** (v0.11.2): webhooks, signed or Slack-shaped, for a
model parked, a node lost, a budget spent and new weights published, and
Alertmanager receiver templates.
- **Traffic and models** (v0.11.2):
- request tags, canaries of new weights with automatic rollback, and
quality suites gating updates;
- a response cache, kept and replayed requests, and batches;
- Anthropic's Messages API, and more of OpenAI's (structured output,
images, rerank, audio, moderations);
- scaling by the clock, and SLO-aware admission;
- TensorRT-LLM, MIG, and models imported from Ollama;
- routing by what each vLLM replica's cache holds, and LoRA adapters
loaded on demand;
- other llmgw clusters as providers, and placement that knows racks,
PDUs and power caps.
- **Operations** (v0.11.2):
- `server restore`, off-machine backup copies, and secrets in Vault;
- forgetting a person, and keeping a project's requests in the cluster;
- offline installs from a bundle, and a Helm chart;
- tracing from the control plane to the engines;
- the admin API in OpenAPI, with a Terraform provider;
- the portal in English and Turkish, with its docs at /docs.
Every one of these is in the console as well as on the command line:
[docs/console.md](docs/console.md).
@@ -124,6 +150,9 @@ Every page is also in the binary: `llmgw docs`, or **Manual** in the console.
official OpenAI and Anthropic SDKs, and llmgw's extensions
- [docs/guardrails.md](docs/guardrails.md) — checking requests and answers:
policies, detectors, a judge model, and where each applies
- [docs/knowledge.md](docs/knowledge.md) — knowledge bases: collections,
who may read them, `llmgw-kb` on people's machines, and answers from
documents
- [docs/choosing-models.md](docs/choosing-models.md) — which models to run,
how many of each, and where the weights should live
@@ -152,7 +181,7 @@ Every page is also in the binary: `llmgw docs`, or **Manual** in the console.
## Building
```sh
make build # bin/llmgw
make build # bin/llmgw and bin/llmgw-kb
make test # unit tests
make race # with the race detector
```
@@ -169,6 +198,7 @@ llmgw ctl cordon gpu-a-01 # stop new placement, leave what runs
sudo llmgw doctor # on a node: what its GPUs, NVLink and fabric need
llmgw ctl audit --since 24h # who changed what
llmgw ctl usage export --from 2026-09-01 --to 2026-10-01 --by project,model
llmgw ctl kb list # knowledge-base collections, grants, defaults
```
Bare `llmgw` opens the console, which does all of this from one screen.
File diff suppressed because it is too large. Load diff
+12
View File
@@ -73,6 +73,7 @@ message ControlMessage {
ControlPlanes control_planes = 11;
Firewall firewall = 13;
RestartAgent restart_agent = 14;
AgentSettings agent_settings = 15;
}
// trace is the W3C trace context (traceparent, and tracestate when there
// is one) of the control plane's span that sent this command, so what the
@@ -103,6 +104,17 @@ message Firewall {
int32 engine_ports_to = 4;
}
// AgentSettings is what the cluster's settings say of this node's agent
// (agents.default, with agents.nodes.<id> over it), sent when it registers
// and whenever they change. The agent keeps it beside agent.yaml, which
// holds only what it needs to start; over that file, it is what the agent
// runs on. A change from what it runs on has the agent start again on it,
// its engines serving throughout.
message AgentSettings {
// yaml is agent.yaml's keys, as YAML; empty for none.
string yaml = 1;
}
// ControlPlanes is every control plane an agent may dial, sent by the leader
// when an agent registers and whenever the list changes. The agent keeps it
// beside its configuration, so a second control plane added after the node
+35 -2
View File
@@ -258,8 +258,14 @@ llmgw ctl config history # who changed what, when, and why
llmgw ctl config rollback 4 # back to version 4, as a new version
```
or on the console's **Settings** screen. A change is checked before it is
kept — one that would not load is refused — and recorded in the audit log.
or on the console's **Settings** screen, or in a browser on the portal's
**Settings** page (`/admin/settings`, for administrators only: the document
can hold webhook secrets). The page edits the same document, lists its
history, and puts an earlier version back. A change is checked before it is
kept, so one that would not load is refused. A change made from a version
someone else has replaced since is refused too, rather than overwriting
theirs. Every change is recorded in the audit log, with the document's size
but not its contents.
Guardrails change live; anything else has every control plane start again on
the new settings, the standbys first and the leader half a minute later, so
one of them is always serving. Engines keep running through it.
@@ -269,6 +275,33 @@ cluster keys — with a `baseline` guard policy, if the file had none — and
from then on those keys in any server.yaml are not read (the server says so
when it starts).
**The nodes' agents** are set the same way, under `agents`. `agents.default`
is for every node, and `agents.nodes.<node-id>` is for one node, laid over
the default:
```
llmgw ctl config set agents.default.heartbeat_interval 5s
llmgw ctl config set agents.nodes.cuda01.labels.gpu rtx6000
llmgw ctl config edit # fabrics, engines, supervisor, discovery…
```
Each node's `agent.yaml` then needs only what its agent must know before it
can be told anything: `node_id`, `server` or `servers`, `hostname`,
`mgmt_addr` and `tls`. Those keys are refused under `agents`.
- **Precedence.** Anything else in the file is still read, with the
cluster's settings over it.
- **Applying a change.** The control plane sends each node its settings, a
second apart. An agent whose settings changed keeps them beside its
`agent.yaml` (in `/var/lib/llmgw-agent/agent-settings.yaml`) and starts
again on them, its engines serving throughout.
- **Bad settings.** Settings that would not load are refused when they are
saved. If they somehow do not load beside one node's own file, that agent
keeps running on what it has and logs why.
- **Editing `agent.yaml` itself.** The agent notices the change within ten
seconds and starts again on it. If the edited file does not load, the
agent reports it and keeps running on what it had.
**This machine's own keys** stay in `/etc/llmgw/server.yaml`: `listen`,
`admin_listen`, `metrics_listen`, `agent_listen`, `agent_tls`, `auth.store`,
`secrets`, `tracing`, `monitoring`, `mirror.api` and `mirror.tls`, the backup
+30
View File
@@ -26,6 +26,8 @@ rather than repeated.
| `queue_capacity_per_band` | `int` | [Admission](#admission) |
| `queue_capacity_total` | `int` | [Admission](#admission) |
| `interval` | `time.Duration` | [Admission](#admission) |
| `default` | `map[string]any` | [AgentSettings](#agentsettings) |
| `nodes` | `map[string]map[string]any` | [AgentSettings](#agentsettings) |
| `enabled` | `bool` | [Auth](#auth) |
| `store` | `string` | [Auth](#auth) |
| `cache_ttl` | `time.Duration` | [Auth](#auth) |
@@ -294,6 +296,7 @@ rather than repeated.
| `mail` | `mail.Config` | [Server](#server) |
| `serving` | `ServingSettings` | [Server](#server) |
| `firewall` | `FirewallSettings` | [Server](#server) |
| `agents` | `AgentSettings` | [Server](#server) |
| `firewall_networks` | `[]string` | [Server](#server) |
| `backup_recipients` | `[]string` | [Server](#server) |
| `backup_copy_to` | `[]string` | [Server](#server) |
@@ -469,6 +472,21 @@ Zero takes half of all the bands' capacities together.
`time.Duration`
## AgentSettings
### `default`
`map[string]any`
Default is agent.yaml's keys for every node.
### `nodes`
`map[string]map[string]any`
Nodes are agent.yaml's keys for one node each, by node id, over
Default.
## Auth
### `enabled`
@@ -2520,6 +2538,18 @@ Firewall turns llmgw's firewall on every machine on or off.
Takes a [FirewallSettings](#firewallsettings) block.
### `agents`
`AgentSettings`
Agents are the nodes' agent settings: agent.yaml's keys but those an
agent needs to start, for every node (default) and for one
(nodes.<id>). Sent to the agents as they change; an agent whose
settings changed starts again on them, its engines serving
throughout.
Takes a [AgentSettings](#agentsettings) block.
### `firewall_networks`
`[]string`
+6
View File
@@ -136,6 +136,10 @@ type Agent struct {
// by mu.
kv *kvevents.Mirror
kvSubs map[string]*kvevents.Subscriber
// settingsCheck says whether the cluster's settings for this agent
// would load beside its agent.yaml; see SetSettingsCheck.
settingsCheck func([]byte) error
}
// connectHTTPClient is the HTTP client connect uses; an interface so tests can
@@ -880,6 +884,8 @@ func (a *Agent) handle(ctx context.Context, msg *llmgwv1.ControlMessage, out cha
a.setControlPlanes(m.ControlPlanes)
case *llmgwv1.ControlMessage_Firewall:
go a.applyFirewall(m.Firewall)
case *llmgwv1.ControlMessage_AgentSettings:
go a.applySettings(m.AgentSettings)
case *llmgwv1.ControlMessage_RestartAgent:
go func() {
a.log.Info("restarting, as the control plane asks; the engines keep serving",
+5
View File
@@ -103,6 +103,11 @@ type Config struct {
Tracing obs.Tracing `koanf:"tracing"`
Version string
// Settings is the cluster's settings for this agent it was started on,
// as the control plane sent them: what a newly sent version is
// compared with.
Settings string `koanf:"-"`
}
// DefaultConfig returns sensible defaults.
+109
View File
@@ -0,0 +1,109 @@
package agent
import (
"context"
"crypto/sha256"
"os"
"path/filepath"
"time"
llmgwv1 "gitea.mynodes.xyz/ilker/llmgw/api/gen/llmgw/v1"
)
// The cluster's settings for this agent (BACKLOG E3): what agent.yaml can
// say but the agent's id, control planes, address and certificate, kept by
// the control planes and sent over the stream. Kept here beside agent.yaml,
// which then holds only what the agent needs to start; at start the two are
// read together, these over the file.
// SettingsFile keeps the settings the control plane last sent.
var SettingsFile = "/var/lib/llmgw-agent/agent-settings.yaml"
// ReadSettings is what SettingsFile holds, nil when nothing.
func ReadSettings() []byte {
b, err := os.ReadFile(SettingsFile)
if err != nil {
return nil
}
return b
}
// SetSettingsCheck says how to tell whether settings would load beside this
// agent's own file; settings that would not are not taken.
func (a *Agent) SetSettingsCheck(f func([]byte) error) { a.settingsCheck = f }
// applySettings takes the settings the control plane sent: nothing when they
// are what this agent runs on, and otherwise kept and started again on.
func (a *Agent) applySettings(m *llmgwv1.AgentSettings) {
next := m.GetYaml()
if next == a.cfg.Settings {
return
}
if a.settingsCheck != nil {
if err := a.settingsCheck([]byte(next)); err != nil {
// Kept running on what it has: settings that will not load
// here must not take the node out of service.
a.log.Error("the cluster's settings for this agent do not load beside its agent.yaml; running on what it has",
"err", err)
return
}
}
if err := os.MkdirAll(filepath.Dir(SettingsFile), 0o755); err != nil {
a.log.Error("could not keep the cluster's settings for this agent", "err", err)
return
}
tmp := SettingsFile + ".new"
if err := os.WriteFile(tmp, []byte(next), 0o600); err != nil {
a.log.Error("could not keep the cluster's settings for this agent", "err", err)
return
}
if err := os.Rename(tmp, SettingsFile); err != nil {
a.log.Error("could not keep the cluster's settings for this agent", "err", err)
return
}
a.log.Info("the cluster's settings for this agent changed; starting again on them, the engines keep serving")
time.Sleep(300 * time.Millisecond)
exitForUpgrade()
}
// WatchFile starts the agent again when its configuration file changes and
// still loads: agent.yaml is read only at start, and a change to it that
// waited for somebody to restart the agent was a change nobody saw take
// effect.
func (a *Agent) WatchFile(ctx context.Context, path string, loads func() error) {
if path == "" {
return
}
sum := func() [32]byte {
b, _ := os.ReadFile(path) //nolint:gosec // the agent's own configuration
return sha256.Sum256(b)
}
was := sum()
t := time.NewTicker(watchEvery)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
}
now := sum()
if now == was {
continue
}
if err := loads(); err != nil {
// Once per change: an editor saving halfway must not stop the
// agent, and a broken file is reported, not restarted into.
a.log.Error("agent.yaml changed and does not load; running on what it had", "path", path, "err", err)
was = now
continue
}
a.log.Info("agent.yaml changed; starting again on it, the engines keep serving", "path", path)
time.Sleep(300 * time.Millisecond)
exitForUpgrade()
return
}
}
// watchEvery is how often agent.yaml is looked at.
var watchEvery = 10 * time.Second
+88
View File
@@ -0,0 +1,88 @@
package agent
import (
"errors"
"os"
"path/filepath"
"sync/atomic"
"testing"
"time"
llmgwv1 "gitea.mynodes.xyz/ilker/llmgw/api/gen/llmgw/v1"
)
func settingsAgent(t *testing.T, running string) (*Agent, chan struct{}) {
t.Helper()
oldFile, oldExit := SettingsFile, exitForUpgrade
SettingsFile = filepath.Join(t.TempDir(), "agent-settings.yaml")
exited := make(chan struct{}, 1)
exitForUpgrade = func() { exited <- struct{}{} }
t.Cleanup(func() { SettingsFile, exitForUpgrade = oldFile, oldExit })
cfg := Config{Server: "127.0.0.1:7443", NodeID: "n1", WeightsCache: t.TempDir(), EnginesDir: t.TempDir(), Settings: running}
cfg.Superv.WorkDir = t.TempDir()
a, err := New(cfg, nil, nil)
if err != nil {
t.Fatal(err)
}
return a, exited
}
// exitedSoon is whether the agent exited within a second: it waits a
// moment for its log line to leave first.
func exitedSoon(ch chan struct{}) bool {
select {
case <-ch:
return true
case <-time.After(time.Second):
return false
}
}
func TestAnAgentOnItsSettingsStaysAndOneOffThemStartsAgain(t *testing.T) {
a, exited := settingsAgent(t, "heartbeat_interval: 5s\n")
a.applySettings(&llmgwv1.AgentSettings{Yaml: "heartbeat_interval: 5s\n"})
if exitedSoon(exited) || ReadSettings() != nil {
t.Error("settings it already runs on restarted it")
}
a.applySettings(&llmgwv1.AgentSettings{Yaml: "heartbeat_interval: 2s\n"})
if !exitedSoon(exited) || string(ReadSettings()) != "heartbeat_interval: 2s\n" {
t.Errorf("new settings: kept %q", ReadSettings())
}
}
func TestSettingsThatWouldNotLoadAreNotTaken(t *testing.T) {
a, exited := settingsAgent(t, "")
a.SetSettingsCheck(func([]byte) error { return errors.New("fabric ib-a lists no HCAs") })
a.applySettings(&llmgwv1.AgentSettings{Yaml: "fabrics: [{name: ib-a}]\n"})
if exitedSoon(exited) || ReadSettings() != nil {
t.Error("settings that do not load were taken")
}
}
func TestAChangedAgentYamlThatLoadsStartsTheAgentAgain(t *testing.T) {
a, exited := settingsAgent(t, "")
old := watchEvery
watchEvery = 10 * time.Millisecond
t.Cleanup(func() { watchEvery = old })
p := filepath.Join(t.TempDir(), "agent.yaml")
_ = os.WriteFile(p, []byte("node_id: n1\n"), 0o600)
var loads atomic.Bool
check := func() error {
if loads.Load() {
return nil
}
return errors.New("broken")
}
go a.WatchFile(t.Context(), p, check)
time.Sleep(30 * time.Millisecond)
// Saved broken: reported, not restarted into.
_ = os.WriteFile(p, []byte("node_id: [\n"), 0o600)
if exitedSoon(exited) {
t.Fatal("a broken agent.yaml restarted the agent")
}
loads.Store(true)
_ = os.WriteFile(p, []byte("node_id: n1\nlabels: {a: b}\n"), 0o600)
if !exitedSoon(exited) {
t.Error("a changed agent.yaml did not restart the agent")
}
}
+22 -1
View File
@@ -40,10 +40,23 @@ func newAgentCmd() *cobra.Command {
RunE: func(cmd *cobra.Command, _ []string) error {
log := logger()
cfg, err := config.LoadAgent(configPath)
// The cluster's settings for this agent over its file, as the
// control plane last sent them. Settings that no longer load
// beside the file are set aside rather than keeping the agent
// from starting: the control plane sends them again, and says
// why they are refused.
settings := agent.ReadSettings()
cfg, err := config.LoadAgentWith(configPath, settings)
if err != nil && settings != nil {
log.Warn("the cluster's settings kept for this agent do not load beside its agent.yaml; starting without them",
"err", err)
settings = nil
cfg, err = config.LoadAgent(configPath)
}
if err != nil {
return err
}
cfg.Settings = string(settings)
if server != "" {
cfg.Server = server
}
@@ -89,9 +102,17 @@ func newAgentCmd() *cobra.Command {
if err != nil {
return err
}
a.SetSettingsCheck(func(b []byte) error {
_, err := config.LoadAgentWith(configPath, b)
return err
})
ctx, stop := signal.NotifyContext(cmd.Context(), syscall.SIGINT, syscall.SIGTERM)
defer stop()
go a.WatchFile(ctx, configPath, func() error {
_, err := config.LoadAgentWith(configPath, agent.ReadSettings())
return err
})
log.Info("agent starting",
"node", cfg.NodeID, "server", cfg.Server,
+2
View File
@@ -241,6 +241,8 @@ var liveSettings = []string{
"serving",
// Sent to the nodes and this machine's firewall unit as they change.
"firewall", "firewall_networks",
// Sent to the nodes' agents as they change.
"agents",
// Read by server init, node add and backups, not by the server.
"backup_recipients", "backup_copy_to",
}
+15
View File
@@ -263,6 +263,15 @@ func newServerCmd() *cobra.Command {
return &llmgwv1.Firewall{Enabled: c.Firewall.Enabled, Networks: c.FirewallNetworks,
EnginePortsFrom: int32(from), EnginePortsTo: int32(to)} //nolint:gosec // ports
})
// What each node's agent runs on, as the settings say now.
svc.SetAgentSettings(func(node string) string {
b, err := config.Marshal(current.Load().Agents.For(node))
if err != nil {
log.Warn("could not write a node's agent settings", "node", node, "err", err)
return ""
}
return string(b)
})
ctl.Reconciler().SetNotifier(notifier)
admin = ctl
agentPath, agentHandler = svc.Handler()
@@ -382,6 +391,8 @@ func newServerCmd() *cobra.Command {
// machine's own keys before it is kept.
api.SetSettings(&adminapi.Settings{Store: st, LocalKeys: config.LocalKeys(),
Check: func(body []byte) error { return checkClusterSettings(configPath, body) }})
// And the same document on the portal, for administrators.
rt.SetSettingsCheck(func(body []byte) error { return checkClusterSettings(configPath, body) }, config.LocalKeys())
}
// Catalog changes need somewhere to persist, so this is wired
// only when there is a store. Without one, the endpoints say so
@@ -717,6 +728,10 @@ func newServerCmd() *cobra.Command {
agentSvc.BroadcastFirewall()
}
}
if slices.ContainsFunc(changed, func(c string) bool { return strings.HasPrefix(c, "agents") }) && agentSvc != nil {
// A moment apart per node, so off this goroutine.
go agentSvc.BroadcastAgentSettings()
}
if slices.ContainsFunc(changed, func(c string) bool { return strings.HasPrefix(c, "guard") }) {
if gate == nil {
// Built at start, and only when there are policies.
+11 -1
View File
@@ -18,7 +18,12 @@ import (
//
// An empty path loads the defaults, which with --server and --node-id on the
// command line is enough for a smoke test on a node with no IB.
func LoadAgent(path string) (agent.Config, error) {
func LoadAgent(path string) (agent.Config, error) { return LoadAgentWith(path, nil) }
// LoadAgentWith is LoadAgent with the cluster's settings for this node
// (agents.default and agents.nodes.<id>, as YAML) laid over the file: the
// file is what the agent needs to start, the settings what it runs on.
func LoadAgentWith(path string, settings []byte) (agent.Config, error) {
cfg := agent.DefaultConfig()
k := koanf.New(".")
@@ -30,6 +35,11 @@ func LoadAgent(path string) (agent.Config, error) {
return cfg, fmt.Errorf("parsing %s: %w", path, err)
}
}
if len(settings) > 0 {
if err := k.Load(bytesProvider(settings), yaml.Parser()); err != nil {
return cfg, fmt.Errorf("parsing the cluster's settings for this agent: %w", err)
}
}
err := k.Load(env.Provider(".", env.Opt{
Prefix: "LLMGW_",
+110
View File
@@ -0,0 +1,110 @@
package config
import (
"fmt"
"slices"
"sort"
)
// AgentSettings are the nodes' agent settings, kept with the cluster's
// (BACKLOG E3): everything agent.yaml can say except what an agent needs to
// start — its id, the control planes to dial, its address and its
// certificate. Default applies to every node, and Nodes, by node id, over
// it; both over the node's own agent.yaml.
type AgentSettings struct {
// Default is agent.yaml's keys for every node.
Default map[string]any `koanf:"default"`
// Nodes are agent.yaml's keys for one node each, by node id, over
// Default.
Nodes map[string]map[string]any `koanf:"nodes"`
}
// agentBootstrapKeys are what an agent needs before it can be told anything:
// they stay in agent.yaml.
var agentBootstrapKeys = []string{"node_id", "server", "servers", "hostname", "mgmt_addr", "tls"}
// For is the settings for one node: Default with the node's own over it.
// Empty when there are none.
func (a AgentSettings) For(node string) map[string]any {
out := clone(stringKeys(a.Default))
if n, ok := a.Nodes[node]; ok {
merge(out, stringKeys(n))
}
return out
}
// Check refuses settings that set an agent's bootstrap keys, or that would
// not load as an agent's configuration for some node.
func (a AgentSettings) Check() error {
check := func(where string, m map[string]any) error {
for _, k := range agentBootstrapKeys {
if _, ok := m[k]; ok {
return fmt.Errorf("%s.%s stays in each node's agent.yaml: an agent needs it before it can be told anything", where, k)
}
}
return nil
}
if err := check("agents.default", stringKeys(a.Default)); err != nil {
return err
}
nodes := make([]string, 0, len(a.Nodes))
for n := range a.Nodes {
nodes = append(nodes, n)
}
sort.Strings(nodes)
for _, n := range nodes {
if err := check("agents.nodes."+n, stringKeys(a.Nodes[n])); err != nil {
return err
}
}
// Each node's settings as its agent would load them, over nothing: what
// its own file adds cannot be known here, and what is wrong in the
// settings alone is wrong on every node.
for _, n := range append([]string{""}, nodes...) {
b, err := Marshal(a.For(n))
if err != nil {
return err
}
if _, err := LoadAgentWith("", b); err != nil {
where := "agents.default"
if n != "" {
where = "agents.nodes." + n
}
return fmt.Errorf("%s: %w", where, err)
}
}
return nil
}
// AgentBootstrapKeys are the keys agent.yaml keeps, for saying so.
func AgentBootstrapKeys() []string { return slices.Clone(agentBootstrapKeys) }
// stringKeys turns YAML's map[any]any, wherever it hides, into the
// map[string]any the rest of this package merges.
func stringKeys(m map[string]any) map[string]any {
out := make(map[string]any, len(m))
for k, v := range m {
out[k] = normalize(v)
}
return out
}
func normalize(v any) any {
switch t := v.(type) {
case map[string]any:
return stringKeys(t)
case map[any]any:
m := make(map[string]any, len(t))
for k, vv := range t {
m[fmt.Sprint(k)] = normalize(vv)
}
return m
case []any:
out := make([]any, len(t))
for i, vv := range t {
out[i] = normalize(vv)
}
return out
}
return v
}
+79
View File
@@ -0,0 +1,79 @@
package config
import (
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestANodesAgentSettingsAreTheDefaultWithItsOwnOver(t *testing.T) {
a := AgentSettings{
Default: map[string]any{"heartbeat_interval": "5s", "supervisor": map[string]any{"work_dir": "/srv/w", "x": 1}},
Nodes: map[string]map[string]any{
"cuda01": {"supervisor": map[string]any{"work_dir": "/fast/w"}, "labels": map[string]any{"gpu": "rtx6000"}},
},
}
got := a.For("cuda01")
sup := got["supervisor"].(map[string]any)
if sup["work_dir"] != "/fast/w" || sup["x"] != 1 || got["heartbeat_interval"] != "5s" || got["labels"] == nil {
t.Errorf("cuda01: %v", got)
}
if got := a.For("node01"); got["labels"] != nil || got["supervisor"].(map[string]any)["work_dir"] != "/srv/w" {
t.Errorf("node01: %v", got)
}
if got := (AgentSettings{}).For("x"); len(got) != 0 {
t.Errorf("none: %v", got)
}
}
func TestAgentSettingsKeepBootstrapKeysInTheFileAndMustLoad(t *testing.T) {
for _, a := range []AgentSettings{
{Default: map[string]any{"servers": []any{"x:7443"}}},
{Nodes: map[string]map[string]any{"n1": {"node_id": "other"}}},
{Nodes: map[string]map[string]any{"n1": {"tls": map[string]any{"ca": "/x"}}}},
} {
if err := a.Check(); err == nil || !strings.Contains(err.Error(), "stays in each node's agent.yaml") {
t.Errorf("%+v: %v", a, err)
}
}
// A fabric with no adapters would not load on the node it names.
bad := AgentSettings{Nodes: map[string]map[string]any{"n1": {"fabrics": []any{map[string]any{"name": "ib-a"}}}}}
if err := bad.Check(); err == nil || !strings.Contains(err.Error(), "agents.nodes.n1") || !strings.Contains(err.Error(), "no HCAs") {
t.Errorf("a fabric without HCAs: %v", err)
}
if err := (AgentSettings{Default: map[string]any{"heartbeat_interval": "3s"}}).Check(); err != nil {
t.Errorf("a plain setting: %v", err)
}
}
func TestTheClusterSettingsCarryAgentsAndRefuseBadOnes(t *testing.T) {
good, err := Parse([]byte("agents:\n default:\n heartbeat_interval: 3s\n nodes:\n cuda01:\n labels: {gpu: rtx6000}\n"))
if err != nil {
t.Fatal(err)
}
cfg, err := FromMap(good)
if err != nil {
t.Fatal(err)
}
if cfg.Agents.For("cuda01")["labels"] == nil {
t.Errorf("agents not read: %+v", cfg.Agents)
}
bad, _ := Parse([]byte("agents:\n default:\n mgmt_addr: 10.0.0.1\n"))
if _, err := FromMap(bad); err == nil {
t.Error("a bootstrap key in the cluster's settings was taken")
}
}
func TestTheSettingsAreLaidOverTheAgentsFile(t *testing.T) {
p := filepath.Join(t.TempDir(), "agent.yaml")
_ = os.WriteFile(p, []byte("node_id: n1\nserver: cp:7443\nheartbeat_interval: 9s\nlabels: {rack: r1}\n"), 0o600)
cfg, err := LoadAgentWith(p, []byte("heartbeat_interval: 2s\nlabels: {gpu: a100}\n"))
if err != nil {
t.Fatal(err)
}
if cfg.NodeID != "n1" || cfg.HeartbeatInterval != 2*time.Second || cfg.Labels["rack"] != "r1" || cfg.Labels["gpu"] != "a100" {
t.Errorf("loaded %+v", cfg)
}
}
+10
View File
@@ -158,6 +158,13 @@ type Server struct {
// Firewall turns llmgw's firewall on every machine on or off.
Firewall FirewallSettings `koanf:"firewall"`
// Agents are the nodes' agent settings: agent.yaml's keys but those an
// agent needs to start, for every node (default) and for one
// (nodes.<id>). Sent to the agents as they change; an agent whose
// settings changed starts again on them, its engines serving
// throughout.
Agents AgentSettings `koanf:"agents"`
// FirewallNetworks are networks, besides those each machine reaches the
// cluster on, that llmgw's internal ports answer: a monitoring host on
// another subnet, nodes behind a router. Written into every machine's
@@ -887,6 +894,9 @@ func load(b []byte) (Server, error) {
// Validate rejects configurations that would otherwise fail confusingly at
// request time.
func (s Server) Validate() error {
if err := s.Agents.Check(); err != nil {
return err
}
for i, o := range s.EngineOverrides {
if o.GPUClass == "" && len(o.Nodes) == 0 && len(o.Labels) == 0 {
return fmt.Errorf("engine_overrides[%d] picks no nodes: give it a gpu_class, nodes or labels", i)
@@ -0,0 +1,50 @@
package agentsvc
import (
"time"
llmgwv1 "gitea.mynodes.xyz/ilker/llmgw/api/gen/llmgw/v1"
)
// SetAgentSettings gives the service each node's agent settings from the
// cluster's (BACKLOG E3), as YAML, read at each use so a change is what is
// sent next.
func (s *Service) SetAgentSettings(f func(node string) string) {
s.mu.Lock()
s.agentSettings = f
s.mu.Unlock()
}
func (s *Service) agentSettingsMsg(node string) *llmgwv1.ControlMessage {
s.mu.RLock()
f := s.agentSettings
s.mu.RUnlock()
if f == nil {
return nil
}
return &llmgwv1.ControlMessage{Msg: &llmgwv1.ControlMessage_AgentSettings{
AgentSettings: &llmgwv1.AgentSettings{Yaml: f(node)}}}
}
// agentSettingsGap is the pause between telling one node and the next: an
// agent whose settings changed starts again, and a whole cluster's agents
// doing so in the same second would all re-register at once.
var agentSettingsGap = time.Second
// BroadcastAgentSettings tells every connected agent what the cluster's
// settings now say it runs on, a moment apart. A node not connected is told
// when it registers.
func (s *Service) BroadcastAgentSettings() {
for _, node := range s.ConnectedNodes() {
msg := s.agentSettingsMsg(node)
if msg == nil {
return
}
if err := s.send(node, msg); err != nil {
s.log.Warn("could not tell a node its agent settings; it is told when it next registers",
"node", node, "err", err)
continue
}
time.Sleep(agentSettingsGap)
}
}
@@ -0,0 +1,28 @@
package agentsvc
import (
"log/slog"
"testing"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
)
func TestEachNodeIsToldItsOwnAgentSettings(t *testing.T) {
s := New(inventory.NewStore(), nil, false, slog.New(slog.DiscardHandler))
if s.agentSettingsMsg("n1") != nil {
t.Error("told settings with nothing to read them from")
}
s.SetAgentSettings(func(node string) string {
if node == "cuda01" {
return "labels:\n gpu: rtx6000\n"
}
return ""
})
if got := s.agentSettingsMsg("cuda01").GetAgentSettings().GetYaml(); got != "labels:\n gpu: rtx6000\n" {
t.Errorf("cuda01 told %q", got)
}
// None is said too, so an agent whose settings were removed drops them.
if m := s.agentSettingsMsg("node01"); m == nil || m.GetAgentSettings().GetYaml() != "" {
t.Errorf("node01 told %v", m)
}
}
+10
View File
@@ -79,6 +79,9 @@ type Service struct {
controlPlanes func(context.Context) ([]string, error)
// firewall is what nodes are told of llmgw's firewall; see SetFirewall.
firewall func() *llmgwv1.Firewall
// agentSettings is what each node's agent is told to run on; see
// SetAgentSettings.
agentSettings func(node string) string
// logWaits are the log requests awaiting a reply, by request id.
logMu sync.Mutex
@@ -296,6 +299,13 @@ func (s *Service) Connect(
return err
}
}
// What the cluster's settings say this agent runs on. An agent already
// on them ignores it; one that is not starts again on them.
if msg := s.agentSettingsMsg(nodeID); msg != nil {
if err := stream.Send(msg); err != nil {
return err
}
}
// Recover replica state from what this node is already running, before any
// reconciliation pass can conclude the cluster is empty and place
+5
View File
@@ -19,6 +19,10 @@ var Reference = []Block{
{Key: "queue_capacity_total", Type: "int", Doc: "QueueCapacityTotal bounds everything queued across bands. Reached, a\nrequest is not refused while a lower band has any queued: the newest of\nthe lowest band is turned away instead, with a 503 and Retry-After.\nZero takes half of all the bands' capacities together.", Nested: ""},
{Key: "interval", Type: "time.Duration", Doc: "", Nested: ""},
}},
{Package: "config", Name: "AgentSettings", Title: "AgentSettings", Doc: "", Fields: []Field{
{Key: "default", Type: "map[string]any", Doc: "Default is agent.yaml's keys for every node.", Nested: ""},
{Key: "nodes", Type: "map[string]map[string]any", Doc: "Nodes are agent.yaml's keys for one node each, by node id, over\nDefault.", Nested: ""},
}},
{Package: "config", Name: "Auth", Title: "Auth", Doc: "", Fields: []Field{
{Key: "enabled", Type: "bool", Doc: "", Nested: ""},
{Key: "store", Type: "string", Doc: "Store is the SQLite file holding keys, tenancy and usage.", Nested: ""},
@@ -372,6 +376,7 @@ var Reference = []Block{
{Key: "mail", Type: "mail.Config", Doc: "Mail is where the gateway's few emails go out from: a link to set a\nforgotten password. Unset, none is sent and people ask an operator.", Nested: "mail.Config"},
{Key: "serving", Type: "ServingSettings", Doc: "Serving stops and starts the whole cluster's service (llmgw stop, llmgw\nstart).", Nested: "config.ServingSettings"},
{Key: "firewall", Type: "FirewallSettings", Doc: "Firewall turns llmgw's firewall on every machine on or off.", Nested: "config.FirewallSettings"},
{Key: "agents", Type: "AgentSettings", Doc: "Agents are the nodes' agent settings: agent.yaml's keys but those an\nagent needs to start, for every node (default) and for one\n(nodes.<id>). Sent to the agents as they change; an agent whose\nsettings changed starts again on them, its engines serving\nthroughout.", Nested: "config.AgentSettings"},
{Key: "firewall_networks", Type: "[]string", Doc: "FirewallNetworks are networks, besides those each machine reaches the\ncluster on, that llmgw's internal ports answer: a monitoring host on\nanother subnet, nodes behind a router. Written into every machine's\nfirewall by `server init`, `node add` and `server add`.", Nested: ""},
{Key: "backup_recipients", Type: "[]string", Doc: "BackupRecipients encrypt every archive `llmgw server backup` writes, to\nwhoever holds one of their keys: age recipients (age1…) or SSH public\nkeys (ssh-ed25519 …, ssh-rsa …), as `age` takes them. An archive holds\nthe certificate authority's key and every secret in the store, and is\ncopied off the machine; encrypted, the copy is safe wherever it lands.\nEmpty writes archives unencrypted, readable by root only.", Nested: ""},
{Key: "backup_copy_to", Type: "[]string", Doc: "BackupCopyTo is where each archive is copied once written, so it\nsurvives this machine: a directory (a mounted share, another disk) or\nssh://user@host[:port]/path, reached with BackupCopyKey and checked\nagainst /etc/llmgw/backup_known_hosts, which learns a host the first\ntime. The same number of archives is kept there as here. Refused\nwithout backup_recipients: an archive leaves the machine encrypted.", Nested: ""},
+1
View File
@@ -32,6 +32,7 @@ var Sources = []string{
"internal/config/oidc.go",
"internal/config/firewall.go",
"internal/config/serving.go",
"internal/config/agents.go",
"internal/mail/mail.go",
"internal/notify/notify.go",
"internal/secrets/secrets.go",
+3
View File
@@ -247,6 +247,9 @@ func (r *Router) Handler() http.Handler {
if r.store != nil && r.guardLibrary != nil {
pages.RegisterAdminGuard(mux, r.store, r.guardLibrary)
}
if r.store != nil && r.settingsCheck != nil {
pages.RegisterAdminSettings(mux, r.store, r.settingsCheck, r.settingsLocal)
}
pages.Register(mux)
// The docs portal, with the menu of whoever is signed in, and only to
// them when the site asked for that.
+11
View File
@@ -177,6 +177,10 @@ type Router struct {
// collections for kbDefaultTTL.
kbDefMu sync.Mutex
kbDefCache map[string]kbDefaultEntry
// settingsCheck and settingsLocal mount the portal's settings page.
settingsCheck func([]byte) error
settingsLocal []string
}
// SetClusterView gives the router the data the /admin page renders.
@@ -200,6 +204,13 @@ func (r *Router) SetGuard(g guard.Guard) { r.guard = g }
// SetGuardLibrary is the policies the guard was built from, for the pages.
func (r *Router) SetGuardLibrary(c func() guard.Config) { r.guardLibrary = c }
// SetSettingsCheck enables the portal's settings page: check says whether a
// document would load as the cluster's settings beside this machine's own
// server.yaml, and local are the keys that file keeps.
func (r *Router) SetSettingsCheck(check func([]byte) error, local []string) {
r.settingsCheck, r.settingsLocal = check, local
}
// SetStopped refuses, or again serves, every request for a model.
func (r *Router) SetStopped(stopped bool) { r.stopped.Store(stopped) }
+3 -1
View File
@@ -90,7 +90,9 @@ type AccountPages struct {
audit audit.Sink
// hasGuard is set when /admin/guard is registered, and guardLibrary
// with it: the policies in force, whose judges the mirror page serves.
hasGuard bool
hasGuard bool
// hasSettings is set when /admin/settings is registered.
hasSettings bool
guardLibrary func() guard.Config
// guardFor names the guardrail policies in force for someone, and
// whether they keep nothing; nil when there are none.
+174
View File
@@ -0,0 +1,174 @@
package web
import (
"context"
"errors"
"net/http"
"strconv"
"strings"
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
)
// The cluster's settings, edited in the browser (BACKLOG E4): the document
// `llmgw ctl config edit` opens, for whoever has no terminal on a control
// plane. The same rules hold here as there — a version is checked as a
// control plane would load it before it is kept, it is made from the newest
// version or refused, and every change is in force on every control plane
// within seconds and in the audit log.
// SettingsAdmin is where the cluster's settings are kept.
type SettingsAdmin interface {
ClusterSettings(ctx context.Context) (store.SettingsVersion, error)
ClusterSettingsAt(ctx context.Context, version int64) (store.SettingsVersion, error)
ClusterSettingsHistory(ctx context.Context, limit int) ([]store.SettingsVersion, error)
PutClusterSettings(ctx context.Context, base int64, body, by, note string) (int64, error)
}
type settingsPageData struct {
Nav Nav
Version string
CSRF string
// Current is the version in force; Shown the one in the editor, which
// is an earlier one when ?version= asked for it.
Current, Shown store.SettingsVersion
Earlier bool
// Body is what the editor holds: the version shown, or what was sent
// and refused, so a refusal does not lose the edit.
Body string
Note string
History []settingsRow
LocalKeys []string
Notice string
Error string
}
type settingsRow struct {
Version int64
At, By string
Note string
InForce bool
Selected bool
}
// RegisterAdminSettings mounts /admin/settings, for administrators only:
// the settings hold webhook addresses and the like, so even reading them is
// not every staff role's.
func (p *AccountPages) RegisterAdminSettings(mux *http.ServeMux, s SettingsAdmin, check func([]byte) error, localKeys []string) {
p.hasSettings = true
page := func(w http.ResponseWriter, r *http.Request, admin store.User, d settingsPageData) {
ctx, l := r.Context(), langOf(r)
d.Nav, d.Version, d.CSRF, d.LocalKeys = p.navFor(r, admin, "settings"), p.version, csrfFor(r), localKeys
cur, err := s.ClusterSettings(ctx)
if err != nil {
d.Error = l.Err(err)
}
d.Current = cur
if d.Shown.Version == 0 {
d.Shown = cur
}
d.Earlier = d.Shown.Version != cur.Version
if d.Body == "" {
d.Body = d.Shown.Body
}
if hist, err := s.ClusterSettingsHistory(ctx, 30); err == nil {
for _, v := range hist {
d.History = append(d.History, settingsRow{Version: v.Version, At: l.Date(v.At.UTC()) + " " + v.At.UTC().Format("15:04"),
By: v.By, Note: v.Note, InForce: v.Version == cur.Version, Selected: v.Version == d.Shown.Version})
}
}
noStore(w)
render(w, r, "admin_settings", d)
}
adminsOnly := func(next func(w http.ResponseWriter, r *http.Request, admin store.User)) http.HandlerFunc {
return p.adminOnly(func(w http.ResponseWriter, r *http.Request, admin store.User) {
if !admin.IsAdmin {
http.NotFound(w, r)
return
}
if r.Method == http.MethodPost {
sess, _, _ := p.session(r)
if !p.accounts.CheckCSRF(sess, r.FormValue("csrf")) {
http.Error(w, "bad request", http.StatusBadRequest)
return
}
}
next(w, r, admin)
})
}
mux.HandleFunc("GET /admin/settings", adminsOnly(func(w http.ResponseWriter, r *http.Request, admin store.User) {
var d settingsPageData
if q := r.URL.Query().Get("version"); q != "" {
n, err := strconv.ParseInt(q, 10, 64)
if v, verr := s.ClusterSettingsAt(r.Context(), n); err == nil && verr == nil && v.Version != 0 {
d.Shown = v
} else {
d.Error = langOf(r).T("settings.err.no_version")
}
}
page(w, r, admin, d)
}))
mux.HandleFunc("POST /admin/settings", adminsOnly(func(w http.ResponseWriter, r *http.Request, admin store.User) {
l := langOf(r)
body := strings.ReplaceAll(r.FormValue("body"), "\r\n", "\n")
note := strings.TrimSpace(r.FormValue("note"))
base, _ := strconv.ParseInt(r.FormValue("base"), 10, 64)
// The audit line records the form; the document can hold webhook
// secrets, so it records the document's size, not the document.
r.PostForm.Set("body", "("+strconv.Itoa(len(body))+" bytes)")
refuse := func(msg string) {
noteAudit(r, "cluster settings", "", msg)
page(w, r, admin, settingsPageData{Body: body, Note: note, Error: msg})
}
if cur, err := s.ClusterSettings(r.Context()); err == nil && cur.Body == body {
refuse(l.T("settings.err.unchanged"))
return
}
if err := check([]byte(body)); err != nil {
refuse(l.T("settings.err.invalid") + " " + err.Error())
return
}
n, err := s.PutClusterSettings(r.Context(), base, body, admin.Email, note)
switch {
case errors.Is(err, store.ErrSettingsChanged):
refuse(l.T("settings.err.changed"))
return
case err != nil:
refuse(l.Err(err))
return
}
noteAudit(r, "cluster settings", "version "+strconv.FormatInt(n, 10), "")
page(w, r, admin, settingsPageData{Notice: l.T("settings.saved", n)})
}))
mux.HandleFunc("POST /admin/settings/rollback", adminsOnly(func(w http.ResponseWriter, r *http.Request, admin store.User) {
l := langOf(r)
n, _ := strconv.ParseInt(r.FormValue("version"), 10, 64)
old, err := s.ClusterSettingsAt(r.Context(), n)
if err != nil || old.Version == 0 {
page(w, r, admin, settingsPageData{Error: l.T("settings.err.no_version")})
return
}
cur, err := s.ClusterSettings(r.Context())
if err != nil {
page(w, r, admin, settingsPageData{Error: l.Err(err)})
return
}
if err := check([]byte(old.Body)); err != nil {
// A version kept before a change to server.yaml may no longer
// load beside it; refused here rather than breaking the cluster.
noteAudit(r, "cluster settings", "", err.Error())
page(w, r, admin, settingsPageData{Error: l.T("settings.err.invalid") + " " + err.Error()})
return
}
v, err := s.PutClusterSettings(r.Context(), cur.Version, old.Body, admin.Email, "back to version "+strconv.FormatInt(n, 10))
if err != nil {
page(w, r, admin, settingsPageData{Error: l.Err(err)})
return
}
noteAudit(r, "cluster settings", "version "+strconv.FormatInt(v, 10)+" from "+strconv.FormatInt(n, 10), "")
page(w, r, admin, settingsPageData{Notice: l.T("settings.rolled_back", n, v)})
}))
}
+143
View File
@@ -0,0 +1,143 @@
package web
import (
"context"
"errors"
"io"
"log/slog"
"net/http"
"net/url"
"strings"
"testing"
"time"
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
)
type fakeSettings struct{ versions []store.SettingsVersion }
func (f *fakeSettings) ClusterSettings(context.Context) (store.SettingsVersion, error) {
if len(f.versions) == 0 {
return store.SettingsVersion{}, nil
}
return f.versions[len(f.versions)-1], nil
}
func (f *fakeSettings) ClusterSettingsAt(_ context.Context, v int64) (store.SettingsVersion, error) {
for _, s := range f.versions {
if s.Version == v {
return s, nil
}
}
return store.SettingsVersion{}, nil
}
func (f *fakeSettings) ClusterSettingsHistory(context.Context, int) ([]store.SettingsVersion, error) {
out := make([]store.SettingsVersion, len(f.versions))
for i, v := range f.versions {
out[len(f.versions)-1-i] = v
}
return out, nil
}
func (f *fakeSettings) PutClusterSettings(_ context.Context, base int64, body, by, note string) (int64, error) {
cur, _ := f.ClusterSettings(context.Background())
if base != cur.Version {
return 0, store.ErrSettingsChanged
}
n := cur.Version + 1
f.versions = append(f.versions, store.SettingsVersion{Version: n, Body: body, By: by, Note: note, At: time.Now()})
return n, nil
}
type auditSink struct{ events []store.AuditEvent }
func (a *auditSink) WriteAudit(_ context.Context, e store.AuditEvent) error {
a.events = append(a.events, e)
return nil
}
func clusterSettingsPage(t *testing.T, user store.User) (*http.ServeMux, *fakeSettings, *auditSink) {
t.Helper()
s := &fakeSettings{versions: []store.SettingsVersion{{Version: 1, Body: "guard:\n enabled: true\n", By: "init", At: time.Now()}}}
p := NewAccountPages(fakeAccounts{user: user}, nil, nil, "v0", slog.New(slog.NewTextHandler(io.Discard, nil)))
sink := &auditSink{}
p.SetAudit(sink)
mux := http.NewServeMux()
p.RegisterAdminSettings(mux, s, func(b []byte) error {
if strings.Contains(string(b), "bogus") {
return errors.New("unknown key bogus")
}
return nil
}, []string{"listen", "store"})
return mux, s, sink
}
func TestTheSettingsPageIsForAdministratorsOnly(t *testing.T) {
for _, u := range []store.User{
{ID: "u1", Email: "v@example.com", Role: store.RoleViewer},
{ID: "u2", Email: "o@example.com", Role: store.RoleOperator},
{ID: "u3", Email: "b@example.com", Role: store.RoleBilling},
} {
mux, _, _ := clusterSettingsPage(t, u)
if w := mirrorGet(t, mux, "/admin/settings"); w.Code != http.StatusNotFound {
t.Errorf("%s saw the settings: %d", u.Role, w.Code)
}
}
mux, _, _ := clusterSettingsPage(t, store.User{ID: "a", Email: "a@example.com", IsAdmin: true})
body := mirrorGet(t, mux, "/admin/settings").Body.String()
for _, want := range []string{"guard:\n enabled: true", "Version 1, in force", "<code>listen</code>", `href="/admin/settings" aria-current="page"`} {
if !strings.Contains(body, want) {
t.Errorf("no %q in:\n%s", want, body)
}
}
}
func TestSettingsAreCheckedAppliedAndRolledBack(t *testing.T) {
mux, s, sink := clusterSettingsPage(t, store.User{ID: "a", Email: "a@example.com", IsAdmin: true})
post := func(form url.Values) string {
form.Set("csrf", "tok")
return mirrorPost(t, mux, "/admin/settings", form).Body.String()
}
// Refused: does not load. The edit stays in the editor; nothing kept.
body := post(url.Values{"base": {"1"}, "body": {"bogus: 1\n"}})
if !strings.Contains(body, "unknown key bogus") || !strings.Contains(body, "bogus: 1") || len(s.versions) != 1 {
t.Errorf("an invalid document: %d versions\n%s", len(s.versions), body)
}
// Applied, with its note and who.
body = post(url.Values{"base": {"1"}, "body": {"guard:\r\n enabled: false\r\n"}, "note": {"maintenance"}})
if !strings.Contains(body, "Version 2 is in force") || len(s.versions) != 2 ||
s.versions[1].Body != "guard:\n enabled: false\n" || s.versions[1].By != "a@example.com" || s.versions[1].Note != "maintenance" {
t.Errorf("applying: %+v\n%s", s.versions, body)
}
// Made from a version no longer the newest: refused, not overwritten.
body = post(url.Values{"base": {"1"}, "body": {"guard:\n enabled: true\n x: 1\n"}})
if !strings.Contains(body, "someone changed the settings") || len(s.versions) != 2 {
t.Errorf("a stale edit: %d versions\n%s", len(s.versions), body)
}
// An earlier version is shown, and put back as a new one.
if body := mirrorGet(t, mux, "/admin/settings?version=1").Body.String(); !strings.Contains(body, "Put version 1 back in force") {
t.Errorf("viewing version 1:\n%s", body)
}
w := mirrorPost(t, mux, "/admin/settings/rollback", url.Values{"csrf": {"tok"}, "version": {"1"}})
if len(s.versions) != 3 || s.versions[2].Body != s.versions[0].Body || !strings.Contains(w.Body.String(), "as version 3") {
t.Errorf("rollback: %+v\n%s", s.versions, w.Body.String())
}
// Without the CSRF token, nothing.
if w := mirrorPost(t, mux, "/admin/settings", url.Values{"base": {"3"}, "body": {"a: 1\n"}}); w.Code != http.StatusBadRequest || len(s.versions) != 3 {
t.Errorf("no token: %d", w.Code)
}
// Audited, without the document itself: it can hold secrets.
var saw bool
for _, e := range sink.events {
if strings.Contains(e.Detail, "enabled") {
t.Errorf("the document is in the audit log: %+v", e)
}
if e.Action == "POST /admin/settings" && strings.Contains(e.Detail, "version 2") {
saw = true
}
}
if !saw {
t.Errorf("no audit line for version 2: %+v", sink.events)
}
}
+2 -1
View File
@@ -10,5 +10,6 @@
"nav.chat": "Chat",
"nav.account": "Account",
"nav.sign_out": "Sign out",
"nav.docs": "Docs"
"nav.docs": "Docs",
"nav.settings": "Settings"
}
+28
View File
@@ -0,0 +1,28 @@
{
"settings.title": "Settings",
"settings.sub": "The cluster's settings: everything that is not one machine's own, the same on every control plane, in force within seconds of a change. <code>llmgw ctl config edit</code> edits the same document.",
"settings.current": "Version %d, in force",
"settings.earlier": "Version %d",
"settings.earlier_sub": "An earlier version; version %d is in force. Rolling back keeps this one as a new version, so nothing is lost.",
"settings.rollback": "Put version %d back in force",
"settings.back_to_current": "back to the version in force",
"settings.local_keys": "Not here, because each machine's server.yaml keeps them:",
"settings.note": "Why",
"settings.note_hint": "kept with the version, for whoever reads the history",
"settings.save": "Check and apply",
"settings.save_hint": "Checked as a control plane would load it before it is kept; a change made since this page was opened is not overwritten.",
"settings.saved": "Version %d is in force; every control plane applies it within seconds.",
"settings.rolled_back": "Version %d's settings are back in force, as version %d.",
"settings.history": "History",
"settings.col.version": "Version",
"settings.col.at": "When",
"settings.col.by": "By",
"settings.col.note": "Why",
"settings.in_force": "in force",
"settings.view": "view",
"settings.no_history": "No versions yet.",
"settings.err.unchanged": "Nothing was changed.",
"settings.err.invalid": "Not applied: a control plane could not load it.",
"settings.err.changed": "Not applied: someone changed the settings since this page was opened. Your edit is below; compare it with the version now in force and apply it again.",
"settings.err.no_version": "There is no such version."
}
+2 -1
View File
@@ -10,5 +10,6 @@
"nav.chat": "Sohbet",
"nav.account": "Hesap",
"nav.sign_out": "Çıkış yap",
"nav.docs": "Belgeler"
"nav.docs": "Belgeler",
"nav.settings": "Ayarlar"
}
+28
View File
@@ -0,0 +1,28 @@
{
"settings.title": "Ayarlar",
"settings.sub": "Kümenin ayarları: tek bir makineye ait olmayan her şey; her kontrol düzleminde aynı, değiştikten saniyeler sonra yürürlükte. <code>llmgw ctl config edit</code> aynı belgeyi düzenler.",
"settings.current": "Sürüm %d, yürürlükte",
"settings.earlier": "Sürüm %d",
"settings.earlier_sub": "Önceki bir sürüm; yürürlükte olan sürüm %d. Geri almak bunu yeni bir sürüm olarak kaydeder, hiçbir şey kaybolmaz.",
"settings.rollback": "%d numaralı sürümü yeniden yürürlüğe koy",
"settings.back_to_current": "yürürlükteki sürüme dön",
"settings.local_keys": "Burada değil, çünkü her makinenin kendi server.yaml dosyası tutar:",
"settings.note": "Neden",
"settings.note_hint": "sürümle birlikte saklanır, geçmişi okuyacak kişi için",
"settings.save": "Denetle ve uygula",
"settings.save_hint": "Saklanmadan önce bir kontrol düzleminin yükleyeceği gibi denetlenir; bu sayfa açıldıktan sonra yapılmış bir değişikliğin üzerine yazılmaz.",
"settings.saved": "%d numaralı sürüm yürürlükte; her kontrol düzlemi saniyeler içinde uygular.",
"settings.rolled_back": "%d numaralı sürümün ayarları yeniden yürürlükte; yeni sürüm numarası %d.",
"settings.history": "Geçmiş",
"settings.col.version": "Sürüm",
"settings.col.at": "Ne zaman",
"settings.col.by": "Kim",
"settings.col.note": "Neden",
"settings.in_force": "yürürlükte",
"settings.view": "görüntüle",
"settings.no_history": "Henüz sürüm yok.",
"settings.err.unchanged": "Hiçbir şey değiştirilmedi.",
"settings.err.invalid": "Uygulanmadı: bir kontrol düzlemi bunu yükleyemezdi.",
"settings.err.changed": "Uygulanmadı: bu sayfa açıldıktan sonra biri ayarları değiştirdi. Düzenlemeniz aşağıda; yürürlükteki sürümle karşılaştırıp yeniden uygulayın.",
"settings.err.no_version": "Böyle bir sürüm yok."
}
+11 -7
View File
@@ -26,6 +26,9 @@ type Nav struct {
Mirror, Quotas bool
// Guard is whether the guardrails page was mounted.
Guard bool
// Settings offers the cluster's settings: mounted, and to an
// administrator only.
Settings bool
// CSRF signs the sign-out form.
CSRF string
// Org offers the organization page: to an administrator, and to anyone
@@ -38,13 +41,14 @@ type navKey struct{}
// navFor builds the menu for a signed-in person.
func (p *AccountPages) navFor(r *http.Request, user store.User, current string) Nav {
n := Nav{
Current: current,
Admin: user.Staff(),
Role: user.StaffRole(),
Mirror: p.hasMirror,
Quotas: p.hasQuotas,
Guard: p.hasGuard,
CSRF: csrfFor(r),
Current: current,
Admin: user.Staff(),
Role: user.StaffRole(),
Mirror: p.hasMirror,
Quotas: p.hasQuotas,
Guard: p.hasGuard,
Settings: p.hasSettings && user.IsAdmin,
CSRF: csrfFor(r),
}
// Only when there is an organization to show: a link that leads to "there
// are none" on every page is noise.
+10
View File
@@ -905,3 +905,13 @@ a.sel-link.small { font-size: .78rem; color: var(--muted); margin-left: .4rem; }
/* The language switcher at the foot of every page. */
.langs a { color: var(--muted); }
.langs b { font-weight: 600; }
/* The cluster's settings, edited as the YAML document they are. */
.settings-form textarea, pre.settings-body {
width: 100%; box-sizing: border-box; font-family: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
font-size: .85rem; line-height: 1.4; padding: .6rem .7rem; border: 1px solid var(--line);
border-radius: 6px; background: var(--card); color: var(--fg); tab-size: 2; overflow-x: auto;
}
.settings-form label { display: block; margin: .8rem 0; }
.settings-form input[type=text] { width: 100%; box-sizing: border-box; }
.sr-only { position: absolute; width: 1px; height: 1px; overflow: hidden; clip: rect(0 0 0 0); white-space: nowrap; }
@@ -0,0 +1,50 @@
{{define "admin_settings"}}<!doctype html>
<html lang="{{lang}}"><head>
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
<title>{{t "settings.title"}} · llmgw</title><link rel="stylesheet" href="{{stylesheet}}">
</head><body><div class="wrap">
<header>
<h1>{{t "settings.title"}}</h1>
<p class="sub">{{th "settings.sub"}}</p>
{{template "nav" .Nav}}
</header>
{{if .Error}}<p class="error" role="alert">{{.Error}}</p>{{end}}
{{if .Notice}}<p class="ok" role="status">{{.Notice}}</p>{{end}}
{{if .Earlier}}
<h2>{{t "settings.earlier" .Shown.Version}}</h2>
<p class="sub">{{t "settings.earlier_sub" .Current.Version}}</p>
<pre class="settings-body">{{.Body}}</pre>
<form method="post" action="/admin/settings/rollback">
<input type="hidden" name="csrf" value="{{.CSRF}}">
<input type="hidden" name="version" value="{{.Shown.Version}}">
<button type="submit">{{t "settings.rollback" .Shown.Version}}</button>
<a href="/admin/settings">{{t "settings.back_to_current"}}</a>
</form>
{{else}}
<h2>{{t "settings.current" .Current.Version}}</h2>
{{if .LocalKeys}}<p class="sub">{{t "settings.local_keys"}} {{range $i, $k := .LocalKeys}}{{if $i}}, {{end}}<code>{{$k}}</code>{{end}}.</p>{{end}}
<form method="post" action="/admin/settings" class="settings-form">
<input type="hidden" name="csrf" value="{{.CSRF}}">
<input type="hidden" name="base" value="{{.Current.Version}}">
<label for="settings-body" class="sr-only">{{t "settings.title"}}</label>
<textarea id="settings-body" name="body" rows="30" spellcheck="false" class="mono">{{.Body}}</textarea>
<label>{{t "settings.note"}} <span class="opt">{{t "settings.note_hint"}}</span>
<input type="text" name="note" value="{{.Note}}" maxlength="200"></label>
<button type="submit">{{t "settings.save"}}</button>
<p class="sub">{{t "settings.save_hint"}}</p>
</form>
{{end}}
<h2>{{t "settings.history"}}</h2>
{{if .History}}
<table class="rows">
<tr><th class="num">{{t "settings.col.version"}}</th><th>{{t "settings.col.at"}}</th><th>{{t "settings.col.by"}}</th><th>{{t "settings.col.note"}}</th><th></th></tr>
{{range .History}}<tr{{if .Selected}} aria-current="true"{{end}}>
<td class="num">{{.Version}}</td><td>{{.At}}</td><td>{{.By}}</td><td>{{.Note}}</td>
<td>{{if .InForce}}<strong>{{t "settings.in_force"}}</strong>{{else}}<a href="/admin/settings?version={{.Version}}">{{t "settings.view"}}</a>{{end}}</td>
</tr>{{end}}
</table>
{{else}}<p class="sub">{{t "settings.no_history"}}</p>{{end}}
<footer>llmgw {{.Version}}</footer>
</div></body></html>{{end}}
+3
View File
@@ -20,6 +20,9 @@
{{- if .Guard}}
<a href="/admin/guard"{{if eq .Current "guard"}} aria-current="page"{{end}}>{{t "nav.guard"}}</a>
{{- end}}
{{- if .Settings}}
<a href="/admin/settings"{{if eq .Current "settings"}} aria-current="page"{{end}}>{{t "nav.settings"}}</a>
{{- end}}
{{- /* An organization is a panel of the people page for an administrator;
/org is the managers' view, and leads an administrator there. */}}
<span class="menu-gap" aria-hidden="true"></span>