Console: rotate a key, cap key lifetimes, test the webhooks; README, onboarding and console docs caught up

The README's status stopped at v0.9 and listed gaps since closed; onboarding
still said there was no SSO and no roles.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 17:28:24 +02:00
1 parent 1b71c38f89
commit 4979e08d64
7 files changed
+111 -22

No files matched your search

+42 -6
View File
@@ -33,6 +33,8 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md).
| v0.7.0 | — | Conversations, agent memory, disk management | released |
| v0.8.0 | — | PostgreSQL, leader election, agent failover | released |
| v0.9.0 | — | Managing people and the tenancy tree | released |
| v0.10.0 | — | InfiniBand fabrics and NVLink, doctor, node add, engines installed by the nodes | released |
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
| v1.0.0 | 5 | Tenancy, prefix affinity, benchmarking | planned |
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
@@ -40,12 +42,34 @@ control plane places replicas — including ones larger than a single machine
models load when someone asks for them and stop when nobody does, and keys,
budgets and rate limits decide who may use any of it.
Before v1.0.0: tracing, reference documentation, and filtering on `ctl nodes`
and `ctl replicas` for fleets too large to read a whole table of. Two items
that were on this list are not gaps: a model falls through to its fallbacks
when there is no capacity as well as when a budget is spent, and routing
profiles are chosen by the key rather than by a request header on purpose —
a header would let any client promote itself into the top priority band.
Since v0.11.0, not yet released (see [CHANGELOG.md](CHANGELOG.md),
"Unreleased"):
- **Security hardening** — signed releases only, a token on the admin API,
engines not run as root, TLS for the agent port, etcd and Patroni, a
firewall for llmgw's internal ports, encrypted backups, bounded
connections, sign-in throttling, a Content-Security-Policy.
- **Guardrails** — policies that check requests and answers (PII, secrets,
prompt injection, deny patterns, a judge model such as Llama Guard), in
observe or enforce mode, attached to any part of the tenancy tree:
[docs/guardrails.md](docs/guardrails.md).
- **Administration** — an audit log of every change (`ctl audit`), staff
roles (viewer, operator, billing, admin), single sign-on through OpenID
Connect and provisioning through SCIM 2.0, key lifetime limits and
rotation with overlap (`keys ttl`, `keys rotate`), usage export as CSV
(`ctl usage export`).
- **Notifications** — webhooks, signed or Slack-shaped, for a model parked,
a node lost, a budget spent and new weights published (`notify.webhooks`,
`ctl notify test`), and Alertmanager receiver templates.
Every one of these is in the console as well as on the command line:
[docs/console.md](docs/console.md).
What is still ahead is in [BACKLOG.md](BACKLOG.md). Two things that look like
gaps are not: a model falls through to its fallbacks when there is no
capacity as well as when a budget is spent, and routing profiles are chosen
by the key rather than by a request header on purpose — a header would let
any client promote itself into the top priority band.
One thing is worth saying plainly:
multi-node replicas are proven on a simulated cluster and on real hardware
@@ -70,6 +94,12 @@ them. On such a machine, run `llmgw doctor --json` first and keep the report;
how many of each, and where the weights should live
- [docs/data.md](docs/data.md) — what llmgw stores, what it does not, and
what leaves the machine; written for a security review
- [docs/console.md](docs/console.md) — the console that bare `llmgw` opens:
every view, and the command each of its actions runs
- [docs/guardrails.md](docs/guardrails.md) — checking requests and answers:
policies, detectors, a judge model, and where each applies
- [docs/onboarding.md](docs/onboarding.md) — accounts, keys, limits, staff
roles and single sign-on
- [docs/reference.md](docs/reference.md) — every configuration key, generated
from the structs and checked by a test, so it cannot describe a setting that
does not exist or miss one that does
@@ -79,6 +109,8 @@ them. On such a machine, run `llmgw doctor --json` first and keep the report;
port, with streaming and the forwarded headers right
- [deploy/ha/](deploy/ha/) — running two control planes, and the one health
check that has to be right
- [deploy/monitoring/](deploy/monitoring/) — Prometheus rules, Alertmanager
with receiver templates, and Grafana dashboards
- [docs/node-onboarding.md](docs/node-onboarding.md),
[docs/onboarding.md](docs/onboarding.md) — adding machines, and adding people
@@ -100,8 +132,12 @@ llmgw ctl explain-placement llama # why a model has not been placed
llmgw ctl drain gpu-a-01 # move work off a node, one replica at a time
llmgw ctl cordon gpu-a-01 # stop new placement, leave what runs
sudo llmgw doctor # on a node: what its GPUs, NVLink and fabric need
llmgw ctl audit --since 24h # who changed what
llmgw ctl usage export --from 2026-09-01 --to 2026-10-01 --by project,model
```
Bare `llmgw` opens the console, which does all of this from one screen.
`ctl` talks to the admin port (`:9090` by default), not the client-facing one.
Add `--json` to any command for machine-readable output.
+9 -4
View File
@@ -67,7 +67,8 @@ and power, and each model's requests a second. The history comes from the
Prometheus named by `monitoring.prometheus` in server.yaml; without one the
charts draw what this console has seen since it opened, and say so. Network
and disk come from each machine's node exporter, which `llmgw node add`
installs. A node's and a model's own pages have their charts too, over the
installs. **N** sends a test notification to each webhook in server.yaml's
`notify` section and says how each answered (`llmgw ctl notify test`). A node's and a model's own pages have their charts too, over the
same range.
**Nodes** lists every node with its state, load, GPUs, replicas, disk and
@@ -158,7 +159,9 @@ project, their limits, a new password (their sessions close), delete.
**API keys** — every key, its project, owner, state and last use. **A**
issues one; the key is shown once, in the result, and never again. On a row:
its limits, or revoke it.
its limits, **Rotate…** — a new key with the same project, owner and limits,
the old one working on for the overlap you give (`llmgw keys rotate`) — or
revoke it.
**Accounts** rows also have **Staff role…**:
- `viewer` looks at every page under /admin and changes nothing;
@@ -182,8 +185,10 @@ with anything secret taken out. `llmgw ctl audit` is the same, with filters.
projects, with members, keys and limits. **A** adds an organization. On a
row: add a team or project inside it, rename, move, limits for the level and
for each of its members, its capacity quota, and for a project how long
conversations and agent notes are kept. **Remove** takes away only something
empty.
conversations and agent notes are kept. **Longest key lifetime…** caps how
long a key made there may live, and is the expiry of one made without any
(`llmgw keys ttl`; `auth.max_key_ttl` in server.yaml caps every key).
**Remove** takes away only something empty.
Limits are requests and tokens a minute, a budget per day or month, and the
models allowed. Each level only narrows what is above it.
+23 -11
View File
@@ -325,8 +325,15 @@ llmgw keys list --store … --project default
minute. It is the column that tells you whether a key is still in use, which is
what makes rotation safe:
**To rotate**, issue the new key, let both work while the user switches, watch
`LAST USED` on the old one stop moving, then revoke it.
**To rotate**, `llmgw keys rotate <key-id> --overlap 48h` issues a new key with
the same project, owner and limits and lets the old one work for the overlap,
then stop. Watch `LAST USED` on the old one stop moving before it runs out.
**To cap how long keys live**, `llmgw keys ttl project:Acme/ml/train 720h` (or
`team:`, `org:`; `off` removes it), and `auth.max_key_ttl` in server.yaml for
every key. A key is made within the shortest limit that applies, and one asked
for without an expiry gets that limit. `keys list --all` shows each key's kind
— service (a project's), member, personal — and when it expires.
```
llmgw keys revoke --store … key-mpmip77twpt7qaxc
@@ -433,21 +440,26 @@ Three things are deliberately not there:
| `llmgw keys create` | Issuing a credential on another person's behalf |
Each of those turns a stolen admin cookie into more than it was, and each is
rare enough that a shell is no hardship. Cordon, drain and everything else that
touches the cluster are also CLI-only for the same reason: the dashboard shows
the cluster, it does not steer it.
rare enough that a shell is no hardship. Cordon and drain are CLI-only for the
same reason; what the pages do change on the cluster — serving models through
the mirror, a node's engines — needs the `operator` or `admin` staff role.
Every administrative action is logged with who did it, including the ones that
were refused.
Every administrative change is recorded in the audit log — who, when, through
what, the fields given with secrets taken out, and how it ended, refusals
included: `llmgw ctl audit`, or **Audit** in the console.
## What this does not do
- **No email.** No approval notice, no self-service password reset, no address
verification. The loop is manual on purpose at this size; if it stops being
the right size, that is the thing to automate.
- **No SSO.** Accounts are local, with local passwords.
- **No roles.** There is one flag, `is_admin`, and it grants looking at the
dashboard. Everything else an administrator does is the CLI, and having a
shell here is what makes someone one.
- **Single sign-on is optional.** With `auth.oidc` people sign in through
your identity provider, and SCIM 2.0 (`auth.scim`) creates, updates and
suspends accounts from it; see [install.md](install.md#single-sign-on).
Without them accounts are local, with local passwords.
- **Staff roles are coarse.** `viewer`, `operator`, `billing` and `admin`
(`llmgw users role`) decide what someone may see and do under /admin; an
organization's manager can view it but not change it. Every change is in
the audit log (`llmgw ctl audit`).
- **Rate limits are per gateway instance.** With two instances each counts its own,
so the effective limit doubles. v1 is single-instance by design.
+6
View File
@@ -648,6 +648,12 @@ func TestGuardrailsSayWhereTheyApplyFrom(t *testing.T) {
if items := m.orgMenu(m.people.orgs[0]).items; !slices.ContainsFunc(items, func(i menuItem) bool { return i.label == "Guardrails…" }) {
t.Error("no Guardrails… in the menu")
}
if items := m.orgMenu(m.people.orgs[0]).items; !slices.ContainsFunc(items, func(i menuItem) bool { return i.label == "Longest key lifetime…" }) {
t.Error("no Longest key lifetime… in the menu")
}
if items := m.keyMenu(store.KeyRow{}).items; !slices.ContainsFunc(items, func(i menuItem) bool { return i.label == "Rotate…" }) {
t.Error("no Rotate… on a key")
}
}
func TestTheAuditViewListsChangesAndOpensOne(t *testing.T) {
+4
View File
@@ -536,6 +536,10 @@ func (m consoleModel) key(k tea.KeyMsg) (tea.Model, tea.Cmd) {
case cOrgs:
m.createOrgForm()
}
case "n":
if m.menu == cMonitor {
return m, m.runCLI("Test the webhooks", "ctl", "--server="+m.client.base, "notify", "test")
}
case "e":
if m.menu == cUsage {
m.usageExportForm()
+1 -1
View File
@@ -258,7 +258,7 @@ func side(w int, a, b string) string {
}
func (m consoleModel) monitorPanel(pw int) (string, string, string) {
keys := "→ or Enter open · ↑↓ choose · Q leave"
keys := "→ or Enter open · ↑↓ choose · N test the webhooks · Q leave"
if m.inPanel {
keys = "1–4 or R range · PgUp/PgDn scroll · ← back · Q leave"
}
+26
View File
@@ -311,6 +311,18 @@ func (m consoleModel) keyMenu(k store.KeyRow) *menuDialog {
name := k.Name + " (" + k.Prefix + ")"
return &menuDialog{title: name, items: []menuItem{
{"Its limits…", func(m *consoleModel) tea.Cmd { m.limitsForm(store.ScopeKey, k.ID, name); return nil }},
{"Rotate…", func(m *consoleModel) tea.Cmd {
m.openForm(&formDialog{title: "Rotate " + name,
note: "A new key with the same project, owner and limits is shown once; this one keeps working for the overlap, then stops.",
fields: []formField{textF("overlap", "24h", "how long both work, like 48h; 0 stops this one now")},
submit: func(m *consoleModel, v []string) (tea.Cmd, error) {
if _, err := time.ParseDuration(v[0]); err != nil {
return nil, fmt.Errorf("overlap is a duration, like 24h")
}
return m.runCLI("Rotate "+name, "keys", "rotate", k.ID, "--overlap="+v[0]), nil
}})
return nil
}},
{"Revoke…", func(m *consoleModel) tea.Cmd {
m.confirmD = &confirmDialog{title: "Revoke " + name,
text: "It stops working at once. It is kept, revoked, for the usage that refers to it.",
@@ -441,6 +453,20 @@ func (m consoleModel) orgMenu(r orgRow) *menuDialog {
return nil
}},
menuItem{"Guardrails…", func(m *consoleModel) tea.Cmd { m.menuD = m.guardMenu(r); return nil }},
menuItem{"Longest key lifetime…", func(m *consoleModel) tea.Cmd {
m.openForm(&formDialog{title: "Longest key lifetime · " + r.path,
note: "Keys made here get at most this, and this when none is asked for. Existing keys are not changed.",
fields: []formField{textF("longest", "", "like 720h; off removes the limit here")},
submit: func(m *consoleModel, v []string) (tea.Cmd, error) {
if v[0] != "off" {
if d, err := time.ParseDuration(v[0]); err != nil || d <= 0 {
return nil, fmt.Errorf("a duration like 720h, or off")
}
}
return m.runCLI("Key lifetime "+r.path, "keys", "ttl", r.level+":"+r.id, v[0]), nil
}})
return nil
}},
menuItem{"Capacity quota…", func(m *consoleModel) tea.Cmd {
q := adminapi.QuotaView{Scope: r.level + ":" + r.id}
for _, x := range m.quotas {