Console: rotate a key, cap key lifetimes, test the webhooks; README, onboarding and console docs caught up
The README's status stopped at v0.9 and listed gaps since closed; onboarding still said there was no SSO and no roles. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
1b71c38f89
commit
4979e08d64
7 files changed
+111
-22
No files matched your search
@@ -33,6 +33,8 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md).
|
||||
| v0.7.0 | — | Conversations, agent memory, disk management | released |
|
||||
| v0.8.0 | — | PostgreSQL, leader election, agent failover | released |
|
||||
| v0.9.0 | — | Managing people and the tenancy tree | released |
|
||||
| v0.10.0 | — | InfiniBand fabrics and NVLink, doctor, node add, engines installed by the nodes | released |
|
||||
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
|
||||
| v1.0.0 | 5 | Tenancy, prefix affinity, benchmarking | planned |
|
||||
|
||||
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
|
||||
@@ -40,12 +42,34 @@ control plane places replicas — including ones larger than a single machine
|
||||
models load when someone asks for them and stop when nobody does, and keys,
|
||||
budgets and rate limits decide who may use any of it.
|
||||
|
||||
Before v1.0.0: tracing, reference documentation, and filtering on `ctl nodes`
|
||||
and `ctl replicas` for fleets too large to read a whole table of. Two items
|
||||
that were on this list are not gaps: a model falls through to its fallbacks
|
||||
when there is no capacity as well as when a budget is spent, and routing
|
||||
profiles are chosen by the key rather than by a request header on purpose —
|
||||
a header would let any client promote itself into the top priority band.
|
||||
Since v0.11.0, not yet released (see [CHANGELOG.md](CHANGELOG.md),
|
||||
"Unreleased"):
|
||||
|
||||
- **Security hardening** — signed releases only, a token on the admin API,
|
||||
engines not run as root, TLS for the agent port, etcd and Patroni, a
|
||||
firewall for llmgw's internal ports, encrypted backups, bounded
|
||||
connections, sign-in throttling, a Content-Security-Policy.
|
||||
- **Guardrails** — policies that check requests and answers (PII, secrets,
|
||||
prompt injection, deny patterns, a judge model such as Llama Guard), in
|
||||
observe or enforce mode, attached to any part of the tenancy tree:
|
||||
[docs/guardrails.md](docs/guardrails.md).
|
||||
- **Administration** — an audit log of every change (`ctl audit`), staff
|
||||
roles (viewer, operator, billing, admin), single sign-on through OpenID
|
||||
Connect and provisioning through SCIM 2.0, key lifetime limits and
|
||||
rotation with overlap (`keys ttl`, `keys rotate`), usage export as CSV
|
||||
(`ctl usage export`).
|
||||
- **Notifications** — webhooks, signed or Slack-shaped, for a model parked,
|
||||
a node lost, a budget spent and new weights published (`notify.webhooks`,
|
||||
`ctl notify test`), and Alertmanager receiver templates.
|
||||
|
||||
Every one of these is in the console as well as on the command line:
|
||||
[docs/console.md](docs/console.md).
|
||||
|
||||
What is still ahead is in [BACKLOG.md](BACKLOG.md). Two things that look like
|
||||
gaps are not: a model falls through to its fallbacks when there is no
|
||||
capacity as well as when a budget is spent, and routing profiles are chosen
|
||||
by the key rather than by a request header on purpose — a header would let
|
||||
any client promote itself into the top priority band.
|
||||
|
||||
One thing is worth saying plainly:
|
||||
multi-node replicas are proven on a simulated cluster and on real hardware
|
||||
@@ -70,6 +94,12 @@ them. On such a machine, run `llmgw doctor --json` first and keep the report;
|
||||
how many of each, and where the weights should live
|
||||
- [docs/data.md](docs/data.md) — what llmgw stores, what it does not, and
|
||||
what leaves the machine; written for a security review
|
||||
- [docs/console.md](docs/console.md) — the console that bare `llmgw` opens:
|
||||
every view, and the command each of its actions runs
|
||||
- [docs/guardrails.md](docs/guardrails.md) — checking requests and answers:
|
||||
policies, detectors, a judge model, and where each applies
|
||||
- [docs/onboarding.md](docs/onboarding.md) — accounts, keys, limits, staff
|
||||
roles and single sign-on
|
||||
- [docs/reference.md](docs/reference.md) — every configuration key, generated
|
||||
from the structs and checked by a test, so it cannot describe a setting that
|
||||
does not exist or miss one that does
|
||||
@@ -79,6 +109,8 @@ them. On such a machine, run `llmgw doctor --json` first and keep the report;
|
||||
port, with streaming and the forwarded headers right
|
||||
- [deploy/ha/](deploy/ha/) — running two control planes, and the one health
|
||||
check that has to be right
|
||||
- [deploy/monitoring/](deploy/monitoring/) — Prometheus rules, Alertmanager
|
||||
with receiver templates, and Grafana dashboards
|
||||
- [docs/node-onboarding.md](docs/node-onboarding.md),
|
||||
[docs/onboarding.md](docs/onboarding.md) — adding machines, and adding people
|
||||
|
||||
@@ -100,8 +132,12 @@ llmgw ctl explain-placement llama # why a model has not been placed
|
||||
llmgw ctl drain gpu-a-01 # move work off a node, one replica at a time
|
||||
llmgw ctl cordon gpu-a-01 # stop new placement, leave what runs
|
||||
sudo llmgw doctor # on a node: what its GPUs, NVLink and fabric need
|
||||
llmgw ctl audit --since 24h # who changed what
|
||||
llmgw ctl usage export --from 2026-09-01 --to 2026-10-01 --by project,model
|
||||
```
|
||||
|
||||
Bare `llmgw` opens the console, which does all of this from one screen.
|
||||
|
||||
`ctl` talks to the admin port (`:9090` by default), not the client-facing one.
|
||||
Add `--json` to any command for machine-readable output.
|
||||
|
||||
|
||||
+9
-4
@@ -67,7 +67,8 @@ and power, and each model's requests a second. The history comes from the
|
||||
Prometheus named by `monitoring.prometheus` in server.yaml; without one the
|
||||
charts draw what this console has seen since it opened, and say so. Network
|
||||
and disk come from each machine's node exporter, which `llmgw node add`
|
||||
installs. A node's and a model's own pages have their charts too, over the
|
||||
installs. **N** sends a test notification to each webhook in server.yaml's
|
||||
`notify` section and says how each answered (`llmgw ctl notify test`). A node's and a model's own pages have their charts too, over the
|
||||
same range.
|
||||
|
||||
**Nodes** lists every node with its state, load, GPUs, replicas, disk and
|
||||
@@ -158,7 +159,9 @@ project, their limits, a new password (their sessions close), delete.
|
||||
|
||||
**API keys** — every key, its project, owner, state and last use. **A**
|
||||
issues one; the key is shown once, in the result, and never again. On a row:
|
||||
its limits, or revoke it.
|
||||
its limits, **Rotate…** — a new key with the same project, owner and limits,
|
||||
the old one working on for the overlap you give (`llmgw keys rotate`) — or
|
||||
revoke it.
|
||||
|
||||
**Accounts** rows also have **Staff role…**:
|
||||
- `viewer` looks at every page under /admin and changes nothing;
|
||||
@@ -182,8 +185,10 @@ with anything secret taken out. `llmgw ctl audit` is the same, with filters.
|
||||
projects, with members, keys and limits. **A** adds an organization. On a
|
||||
row: add a team or project inside it, rename, move, limits for the level and
|
||||
for each of its members, its capacity quota, and for a project how long
|
||||
conversations and agent notes are kept. **Remove** takes away only something
|
||||
empty.
|
||||
conversations and agent notes are kept. **Longest key lifetime…** caps how
|
||||
long a key made there may live, and is the expiry of one made without any
|
||||
(`llmgw keys ttl`; `auth.max_key_ttl` in server.yaml caps every key).
|
||||
**Remove** takes away only something empty.
|
||||
|
||||
Limits are requests and tokens a minute, a budget per day or month, and the
|
||||
models allowed. Each level only narrows what is above it.
|
||||
|
||||
+23
-11
@@ -325,8 +325,15 @@ llmgw keys list --store … --project default
|
||||
minute. It is the column that tells you whether a key is still in use, which is
|
||||
what makes rotation safe:
|
||||
|
||||
**To rotate**, issue the new key, let both work while the user switches, watch
|
||||
`LAST USED` on the old one stop moving, then revoke it.
|
||||
**To rotate**, `llmgw keys rotate <key-id> --overlap 48h` issues a new key with
|
||||
the same project, owner and limits and lets the old one work for the overlap,
|
||||
then stop. Watch `LAST USED` on the old one stop moving before it runs out.
|
||||
|
||||
**To cap how long keys live**, `llmgw keys ttl project:Acme/ml/train 720h` (or
|
||||
`team:`, `org:`; `off` removes it), and `auth.max_key_ttl` in server.yaml for
|
||||
every key. A key is made within the shortest limit that applies, and one asked
|
||||
for without an expiry gets that limit. `keys list --all` shows each key's kind
|
||||
— service (a project's), member, personal — and when it expires.
|
||||
|
||||
```
|
||||
llmgw keys revoke --store … key-mpmip77twpt7qaxc
|
||||
@@ -433,21 +440,26 @@ Three things are deliberately not there:
|
||||
| `llmgw keys create` | Issuing a credential on another person's behalf |
|
||||
|
||||
Each of those turns a stolen admin cookie into more than it was, and each is
|
||||
rare enough that a shell is no hardship. Cordon, drain and everything else that
|
||||
touches the cluster are also CLI-only for the same reason: the dashboard shows
|
||||
the cluster, it does not steer it.
|
||||
rare enough that a shell is no hardship. Cordon and drain are CLI-only for the
|
||||
same reason; what the pages do change on the cluster — serving models through
|
||||
the mirror, a node's engines — needs the `operator` or `admin` staff role.
|
||||
|
||||
Every administrative action is logged with who did it, including the ones that
|
||||
were refused.
|
||||
Every administrative change is recorded in the audit log — who, when, through
|
||||
what, the fields given with secrets taken out, and how it ended, refusals
|
||||
included: `llmgw ctl audit`, or **Audit** in the console.
|
||||
|
||||
## What this does not do
|
||||
|
||||
- **No email.** No approval notice, no self-service password reset, no address
|
||||
verification. The loop is manual on purpose at this size; if it stops being
|
||||
the right size, that is the thing to automate.
|
||||
- **No SSO.** Accounts are local, with local passwords.
|
||||
- **No roles.** There is one flag, `is_admin`, and it grants looking at the
|
||||
dashboard. Everything else an administrator does is the CLI, and having a
|
||||
shell here is what makes someone one.
|
||||
- **Single sign-on is optional.** With `auth.oidc` people sign in through
|
||||
your identity provider, and SCIM 2.0 (`auth.scim`) creates, updates and
|
||||
suspends accounts from it; see [install.md](install.md#single-sign-on).
|
||||
Without them accounts are local, with local passwords.
|
||||
- **Staff roles are coarse.** `viewer`, `operator`, `billing` and `admin`
|
||||
(`llmgw users role`) decide what someone may see and do under /admin; an
|
||||
organization's manager can view it but not change it. Every change is in
|
||||
the audit log (`llmgw ctl audit`).
|
||||
- **Rate limits are per gateway instance.** With two instances each counts its own,
|
||||
so the effective limit doubles. v1 is single-instance by design.
|
||||
@@ -648,6 +648,12 @@ func TestGuardrailsSayWhereTheyApplyFrom(t *testing.T) {
|
||||
if items := m.orgMenu(m.people.orgs[0]).items; !slices.ContainsFunc(items, func(i menuItem) bool { return i.label == "Guardrails…" }) {
|
||||
t.Error("no Guardrails… in the menu")
|
||||
}
|
||||
if items := m.orgMenu(m.people.orgs[0]).items; !slices.ContainsFunc(items, func(i menuItem) bool { return i.label == "Longest key lifetime…" }) {
|
||||
t.Error("no Longest key lifetime… in the menu")
|
||||
}
|
||||
if items := m.keyMenu(store.KeyRow{}).items; !slices.ContainsFunc(items, func(i menuItem) bool { return i.label == "Rotate…" }) {
|
||||
t.Error("no Rotate… on a key")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheAuditViewListsChangesAndOpensOne(t *testing.T) {
|
||||
|
||||
@@ -536,6 +536,10 @@ func (m consoleModel) key(k tea.KeyMsg) (tea.Model, tea.Cmd) {
|
||||
case cOrgs:
|
||||
m.createOrgForm()
|
||||
}
|
||||
case "n":
|
||||
if m.menu == cMonitor {
|
||||
return m, m.runCLI("Test the webhooks", "ctl", "--server="+m.client.base, "notify", "test")
|
||||
}
|
||||
case "e":
|
||||
if m.menu == cUsage {
|
||||
m.usageExportForm()
|
||||
|
||||
@@ -258,7 +258,7 @@ func side(w int, a, b string) string {
|
||||
}
|
||||
|
||||
func (m consoleModel) monitorPanel(pw int) (string, string, string) {
|
||||
keys := "→ or Enter open · ↑↓ choose · Q leave"
|
||||
keys := "→ or Enter open · ↑↓ choose · N test the webhooks · Q leave"
|
||||
if m.inPanel {
|
||||
keys = "1–4 or R range · PgUp/PgDn scroll · ← back · Q leave"
|
||||
}
|
||||
|
||||
@@ -311,6 +311,18 @@ func (m consoleModel) keyMenu(k store.KeyRow) *menuDialog {
|
||||
name := k.Name + " (" + k.Prefix + ")"
|
||||
return &menuDialog{title: name, items: []menuItem{
|
||||
{"Its limits…", func(m *consoleModel) tea.Cmd { m.limitsForm(store.ScopeKey, k.ID, name); return nil }},
|
||||
{"Rotate…", func(m *consoleModel) tea.Cmd {
|
||||
m.openForm(&formDialog{title: "Rotate " + name,
|
||||
note: "A new key with the same project, owner and limits is shown once; this one keeps working for the overlap, then stops.",
|
||||
fields: []formField{textF("overlap", "24h", "how long both work, like 48h; 0 stops this one now")},
|
||||
submit: func(m *consoleModel, v []string) (tea.Cmd, error) {
|
||||
if _, err := time.ParseDuration(v[0]); err != nil {
|
||||
return nil, fmt.Errorf("overlap is a duration, like 24h")
|
||||
}
|
||||
return m.runCLI("Rotate "+name, "keys", "rotate", k.ID, "--overlap="+v[0]), nil
|
||||
}})
|
||||
return nil
|
||||
}},
|
||||
{"Revoke…", func(m *consoleModel) tea.Cmd {
|
||||
m.confirmD = &confirmDialog{title: "Revoke " + name,
|
||||
text: "It stops working at once. It is kept, revoked, for the usage that refers to it.",
|
||||
@@ -441,6 +453,20 @@ func (m consoleModel) orgMenu(r orgRow) *menuDialog {
|
||||
return nil
|
||||
}},
|
||||
menuItem{"Guardrails…", func(m *consoleModel) tea.Cmd { m.menuD = m.guardMenu(r); return nil }},
|
||||
menuItem{"Longest key lifetime…", func(m *consoleModel) tea.Cmd {
|
||||
m.openForm(&formDialog{title: "Longest key lifetime · " + r.path,
|
||||
note: "Keys made here get at most this, and this when none is asked for. Existing keys are not changed.",
|
||||
fields: []formField{textF("longest", "", "like 720h; off removes the limit here")},
|
||||
submit: func(m *consoleModel, v []string) (tea.Cmd, error) {
|
||||
if v[0] != "off" {
|
||||
if d, err := time.ParseDuration(v[0]); err != nil || d <= 0 {
|
||||
return nil, fmt.Errorf("a duration like 720h, or off")
|
||||
}
|
||||
}
|
||||
return m.runCLI("Key lifetime "+r.path, "keys", "ttl", r.level+":"+r.id, v[0]), nil
|
||||
}})
|
||||
return nil
|
||||
}},
|
||||
menuItem{"Capacity quota…", func(m *consoleModel) tea.Cmd {
|
||||
q := adminapi.QuotaView{Scope: r.level + ":" + r.id}
|
||||
for _, x := range m.quotas {
|
||||
|
||||
Reference in new issue
Block a user