/forgot mails a link to set a new password when the cluster's mail settings
say how - SMTP relay, sender, and the public base_url the link points at,
never the request's Host - and says the same whether or not the address has
an account. /reset sets it once, closing every session. Links are kept by
their hash in password_resets, good for an hour, a newer one replacing an
older. Without mail, /forgot says to ask an operator, and llmgw users
reset-link makes the link to hand over. Single sign-on accounts get none.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
firewall.enabled (default true) is a cluster setting, applied live: the
leader sends a Firewall message to every agent when it registers and when
the setting or firewall_networks changes, and the agent (root) takes llmgw's
nft table out or installs it. A control plane writes the state to
/var/lib/llmgw/firewall.json, which llmgw-firewall-sync.path applies as root
with `llmgw firewall sync`; server init and server add install the units,
`llmgw firewall sync --install` on one set up before. node add and server
add put no firewall on a machine while the setting is off.
`llmgw wipe` removes llmgw from a machine after listing what goes and asking
for its name: units, engines' processes, the nft table, /etc/llmgw,
/var/lib/llmgw, the agent's state, the sudo grant, the CA from the trust
store, keepalived's configuration when llmgw wrote it, and the users.
--models, --store (Patroni member, etcd, or the local database) and --all
take more. The console has it under Do.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Conflicts: config imports and docgen sources (both kept); agent.pb.go
regenerated with buf from the merged proto (MIG devices and trace context).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A reconcile pass that does something is a trace, with placement (where it
went, or every candidate turned down), scale-up/down, drain, stop, eviction,
preemption, reaping and restarts beneath it. A replica's start is a span open
from placement until it serves; the start command carries its context in a
new ControlMessage.trace field, and the agent continues the trace through
the weights fetch, engine launch, rendezvous and readiness wait. Stops,
drains, adoption and engine installs are traced on the node, model jobs are
a trace with a span per step linked to the replica's start, and agent.yaml
takes the same tracing block as server.yaml. Off, every span is the no-op one
and commands carry nothing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Engine trtllm runs trtllm-serve on one node, through a launcher of
llmgw's that demands the replica's key and turns on the Prometheus
endpoint its queue and cache gauges are read from; the router reads it
at /prometheus/metrics. Brought by the operator (engines.trtllm.python),
not installed; checked by ctl engine-check trtllm. Written against the
documented CLI, server and metrics and tested with stand-ins: not yet
run on a datacenter GPU.
mirror import-ollama copies a model's GGUF, and a vision model's
projector, from Ollama's directory into the mirror, checked against the
digest Ollama named each blob by; mirror import does the same for any
local file. The console's Mirror view imports from Ollama with I.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The console gains live views of nodes, GPUs, models, replicas, usage and
agent logs, and a settings tree over every key of server.yaml (agent.yaml on
a node), generated from the configuration structs: values with their
defaults and doc comments, edited in dialogs, saved only once the service's
own loader accepts them, comments kept. The generator now tells blocks apart
by package, which brings the TLS, engine and supervisor settings back into
the reference.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CPU replicas start on half the machine's CPUs, not all of them: two busy
llama.cpp servers each spinning on all 8 vCPUs fell to 0.8 tok/s apiece,
where at 4 threads they keep 5.5. llamacpp.threads sets it per node.
Inference requests can be traced over OTLP/HTTP (tracing.endpoint), with
admission, schedule and upstream spans and traceparent sent to the engine.
Security: onboarding no longer puts the cluster root in the system trust
store and removes it where it was; the mirror's mutual-TLS API can be
limited to named certificates (--api-client); init.log no longer keeps the
administrator's password; a warning when auth is off on a public listen;
the recorded actor is sanitised and sudo-aware; Ray's dashboard is off, and
the remaining engine gaps are documented.
reconciler.go and agent.go are split by topic, the status mapping of each
failure is tested, and the plan's checklist is brought up to date.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was no reference. Four narrative documents explain how to install and
operate llmgw, and the only way to learn that `lease_ttl` exists was to read
the one that happened to mention it — or the struct.
Written by hand it would have been wrong within a fortnight. Six settings
arrived in the last two weeks alone: store, servers, lease_ttl, the two
retention windows, min_disk_gb. A reference nobody maintains is worse than
none, because it is the same list with the setting you came to look up quietly
missing from it.
So it is derived from the structs, with the field comments as the descriptions
— which means the only way for it to be wrong is for the code to be wrong, and
it makes the comments worth writing twice over. 1163 lines, 24 blocks, 87 keys.
Two tests keep that true after everyone has forgotten this exists. One
regenerates and compares, so adding a setting without regenerating is a
failure rather than an omission — checked by adding one and watching it fail.
The other walks the tree for koanf tags and fails on a configuration file
that is in neither the source list nor the short list of structs an operator
never writes, because a list is a thing to forget.
The generator finds the repository from its own path rather than the working
directory, so `go generate ./...` produces the same document from anywhere.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
