Engine trtllm runs trtllm-serve on one node, through a launcher of
llmgw's that demands the replica's key and turns on the Prometheus
endpoint its queue and cache gauges are read from; the router reads it
at /prometheus/metrics. Brought by the operator (engines.trtllm.python),
not installed; checked by ctl engine-check trtllm. Written against the
documented CLI, server and metrics and tested with stand-ins: not yet
run on a datacenter GPU.
mirror import-ollama copies a model's GGUF, and a vision model's
projector, from Ollama's directory into the mirror, checked against the
digest Ollama named each blob by; mirror import does the same for any
local file. The console's Mirror view imports from Ollama with I.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A starter model with no room failed init after ten minutes and left the
firewall and the judge uninstalled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The README's status stopped at v0.9 and listed gaps since closed; onboarding
still said there was no SSO and no roles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Members' etcd speaks TLS to peers and clients with a certificate from the
cluster's authority and asks clients for one; Patroni reaches it with the
member's certificate and serves its API over TLS. A running plain-HTTP
cluster is moved with Patroni paused, so PostgreSQL serves throughout:
Patroni's lease-free keys are carried over to the new etcd.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new internal/clientip says who sent a request and how: the connection's
address, scheme and Host, or through a configured proxy what it forwarded.
The base URL in pages and setup scripts, isHTTPS for cookies, and the
per-address limits of auth and the sign-in forms use it. /setup scripts are
no-store.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A wrong password from anywhere counted against the address, so ten locked
its owner out. Attempts are now counted per address and client, with a
looser bound per address alone. The sign-in form carries a token matched
by a cookie (login CSRF). POST /login and /signup are rationed per client
address, a hash waits at most ten seconds for a turn, and server init
writes signup_open: false.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Patroni's REST API accepted changes from anyone on port 8008, and
PostgreSQL ran with sslmode=disable between machines. Setup now keeps a
fourth secret for the API (carried to members of older clusters), issues
each member a certificate from the cluster's authority, writes
restapi.authentication into each member's config and reloads it, and
turns ssl on with hostssl remote rules through an authenticated PATCH of
the shared configuration. The DSN becomes sslmode=verify-full against the
authority's certificate whenever the cluster has TLS, so a control plane
without the authority's key reconnects too. server add authenticates its
PATCH and lets the new control plane in by hostssl. Tried on the e2e
cluster: unauthenticated PATCH 401, plaintext refused, replication
streaming, the control plane on verify-full.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The admin listener answered any local process: models added, nodes
upgraded, engine keys listed, the heap dumped. /api and /debug now demand
the token the server writes 0600 to its state directory on first start;
ctl, the console and node add send it (sudo, LLMGW_ADMIN_TOKEN or
--token-file). pprof is off unless debug_pprof. admin_listen off loopback
is refused unless admin_remote. Dashboard pages, /metrics and /health are
unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Without agent_tls the control plane served plaintext h2c on every
interface, and any machine reaching it could register as any node. The
server now refuses to start that way unless agent_insecure: true, and then
binds loopback when agent_listen names only a port.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Five minutes at five-second steps is sixty samples on a chart a hundred and
thirty dots wide, and resampling by count drew them on its right half.
Points are now put in the column of their time, held across the columns
between samples, a series that began late starting where it began.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Monitor draws CPU, memory, GPUs, requests, tokens, latency, network and disk
as Braille charts over 5m/1h/24h/7d, then per node, card and model
sparklines; node and model pages have their own. History is read from
monitoring.prometheus through POST /api/v1/metrics/range, or without it
from what the console saw. The control plane serves the cluster's machines
for Prometheus http_sd at /prometheus/targets on its metrics port, and node
add installs prometheus-node-exporter.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The console starts a serve or update job over the admin API and stays,
instead of closing to follow the download. Jobs lists them with their
progress and steps; the sidebar counts those running. ctl model serve and
update take --detach; ctl jobs lists jobs and follows one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ranking what was free first made every file come out on the CPU on a cluster
whose one card was in use. The plan now ranks by hardware alone, free now
only breaking a tie; a model served onto cards that are busy is added and
waits for them, the job ending as waiting rather than failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
