The manual the binary carries, rendered on the client port: a sidebar of
the documents in the console's reading order (now one list, llmgw.Manual,
for both), heading anchors, links between documents kept in the portal and
links to files outside the binary sent to the repository, no script.
Public by default; landing.docs_sign_in sends visitors to sign in first.
docs/clients.md shows the official OpenAI and Anthropic SDKs against
llmgw in Python, TypeScript, Go and curl, with llmgw's headers; the
portal's index, the landing page and the account page link to it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A starter model with no room failed init after ten minutes and left the
firewall and the judge uninstalled.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The README's status stopped at v0.9 and listed gaps since closed; onboarding
still said there was no SSO and no roles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
notify.webhooks sends signed JSON or Slack-compatible messages, queued,
retried, de-duplicated hourly; ctl notify test checks each webhook.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new internal/clientip says who sent a request and how: the connection's
address, scheme and Host, or through a configured proxy what it forwarded.
The base URL in pages and setup scripts, isHTTPS for cookies, and the
per-address limits of auth and the sign-in forms use it. /setup scripts are
no-store.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The admin listener answered any local process: models added, nodes
upgraded, engine keys listed, the heap dumped. /api and /debug now demand
the token the server writes 0600 to its state directory on first start;
ctl, the console and node add send it (sudo, LLMGW_ADMIN_TOKEN or
--token-file). pprof is off unless debug_pprof. admin_listen off loopback
is refused unless admin_remote. Dashboard pages, /metrics and /health are
unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The supervisor drops to the run_as user (default llmgw-engine) when the
agent is root: its credential and groups, a 0700 home under the work
directory as the working directory, HOME and XDG_CACHE_HOME there. A node
without the user keeps engines as root and logs why. node add creates the
user in the video and render groups.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Without agent_tls the control plane served plaintext h2c on every
interface, and any machine reaching it could register as any node. The
server now refuses to start that way unless agent_insecure: true, and then
binds loopback when agent_listen names only a port.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Monitor draws CPU, memory, GPUs, requests, tokens, latency, network and disk
as Braille charts over 5m/1h/24h/7d, then per node, card and model
sparklines; node and model pages have their own. History is read from
monitoring.prometheus through POST /api/v1/metrics/range, or without it
from what the console saw. The control plane serves the cluster's machines
for Prometheus http_sd at /prometheus/targets on its metrics port, and node
add installs prometheus-node-exporter.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The console starts a serve or update job over the admin API and stays,
instead of closing to follow the download. Jobs lists them with their
progress and steps; the sidebar counts those running. ctl model serve and
update take --detach; ctl jobs lists jobs and follows one.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Ranking what was free first made every file come out on the CPU on a cluster
whose one card was in use. The plan now ranks by hardware alone, free now
only breaking a tie; a model served onto cards that are busy is added and
waits for them, the job ending as waiting rather than failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The control plane reads the model's shape from one GGUF header or its
config.json and lists, for every file, the best way it runs here: one card,
several of a node's (by layer for llama.cpp, tensor-parallel for vLLM),
several nodes' over one fabric (vLLM, pipeline-parallel between them), or a
CPU node; what each card holds and whether it is free now. ctl model plan
prints it; model serve takes its file and way unless told, gives a GGUF on
GPUs a GPU profile, and with --fallbacks adds the next best ways. The
console shows it per file and asks for the way and the fallback.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
model serve takes --projector, fetched beside the weights and loaded by
llama.cpp with --mmproj, and --format safetensors with --gpu-class and --tp,
which fetches the weights with their config and tokenizer and runs them on
vLLM. The console offers both, F switching between GGUF and original
weights. The mirror view is one row per version with its files on Enter.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every document and a page for every command, built from their help, read
like info: topics, contents, find. ? opens the console's own page, which is
new and covers every screen.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A model served from the registry records its origin: repository, commit and
each file's checksum; one served before is worked out from the mirror's
record of where it fetched the file. Every mirror.update_check (6h) the
control plane asks whether those files changed there. ctl model updates
lists it, ctl model update fetches beside the old files, checks, swaps and
replaces replicas one at a time, ctl model rollback goes back. The console
shows it on Models and offers the same.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
