Every template and handler message in internal/web goes through a JSON
catalogue under internal/web/locales/<code>/, with English as the fallback.
The language is the person's choice (a cookie set by /_lang/{code}), else
Accept-Language, else landing.language. Tests refuse a key some language
lacks, a translation with other arguments than English, and template text
left in English.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cluster page draws every card coloured by utilisation, memory or
temperature, and lists replicas whose every card is under 5% busy.
llmgw_gpu_assigned says which replica, model and owner hold each card.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
They were read once, at agent start, and published unchanged since.
Each sample now carries utilisation, memory and bandwidth in use, the SM
clock and uncorrected ECC errors. The node page charts each card, the
cluster page the fleet by class, Grafana has a GPU row, and three GPU
alerts are added.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A node that cannot run the CUDA 13 build installs the same version's
CUDA 12.9 build from vLLM's GitHub release, with PyTorch's cu129 torch;
the install computes on the GPU before it counts. The build can be
chosen per node — auto, cu13, cu129, none — on the cluster page, with
ctl engine-build, or over the admin API; the control plane keeps the
choice and tells the node at once.
Refs #2.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Headers never wrap, free text is clipped with the whole on hover, full-width
rows are only notes and forms. The cluster page hides columns empty on every
row and shortens node names; People shows two memberships and a count.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A GB200 NVL72 rack or an NVLink Switch system is a scale-up island: its
GPUs reach each other over NVLink whatever node they are in. Each GPU
now reports its fabric from nvidia-smi -q - state, status, ClusterUUID,
CliqueId, bandwidth - at registration and once a minute after, since a
fabric manager restarting drops every card out at once; and each node
reports nvidia-imex and its channels, without which the domain's GPUs
cannot share memory across nodes. A single machine's NVSwitch, which
reports a cluster of zeros, is no domain.
A replica spanning nodes is placed inside one domain first, on nodes
whose every card is joined and whose IMEX is up, and its shards are
told NCCL_MNNVL_ENABLE=1; InfiniBand is the fallback.
/admin/nvlink draws each domain as a rack - the NVLink Switch band on
top, light running down a backplane, trays of GPU chips that glow when
joined, pulse red when not, turn amber on degraded bandwidth - and
lists what keeps a tray out. A node's page shows its GPU-by-GPU NVLink
matrix, each card's NUMA node and nearest adapter, and its domain and
IMEX. doctor checks joining, bandwidth, IMEX and its channel; metrics
llmgw_gpu_nvlink_joined and llmgw_imex_up; /api/v1/nvlink.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A fabric's identity was its master subnet manager, which changes on a
failover: for the minute its ports took to agree, one fabric read as
two, and a name covering it as two separate fabrics. Each agent now
also lists every subnet manager on a fabric (saquery SMIR, through one
port per master, once a minute); managers listed together are one
fabric, named by the lowest GUID among them, so the identity survives
the master changing.
The same list says what a fabric can survive. The cluster page's new
subnet manager table, /api/v1/fabrics, doctor and metrics
(llmgw_fabric_master_sm, llmgw_fabric_subnet_managers,
llmgw_fabric_problems) show each fabric's master, priority and
standbys, and say when a fabric has no standby, its ports disagree on
the master, ports are up with no subnet manager (INIT), or the master
changed in the last hour. Every change of master a port reports is
kept, and a fabric's last day of them is listed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A fabric was what agent.yaml called it. With several independent
InfiniBand fabrics at a site, two given one name would have had a
replica spread across both, starting and then hanging on its first
collective. Each active InfiniBand port now reports the master subnet
manager it sees (sminfo, asked once a minute), which every port on one
fabric shares and ports on separate fabrics cannot; a RoCE port reports
the IP network of its RoCE v2 address. Ports also carry their node and
port GUIDs.
Placement divides a declared fabric by that physical identity and forms
a replica on one part only; a node whose adapters for one name are on
two fabrics is refused, and a node that cannot say is kept apart from
those that can. /api/v1/fabrics and the cluster page list each physical
fabric with its subnet manager, and say when one name covers two
fabrics, two names one fabric, an adapter is up but undeclared, or a
node cannot be checked. doctor reports each port's subnet manager, and
an agent on a node with adapters installs infiniband-diags when sminfo
is missing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Capacity, nodes, models and replicas come first, then a rule, then
traffic, people, what is being run and usage. The people and per-model
bar charts, which repeated the tables under them, are gone.
The page swaps its content every few seconds with htmx instead of
reloading, so a reader keeps their place, and it stops while the tab is
hidden. Without script the meta refresh still reloads it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The ranked bars said "8825 ms" beside a table that said "8.8 s". One formatter, humanLatency, now serves both, with a test.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Numeric column headers are right-aligned like their values; the
stylesheet only aligned the cells, so "REQUESTS" sat nowhere near its
numbers in every table that has them.
- Ranked bars are an HTML grid instead of a fixed-width SVG: the value
had 96 units and "34.2k tok 114 req" did not fit, so the figure a
reader came for was the one cut off. Labels ellipsise in CSS and give
their full text on hover.
- Mean latency reads as 5 ms, 8.8 s or 1m 51s rather than 110695 ms.
- The "What the cluster is doing" charts sat between the Models heading
and the models table; they now come before it.
- A fleet without GPUs shows its CPU capacity — nodes ready, memory,
disk free for weights — instead of "No GPUs have been reported", and
a node with no GPUs shows "—" rather than 0/0.
- The profile's forms are a form's width rather than the page's, and
the quotas page's price warning is a boxed banner rather than a line
of red text.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each page wrote its own nav: the same links in a different order, some
missing, the current page sometimes listed and sometimes not. A shared
"nav" template now draws it everywhere — Cluster, People, Quotas, Usage,
Mirror for an administrator, then Chat, Your keys and Sign out — with
the current page marked by aria-current. Pages rendered outside the
account pages (the dashboard, node logs) get it through the request
context their admin wrapper sets. The admin listener's own page keeps
its metrics/pprof/api links.
The chat and key pages said "project proj-3kq7…"; they now show the
project's path, and the key page no longer repeats the email.
A test renders several admin pages and requires the same menu on each.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The dashboard charted what the cluster had been billed for — requests, tokens,
spend, all from llmgw's own store — and nothing about whether it was healthy.
Saturation, queue depth, cache occupancy and replica counts were numbers in a
table at the instant the page loaded, with no way to see that one of them had
been climbing for an hour.
Four line charts now, read back from wherever the metrics were scraped to.
Read back rather than kept twice. llmgw publishes these already; what a gauge
cannot do is remember, and a ring buffer in this process would be a rival copy
of the same numbers with its own retention, lost on every restart and
disagreeing with the one an operator has on screen elsewhere. monitoring.prometheus
is empty by default and the panels are then absent rather than empty.
One axis per chart, always. Two measures of different scale belong in two
charts: a second y-axis lets any two lines be made to cross wherever the author
likes and the reader cannot tell it happened.
Four colours, handed out in fixed order, never cycled. A fifth series is summed
into "other" and drawn in grey — dropping it would make a chart that does not
add up, and inventing a hue for it would put two lines on the page that nobody
can tell apart. The fold is by timestamp, not by position: the tail's series do
not share sample times, and adding them by index slides points sideways onto
each other's clock.
A legend for every chart with more than one line, because colour alone is not
an encoding a reader can rely on.
A flat zero is drawn along the bottom rather than left blank: nothing happening
is a real answer, and an empty frame reads as a broken chart. A single sample
is not a line and says so instead of putting a dot in the corner.
One failed query fails all four panels. Three drawn and one blank invites
reading the blank one as "nothing is happening there".
NaN is dropped rather than plotted. Prometheus sends it as a string for a
sample it has no value for, Go's ParseFloat accepts it happily, and one NaN
coordinate is an SVG path that does not draw at all — taking the rest of the
line with it.
Axis maxima are rounded to a number a person would pick. An axis topping out at
0.8347 makes every gridline unreadable, and the shape is what a chart is for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
/admin/mirror, behind the same admin session as the other admin pages. What is
held, what is arriving with its byte count, what failed and why; start a fetch,
retry a failed one, remove a version.
Not mounted at all without a mirror configured. The right page for a deployment
with no mirror is no page, rather than an empty one pointing at nothing.
It reloads itself only while something is downloading. A reload at any other
moment takes with it whatever is half-typed into the fetch form, and there is
nothing to watch when nothing is moving.
A failed row offers Retry, not Remove. A fetch that failed left nothing on disk,
so the remove could only answer "the mirror holds nothing for this" — a button
whose single outcome is a confusing error. Starting the same download again is
what somebody looking at that row wants, and the mirror allows it: only a
download still running is refused.
An unreachable mirror is said outright instead of rendering as an empty table.
The two look nearly identical and mean opposite things: one is a mirror waiting
to be filled, the other is a machine to go and look at.
Dashboard takes NavExtras rather than a second bool. Two adjacent booleans at a
call site get swapped silently and are discovered by somebody clicking the
wrong link.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The dashboard could say what the cluster was made of and not who was
spending it. Usage was per key, which is the right unit for revoking
something and the wrong one for the question actually being asked: one
person holds a laptop key, a CI key and a notebook key, and adding three
rows up by eye is a chore that gets skipped.
Per person now, joined through the keys they own, and per model — counted
by the model that answered rather than the one requested, since those differ
whenever a fallback serves and what actually ran is the question. Headline
figures are stat tiles with a sparkline: a single current value is a number,
and plotting one is how a chart ends up with a single bar in it. The
sparkline says only rising, falling or flat, which is the one thing the
number cannot.
Ranked horizontal bars, because the labels are email addresses and model
names and the label is the point — the question is which person, not what
shape the distribution has. Rotated or truncated axis labels say the chart
matters more than reading it. Trimmed where drawn and complete on hover, so
the plot keeps its geometry and a reader can still find out whose row it is.
The palette is validated rather than chosen: every adjacent pair clears the
colour-vision separation threshold, and the dark values are the same hues
re-stepped for the dark surface instead of an automatic flip. Run through a
validator, because that is the only way anyone can honestly claim a palette
is colourblind-safe.
Which turned up a bug in the existing chart code. cssVar is an allowlist of
four names that silently returns --accent for anything else, and callers
wrap it in var() themselves — so every new series would have rendered in one
colour with accent-coloured text. Caught by a test written from the rule
that chart text wears text tokens, never the series colour.
A key with no owner is labelled "operator-issued keys" rather than unknown:
it predates accounts and is doing its job, and "unknown" would imply
something is wrong.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The metrics were exported and the only way to look at them was to stand up
Prometheus beside the binary. For a project whose whole shape is one static
binary with no dependencies, that is the wrong default: the common case is
somebody wanting to know how full their cluster is, and that should not cost
a monitoring stack.
/admin now shows capacity per GPU class with a usage bar, free whole nodes
per fabric — the number that decides whether a replica spanning nodes can
land, since free cards on the wrong fabric are not capacity for a model that
needs this one — and which replicas are holding hardware nobody is asking
for. Nodes gain their memory and how many model files they can see.
Taken from the same Summary that ctl status uses rather than derived in the
page, because two implementations of the same arithmetic drift apart and the
one on the screen is the one people would believe.
A faulty card counts as neither used nor free, matching the scheduler.
Counting it free is how a cluster looks like it has room it does not, and a
page that disagreed with placement about that would be worse than no page.
Prometheus stays supported and is now the answer to a narrower question:
history, alerting, several clusters at once. The dashboard shows what is
true now and keeps nothing, which is the honest division between them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Clicking "metrics" on the web dashboard answered "no api key was provided".
The page is served twice and its navigation assumed one of them. Metrics,
pprof and the management API exist on the admin listener; account
administration, the profile and the chat exist on the client-facing port. The
nav offered the first set in both places, so on the public port those links
fell through to the API-key middleware — which is a far more confusing way to
say "that is not here" than a 404 would have been.
Each page now offers what is actually reachable from it, and the public one
says where the rest lives rather than pretending it does not exist, including
why that port is not exposed: pprof can dump the process heap, and the heap
holds every API key in plaintext.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things the dashboard was missing: it could not act, and it was a column of
numbers.
Account administration moves to /admin/users — approving into a project,
suspending, reinstating. That is the work an operator does constantly and it
should not need a terminal. Three things stay on the CLI, and the line is not
arbitrary: promoting someone to admin, resetting a password, and issuing a key
on another person's behalf each turn a stolen admin session into more than it
was — permanent access, or somebody else's account. Approving and suspending
do not. Suspending signs the account out everywhere, or suspension is advisory
until the session happens to expire, and an admin cannot suspend themselves,
which is not a thing an interface should let you do by misclicking.
The charts are inline SVG generated in Go. A charting library would mean a
JavaScript toolchain in CI and a megabyte of bundle for what is a list of
rectangles, and the single-binary goal rules that out. Hourly over a day: fine
enough to show a working day, coarse enough that one slow minute is not an
incident. Empty hours are filled in, because a chart with hours missing where
nothing happened reads as a chart with hours missing from the data.
The SVG is marked as trusted HTML, so everything that reaches it is either a
number this package computed or escaped explicitly, and a bar's colour can only
be one of four names — a colour that could be anything is a way to write CSS
into every page the chart appears on. There are tests for both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pasting the gateway's address into a browser returned 404, which looks like a
broken endpoint rather than one that tells you how to use it.
Two pages, and the split between them is a security boundary rather than a
layout choice. The landing page is served without a key — it has to be,
because the person reading it is the one who does not have one yet — so it
contains no data at all: it says how to connect, not what is here. Model names
stay behind /v1/models. It reconstructs the address from the forwarded headers
so the copy-pasteable example names the address the client actually used
rather than the internal one; those headers are reflected into displayed text
and never used for a decision.
The dashboard shows nodes, models, replicas, placement failures and usage, and
lives on the admin listener only. That listener is loopback-bound and also
serves pprof, which can dump the process heap — where API keys sit in
plaintext — so reaching the page from elsewhere is an SSH tunnel's job. It is
read-only: anything that mints a credential stays in `llmgw keys`, where it
needs a shell rather than a browser session this project has no way to
authenticate.
Mounting the landing page uncovered a hole in the exemption list. An entry
ending in "/" was a prefix rule, and "/" ends in "/", so exempting the landing
page would have exempted every path there is — authentication still installed,
still logging, and letting the whole API through. "/" is now exact, with a
test enumerating what must and must not be exempt.
Server-rendered templates and one stylesheet, embedded. The single-binary goal
rules out a JavaScript toolchain and nothing on either page needs one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
