Commit Graph
20 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5 753f4e3265 Let a node say how much room it has left
CI / test (push) Failing after 1m31s
Disk was the one resource nothing reported. `statfs` was already being called
on every model directory, but only to ask whether the filesystem was a network
one — the free space it returns in the same struct was thrown away. So the
only way to find out why a node was filling up was to ssh to it, which is what
this system exists to avoid.

Reported per filesystem rather than per configured directory. Free space
belongs to the disk, not the path: two model directories on one disk are one
pool of bytes, and counting them separately would let the control plane add
24 GB to 24 GB and place a 40 GB model on a node that has 24.

Shared filesystems are summed separately from local ones and left out of a
node's own total. Space on a filer belongs to everyone mounting it, so twenty
nodes would otherwise each report the same free terabyte, and a scheduler
would place twenty models into room for one.

Nothing reported means nothing is known, everywhere: an agent too old to send
storage and a node with no model directories both send nothing, and both must
read as unknown rather than as a full disk. `ctl nodes` and the dashboard show
a dash, the metric is absent rather than zero, and a missing directory — which
is what an unmounted filer looks like — is skipped rather than reported as
having no space.

Shown as free out of total, because the first is useless without the second:
8 GB free is comfortable on a 10 TB filer and nearly full on a 10 GB root
partition, and only the second stops a model being placed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:38:57 +02:00
Ilker ManapandClaude Opus 5 0cac9d6986 Make the landing page look like something worth having a key for
CI / test (push) Successful in 1m30s
It is the only page a stranger sees, and until they have a key it is the
whole product: a stack of headings and shell snippets said "internal tool
you probably should not be here".

Now it opens with what this is and a diagram of why it exists — one address,
many GPUs, fabrics that cannot see each other — then the base URL as a
headline rather than a line of prose with a colon in it, the route to a key
as three cards, and the client examples as tabs instead of a pile.

The tabs are radio buttons and a stylesheet. No JavaScript decides what is on
the page, so someone who arrives with scripting off still gets every example;
the only script is the copy buttons, which exist only where the clipboard API
does and so cannot leave a dead control behind.

The page says exactly what it said before, including that it lists no models
and why. The diagram names no model, no node and no count: fabrics are drawn
because llmgw keeps them apart, not because this install has two.

Tests cover the parts that are easy to break later: every example lives in
the page itself rather than being fetched per tab, the reflected base URL —
now in five places rather than one — still cannot write markup, and the
diagram carries a label for anyone not looking at it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 19:14:04 +02:00
Ilker ManapandClaude Opus 5 f8bb78103f Stop the ranked charts from spilling over the tables below them
CI / test (push) Successful in 1m32s
The bars were wrapped in <div class="chart">, and .chart was already the
fixed-height box the time-series charts are drawn in: 120px, with a border
and a background. A chart that grows a row at a time does not fit in a cage,
so the SVG overflowed it and landed on the table underneath.

Renamed to .ranked, which is a wrapper rather than the mark itself, and
capped at the design width so a wide window scales the box instead of
inflating the 12px labels.

The test reads the class back out of the markup and checks the stylesheet
does not cap it, because the collision was invisible in both files on their
own — each rule was reasonable, and only the shared name was wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 19:06:53 +02:00
Ilker ManapandClaude Opus 5 eb9e9b348c Show who is using the cluster and what they are running
CI / test (push) Successful in 1m28s
The dashboard could say what the cluster was made of and not who was
spending it. Usage was per key, which is the right unit for revoking
something and the wrong one for the question actually being asked: one
person holds a laptop key, a CI key and a notebook key, and adding three
rows up by eye is a chore that gets skipped.

Per person now, joined through the keys they own, and per model — counted
by the model that answered rather than the one requested, since those differ
whenever a fallback serves and what actually ran is the question. Headline
figures are stat tiles with a sparkline: a single current value is a number,
and plotting one is how a chart ends up with a single bar in it. The
sparkline says only rising, falling or flat, which is the one thing the
number cannot.

Ranked horizontal bars, because the labels are email addresses and model
names and the label is the point — the question is which person, not what
shape the distribution has. Rotated or truncated axis labels say the chart
matters more than reading it. Trimmed where drawn and complete on hover, so
the plot keeps its geometry and a reader can still find out whose row it is.

The palette is validated rather than chosen: every adjacent pair clears the
colour-vision separation threshold, and the dark values are the same hues
re-stepped for the dark surface instead of an automatic flip. Run through a
validator, because that is the only way anyone can honestly claim a palette
is colourblind-safe.

Which turned up a bug in the existing chart code. cssVar is an allowlist of
four names that silently returns --accent for anything else, and callers
wrap it in var() themselves — so every new series would have rendered in one
colour with accent-coloured text. Caught by a test written from the rule
that chart text wears text tokens, never the series colour.

A key with no owner is labelled "operator-issued keys" rather than unknown:
it predates accounts and is doing its job, and "unknown" would imply
something is wrong.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 18:38:21 +02:00
Ilker ManapandClaude Opus 5 c232f41a66 Put the fleet view in the dashboard, so nothing else is needed to see it
CI / test (push) Successful in 1m32s
The metrics were exported and the only way to look at them was to stand up
Prometheus beside the binary. For a project whose whole shape is one static
binary with no dependencies, that is the wrong default: the common case is
somebody wanting to know how full their cluster is, and that should not cost
a monitoring stack.

/admin now shows capacity per GPU class with a usage bar, free whole nodes
per fabric — the number that decides whether a replica spanning nodes can
land, since free cards on the wrong fabric are not capacity for a model that
needs this one — and which replicas are holding hardware nobody is asking
for. Nodes gain their memory and how many model files they can see.

Taken from the same Summary that ctl status uses rather than derived in the
page, because two implementations of the same arithmetic drift apart and the
one on the screen is the one people would believe.

A faulty card counts as neither used nor free, matching the scheduler.
Counting it free is how a cluster looks like it has room it does not, and a
page that disagreed with placement about that would be worse than no page.

Prometheus stays supported and is now the answer to a narrower question:
history, alerting, several clusters at once. The dashboard shows what is
true now and keeps nothing, which is the honest division between them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 18:15:30 +02:00
Ilker ManapandClaude Opus 5 c174e46902 Read the filename from a path comment inside the block
CI / test (push) Successful in 1m35s
The commonest convention of all, and the one the extractor missed. Asked
for a two-file Go server, qwen2.5-coder-3b opened a plain ```go fence and
wrote "// cmd/server/main.go" as the first line of the file — not on the
fence, not above it. Both files downloaded as snippet-1.go and
snippet-2.go, which is the feature failing at the first real use of it.

Found by running it rather than by reading it: the unit tests covered the
conventions I had thought of, and passed throughout.

Only the first line or two, only when the line is a comment holding one
token that looks like a path. "// this reads config.json at startup" is a
sentence, "// Package main implements the server." is a doc comment, and a
path comment further down the file is just a comment. The markers cover the
languages people actually ask for, including block comments closed on the
same line.

The comment stays in the body. People write a path at the top of a file
anyway, and editing model output to make a filename look tidier is a
surprise nobody asked for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:12:00 +02:00
Ilker ManapandClaude Opus 5 45d0dee470 Let a reply full of code be downloaded as a zip
CI / test (push) Successful in 1m25s
Asking a coding model for anything real produces a small tree, and the only
way to get it out of the page was to select each block and paste it
somewhere. A reply containing fenced code now offers a zip, with the
directory structure the model described.

Server-side, because the interesting part is the filenames. They are written
by a language model and they go into an archive that somebody extracts with
a tool that trusts archives, so "../../.ssh/authorized_keys" and
"/etc/cron.d/x" are inputs to expect rather than hope against. archive/zip
and a sanitiser with tests beat string handling in a browser. An absolute
path is made relative — it cannot escape once the leading separator is gone,
and refusing a file over a leading slash throws away work. Anything that
still escapes after cleaning is rejected rather than repaired, because
silently turning ../../evil.go into evil.go puts a file somewhere the reader
did not expect; it gets a generated name instead.

Naming follows what models actually do: on the fence (```go:cmd/main.go,
```main.go, ```go main.go, title="x"), or on the line above it as a heading
or comment. Only the two lines immediately above, so a sentence mentioning
main.go is not read as a filename. A block nobody named still comes back
under a generated name, because losing code because it was unlabelled is
worse than naming it badly, and two blocks claiming one path both survive —
models show a file and then show it again with one line changed.

Nothing is stored: the reply arrives, the zip goes back, neither is kept.
The endpoint sits behind the same session guard and CSRF check as
/chat/send.

Two of these tests asserted the wrong thing when first written — that an
absolute path must be refused, and that hyphens are stripped from the
archive name. Both were corrected to what the code should do rather than
the code changed to match them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:06:07 +02:00
Ilker ManapandClaude Opus 5 0a98d01361 Say when a chat reply was cut off, and stop cutting it so early
CI / test (push) Successful in 1m20s
Reported as the model not completing a request and stopping. It did
complete — it ran out of room. The chat page caps a turn at max_tokens and
then never looked at finish_reason, so a reply that hit the cap ended
mid-word and looked exactly like one that was finished.

The silence was the real defect. A reader has no way to tell "the model had
nothing more to say" from "the page stopped it", and the only clue is that
the last sentence ends abruptly.

512 was also too low. It was chosen when every model here was a 0.5B
answering in a paragraph; it is about a screenful of code, so a coding
model gets cut off inside the first function it writes. 2048 now, which is
still a bound — the reason for having one has not changed.

The notice is appended beside the reply, never into it: that element holds
model output placed with textContent, and building markup in the same node
is how the careful handling everywhere else gets undone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 21:03:07 +02:00
Ilker ManapandClaude Opus 5 082d52bc97 Stop shipping a placeholder that looks like a value, and say what is served
CI / test (push) Successful in 1m23s
A user copied the curl from the profile page and sent it as it stood. Every
other field in that snippet is filled in for them — the base URL, the key
they had just been issued — so "model": "MODEL" reads as a value rather
than a blank, and they got:

  no endpoint serves "MODEL"

which is true and useless. It reads the same whether the name was a typo,
a placeholder left in from the documentation, or a real outage, and it
never says what would have worked.

The message now lists what is being served, capped at ten with a count for
the rest, and says plainly when nothing is. That list is what /v1/models
already returns to the same caller, so it discloses nothing new.

The profile page's example now names a model that is actually running,
real ones before simulated — the point of an example someone can run is
that it runs, and sending a first request to the sim engine would answer
in Greek letters. The landing and claim pages keep a placeholder, because
they are served without a key and deliberately say nothing about what is
here; theirs is <model-name>, which cannot be mistaken for a value, with a
line saying where to get the real one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 18:23:19 +02:00
Ilker ManapandClaude Opus 5 72b03a1a1c Say which model is simulated instead of letting it look broken
CI / test (push) Successful in 1m31s
big-model runs on the sim engine, whose whole vocabulary is the Greek
alphabet, cycled at a fixed rate until max_tokens. That is what it is for:
with no H100s it is the only thing exercising the TP8 two-node path, the
fabric invariant and rebuild-after-node-loss. The output is the point.

What was wrong is that nothing said so. /v1/models returned it with
owned_by "llmgw", identical to the real model, and the chat picker sorted
purely by name — so "big-model" led the list, was preselected by the
browser, and anyone who typed a question and pressed Send got Greek
letters back with no explanation. Reported as the model spitting out crap,
which is a fair reading of what it looks like from outside.

Simulated is derived from the profiles rather than configured: an operator
who adds a sim profile and forgets a flag would be shipping placeholder
text under a real model's name. A model counts as simulated only when
every profile is, because one real profile means the request can still
come back as inference.

The marker on /v1/models is owned_by "llmgw-sim", which is already what
the sim engine calls itself on its own /v1/models — the gateway was
flattening a distinction that existed one layer down.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 17:11:56 +02:00
Ilker ManapandClaude Opus 5 cbf276a41b Render maths in the chat, without letting the model write markup
CI / test (push) Successful in 1m20s
Replies are plain text, so a model asked about an integral answers with
\int_0^1 x^2 dx and the reader gets the source.

KaTeX is vendored into the binary rather than loaded from a CDN. This gateway
makes no outbound requests of its own and is expected to run where it could
not make one; a chat page that silently loses its maths on an air-gapped
cluster would be worse than one that never had it. 600 KB of JS, CSS and WOFF2
fonts, which the stripped binary absorbs without changing its rounded size.

The ordering is the safety argument, and there is a test for it rather than a
comment alone. The reply is put in place with textContent exactly as before,
and only then walked for maths: renderMathInElement replaces the spans it
recognises, so nothing the model wrote is ever parsed as HTML and the markup
that appears is KaTeX's. trust stays false — KaTeX's default, and what keeps
\href and \includegraphics out of a reply — and throwOnError is off so one
malformed expression does not take the rest with it.

Rendered once the reply is complete. Typesetting each chunk would try to set
half an expression on every token and flicker through the whole answer.

The static handler now serves a small embedded asset tree rather than one
file, so it moved to its own embed: a file server over the tree that also
carries the templates would serve those too. Directory listings are refused —
nothing links to one, so the only requests are from somebody looking around.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 17:03:46 +02:00
Ilker ManapandClaude Opus 5 3b67b1fe42 Stop the dashboard linking to pages that are not where it is
CI / test (push) Successful in 1m25s
Clicking "metrics" on the web dashboard answered "no api key was provided".

The page is served twice and its navigation assumed one of them. Metrics,
pprof and the management API exist on the admin listener; account
administration, the profile and the chat exist on the client-facing port. The
nav offered the first set in both places, so on the public port those links
fell through to the API-key middleware — which is a far more confusing way to
say "that is not here" than a 404 would have been.

Each page now offers what is actually reachable from it, and the public one
says where the rest lives rather than pretending it does not exist, including
why that port is not exposed: pprof can dump the process heap, and the heap
holds every API key in plaintext.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:57:58 +02:00
Ilker ManapandClaude Opus 5 de21649ad6 Add an in-browser chat, without putting a key in the browser
CI / test (push) Successful in 1m13s
So someone can try a model without setting up a client, and — more often —
tell whether a problem is the gateway or their code.

The page carries a session, and the server turns that into the identity an API
key would have produced, resolved from the person's project. The request then
goes down the ordinary path by being handed to the same handler /v1 uses:
admission, model allowlist, rate limits, budget, accounting. There is exactly
one request path, so there is no second place for its rules to drift — and
without that, a playground is a hole in every limit the rest of the gateway
enforces. The allowlist test posts a forbidden model and expects 403 for that
reason.

The body is rebuilt rather than forwarded, so the page cannot set fields it has
no business setting: a stream:false that would hang the reader, a max_tokens
large enough to occupy a replica for an hour. One turn is capped at 512 tokens.

The picker offers what is actually served and allowed, not what is catalogued,
because offering a model with no replica produces a 503 the reader cannot
interpret.

Inline vanilla JavaScript, for the same reason the charts are hand-drawn SVG.
It streams, because on CPU inference ten seconds is normal and a chat that
hangs and then dumps a paragraph feels broken. Replies are written with
textContent throughout: model output and user input have no business being
parsed as markup. Its POSTs carry the CSRF token in a header, which a
cross-site form cannot set at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 12:05:30 +02:00
Ilker ManapandClaude Opus 5 859f08da9b Approve accounts from the browser, and draw the traffic
CI / test (push) Successful in 1m19s
Two things the dashboard was missing: it could not act, and it was a column of
numbers.

Account administration moves to /admin/users — approving into a project,
suspending, reinstating. That is the work an operator does constantly and it
should not need a terminal. Three things stay on the CLI, and the line is not
arbitrary: promoting someone to admin, resetting a password, and issuing a key
on another person's behalf each turn a stolen admin session into more than it
was — permanent access, or somebody else's account. Approving and suspending
do not. Suspending signs the account out everywhere, or suspension is advisory
until the session happens to expire, and an admin cannot suspend themselves,
which is not a thing an interface should let you do by misclicking.

The charts are inline SVG generated in Go. A charting library would mean a
JavaScript toolchain in CI and a megabyte of bundle for what is a list of
rectangles, and the single-binary goal rules that out. Hourly over a day: fine
enough to show a working day, coarse enough that one slow minute is not an
incident. Empty hours are filled in, because a chart with hours missing where
nothing happened reads as a chart with hours missing from the data.

The SVG is marked as trusted HTML, so everything that reaches it is either a
number this package computed or escaped explicitly, and a bar's colour can only
be one of four names — a colour that could be anything is a way to write CSS
into every page the chart appears on. There are tests for both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 12:01:42 +02:00
Ilker ManapandClaude Opus 5 e519dc587c Serve the cluster dashboard over the web, without serving pprof with it
CI / test (push) Successful in 1m21s
Reaching it meant an SSH tunnel, because the page lived on the admin listener
and that listener is bound to loopback. Opening the port was never the way to
fix that: it also serves pprof, and pprof can dump the process heap, where
every API key sits in plaintext. Exposing the port would have been exposing
the key store.

So the page moves rather than the port. The same dashboard is now mounted at
/admin on the client-facing port, behind the session and CSRF machinery that
already guards /profile, gated on a flag on the account. The admin listener
keeps pprof and metrics and stays on loopback.

One bit, not a role system: it grants looking. Everything that changes the
cluster or issues a credential stays on the CLI, where it needs a shell rather
than a session — which keeps the worst case of a stolen admin cookie to "saw
the cluster" rather than "drained your fleet".

Someone without the flag gets 404, not 403. A 403 confirms the page is there
and that admin accounts exist here, which is free reconnaissance; 404 is what
every other unknown path returns. The profile page does not show a link to a
door the visitor cannot open.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 10:46:45 +02:00
Ilker ManapandClaude Opus 5 79e459d9ff Let people change their own password, and operators reset a lost one
CI / test (push) Failing after 1m11s
The account system shipped without either, which made the first forgotten
password an unanswerable support request.

Changing it requires the current one. A session is what an attacker gets from
a borrowed laptop or a stolen cookie; the password is what they do not have,
and without the check the first thing they would do is take the account.

Every session is closed, including the browser doing the changing — leaving
the others open would mean a password change does nothing about the reason
someone is changing it. A new session is then opened for the browser in front
of us, so the person who just proved their password is not thrown back to the
sign-in page. That response also carries a new CSRF cookie, so the forms it
renders take the new token rather than the one the request arrived with; there
is a test that posts a form from that exact page.

Guessing the current password is bounded the same way sign-in is, because
someone holding a stolen session should not get unlimited attempts at the
password that would let them keep it.

There is no self-service reset: this gateway sends no email, and a reset link
with nowhere to send it is not a reset. `llmgw users reset-password` is the
operator's answer, and it closes their sessions too — if someone lost their
password, the reason might be that somebody else has it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 09:39:11 +02:00
Ilker ManapandClaude Opus 5 216fc9bb03 Let people sign up, be approved, and issue their own keys
CI / test (push) Successful in 1m9s
Every key needed an operator with a shell. This adds the login system that
made that necessary, which is the thing I argued against building until it was
asked for — so it is built as if it will be attacked, because a public login
form is.

Signing up produces an account that can do nothing. Approval is what makes it
work and stays on the CLI: having a shell on the control plane is what makes
someone an administrator, and that is a smaller thing to get right than roles
and an admin UI that mints credentials.

The details that matter:

Passwords are argon2id, where a slow hash is correct — the input is chosen by
a human and has a dictionary, which is the opposite of an API key. Both kinds
of authentication now live in this codebase and they are in separate packages
so neither set of assumptions leaks into the other. Length is the only rule:
composition rules push people towards Password1! and cost more entropy than
they buy.

Signup never says whether an address already has an account, and sign-in
answers the same way for a wrong password and a missing account — verifying
against a dummy hash when there is no user, so a stopwatch cannot enumerate
accounts. Account state is checked after the password, not before, so guessing
an address reveals nothing about it. Guessing is bounded per address rather
than per IP: an attacker has many addresses to come from and one to guess
against, and limiting by IP locks out an office behind one NAT.

Sessions are server-side, so signing out and suspending take effect
immediately rather than whenever a token expires; suspension is checked in the
same query that resolves the session, because a second round trip is a check
somebody eventually forgets. Every state-changing form carries a CSRF token
verified against a hash in the session — a cookie authenticates the browser,
not the page that asked.

Key ownership is enforced in the UPDATE rather than by reading first, so a
guessed key id from another account changes nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 09:32:35 +02:00
Ilker ManapandClaude Opus 5 8d35faf6f8 Give the landing page somewhere to send people
CI / test (push) Successful in 56s
The page explained the API, said keys come from an operator, and left the
reader there. Someone who reaches the address without a key had no route to
getting one, which is where onboarding stopped.

`landing.contact` is that route, because the gateway cannot guess who to ask.
An address becomes a mailto link, an http(s) URL stays a link, and anything
else — a channel, a queue, a person — is shown as text. Other schemes are
never linked: this string comes from configuration, configuration is not
always written by whoever runs the gateway, and a javascript: href on a page
served to every visitor is a short path to a stored XSS. Escaping the text
does not make a scheme safe.

Unconfigured, it now says something rather than nothing, and the page explains
that a key arrives as a one-time link so the reader knows what to expect
before it lands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 09:22:42 +02:00
Ilker ManapandClaude Opus 5 3ed211b2f4 Onboard users with one-time links instead of pasted keys
CI / test (push) Successful in 2m43s
Issuing a key worked; getting it to the person it was issued for did not. The
operator had a secret on their terminal and no way to send it that did not
leave a bearer credential in a chat log forever.

`keys create --claim` prints a link instead. GET shows a button and consumes
nothing; POST reveals the key and burns the link. That split is not politeness
— chat clients, mail scanners and link previewers fetch URLs the moment they
are posted, so a GET that consumed the claim would mean the first thing to
unfurl the link destroys the key before anyone reads it.

A pending claim has to hold the key somewhere, which would otherwise put a
working credential in the database and undo what hashing bought. So the key is
sealed with a key derived from the link's own token, and the token is never
stored — only its hash. A database leak yields two useless hashes and a blob
nothing in the database can open. HKDF rather than the raw token, so the value
used for lookup can never become the value that decrypts.

Consuming is one UPDATE moving claimed_at off NULL, so two people opening the
same link produce one key rather than two copies. A used link, an expired one
and a token that was never real all render identically, because saying which
confirms that a token was once valid. The page is no-store, no-referrer and
noindex: the URL is itself the credential.

`GET /me` reports a key's own limits and spend, so nobody has to ask an
operator to read a dashboard they cannot reach. Outside /v1, because it is not
part of the OpenAI API.

docs/onboarding.md is written for whoever operates this, including the parts
that are deliberately absent — no self-service, no SSO, and why.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 09:11:28 +02:00
Ilker ManapandClaude Opus 5 1839832b8b Add a landing page and a read-only cluster dashboard
CI / test (push) Successful in 1m2s
Pasting the gateway's address into a browser returned 404, which looks like a
broken endpoint rather than one that tells you how to use it.

Two pages, and the split between them is a security boundary rather than a
layout choice. The landing page is served without a key — it has to be,
because the person reading it is the one who does not have one yet — so it
contains no data at all: it says how to connect, not what is here. Model names
stay behind /v1/models. It reconstructs the address from the forwarded headers
so the copy-pasteable example names the address the client actually used
rather than the internal one; those headers are reflected into displayed text
and never used for a decision.

The dashboard shows nodes, models, replicas, placement failures and usage, and
lives on the admin listener only. That listener is loopback-bound and also
serves pprof, which can dump the process heap — where API keys sit in
plaintext — so reaching the page from elsewhere is an SSH tunnel's job. It is
read-only: anything that mints a credential stays in `llmgw keys`, where it
needs a shell rather than a browser session this project has no way to
authenticate.

Mounting the landing page uncovered a hole in the exemption list. An entry
ending in "/" was a prefix rule, and "/" ends in "/", so exempting the landing
page would have exempted every path there is — authentication still installed,
still logging, and letting the whole API through. "/" is now exact, with a
test enumerating what must and must not be exempt.

Server-rendered templates and one stylesheet, embedded. The single-binary goal
rules out a JavaScript toolchain and nothing on either page needs one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 08:58:05 +02:00