Commit Graph
240 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 4dba2a54ea Tighten the People page
An account's "add to" picker and its button sit on one line; suspend
and remove, the two things done least and the two drawn in red on every
row, are in a menu behind "⋯"; and the box for a new project under
every team, and for a new team under every organization, is folded
behind "+ project" and "+ team" until wanted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:18:38 +02:00
Ilker ManapandClaude Opus 5.5 ac7f5063b6 Give every choice-link one pill, and stop the NVLink page breaking them
The usage and organization pages drew their choices as filled green
chips, the newer pages as outlined pills. They are all the outlined
pill now.

The NVLink page's GPU squares were styled as .chip, the class those
choices already used, which shrank the usage page's filters into
1.7rem squares wherever the stylesheet loaded. They are .gpu-chip.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:15:44 +02:00
Ilker ManapandClaude Opus 5.5 45e3f52da3 Launch multi-node vLLM its own way by default, not through Ray
An empty launch_mode meant ray: a Ray head and joins on every node, a
dependency to install, and a path never checked against a real cluster.
It now means mp - every node runs vllm serve with its rank and the
leader's address, the workers headless - which is how vLLM documents a
multi-node launch today and needs nothing but vLLM. ray is still there
for a vLLM too old for mp's flags. The example agent configuration,
doctor and the reference say so.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:13:59 +02:00
Ilker ManapandClaude Opus 5.5 20479a2d5f Find multi-node NVLink domains, place across them, and show them as racks
A GB200 NVL72 rack or an NVLink Switch system is a scale-up island: its
GPUs reach each other over NVLink whatever node they are in. Each GPU
now reports its fabric from nvidia-smi -q - state, status, ClusterUUID,
CliqueId, bandwidth - at registration and once a minute after, since a
fabric manager restarting drops every card out at once; and each node
reports nvidia-imex and its channels, without which the domain's GPUs
cannot share memory across nodes. A single machine's NVSwitch, which
reports a cluster of zeros, is no domain.

A replica spanning nodes is placed inside one domain first, on nodes
whose every card is joined and whose IMEX is up, and its shards are
told NCCL_MNNVL_ENABLE=1; InfiniBand is the fallback.

/admin/nvlink draws each domain as a rack - the NVLink Switch band on
top, light running down a backplane, trays of GPU chips that glow when
joined, pulse red when not, turn amber on degraded bandwidth - and
lists what keeps a tray out. A node's page shows its GPU-by-GPU NVLink
matrix, each card's NUMA node and nearest adapter, and its domain and
IMEX. doctor checks joining, bandwidth, IMEX and its channel; metrics
llmgw_gpu_nvlink_joined and llmgw_imex_up; /api/v1/nvlink.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:07:59 +02:00
Ilker ManapandClaude Opus 5.5 5bbc619a31 Map every InfiniBand fabric, draw it, and place replicas by it
The control plane asks one node on each physical fabric to run
ibnetdiscover every five minutes and parses the text itself, so a
format the parser has not met is a control plane upgrade, not one on
every node. From the map it works out the switch tiers (leaf, spine,
core, by distance from the adapters), which adapter is which node's,
which cables trained below the rate the rest of their kind run at, and
whether the rails line up: every node's Nth adapter on its own leaf.

A page per fabric (/admin/fabrics) draws it: switches in their tiers,
adapters fanned out under their leaves, cables coloured by rate and
bundled, degraded ones marching red with a marker, and the cluster's
own adapters live - down ports glowing red, and particles running each
cable as fast as its adapter is sending and receiving, from the
counters. It zooms, pans, fades what a hovered switch or adapter does
not touch, opens a panel of detail on a click, colours by rail, finds a
node, and refreshes every fifteen seconds. The data is also at
/api/v1/fabrics/{id}/topology.

Placement reads the map: a replica spanning nodes takes nodes of one
leaf together when it can, and nodes whose cable trained short last.

Static assets are named by their content, as the stylesheet already
was.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 08:46:23 +02:00
Ilker ManapandClaude Opus 5.5 23648fa19a Watch each fabric's subnet managers, and keep a fabric one through a failover
A fabric's identity was its master subnet manager, which changes on a
failover: for the minute its ports took to agree, one fabric read as
two, and a name covering it as two separate fabrics. Each agent now
also lists every subnet manager on a fabric (saquery SMIR, through one
port per master, once a minute); managers listed together are one
fabric, named by the lowest GUID among them, so the identity survives
the master changing.

The same list says what a fabric can survive. The cluster page's new
subnet manager table, /api/v1/fabrics, doctor and metrics
(llmgw_fabric_master_sm, llmgw_fabric_subnet_managers,
llmgw_fabric_problems) show each fabric's master, priority and
standbys, and say when a fabric has no standby, its ports disagree on
the master, ports are up with no subnet manager (INIT), or the master
changed in the last hour. Every change of master a port reports is
kept, and a fabric's last day of them is listed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 07:47:03 +02:00
Ilker ManapandClaude Opus 5.5 77fe801892 Tell independent fabrics apart by their subnet manager, not their name
A fabric was what agent.yaml called it. With several independent
InfiniBand fabrics at a site, two given one name would have had a
replica spread across both, starting and then hanging on its first
collective. Each active InfiniBand port now reports the master subnet
manager it sees (sminfo, asked once a minute), which every port on one
fabric shares and ports on separate fabrics cannot; a RoCE port reports
the IP network of its RoCE v2 address. Ports also carry their node and
port GUIDs.

Placement divides a declared fabric by that physical identity and forms
a replica on one part only; a node whose adapters for one name are on
two fabrics is refused, and a node that cannot say is kept apart from
those that can. /api/v1/fabrics and the cluster page list each physical
fabric with its subnet manager, and say when one name covers two
fabrics, two names one fabric, an adapter is up but undeclared, or a
node cannot be checked. doctor reports each port's subnet manager, and
an agent on a node with adapters installs infiniband-diags when sminfo
is missing.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 07:29:16 +02:00
Ilker ManapandClaude Opus 5.5 51f1606b53 Resolve the management address from the servers list too
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 05:07:15 +02:00
Ilker ManapandClaude Opus 5.5 1b1ca67e35 Add llmgw doctor: what a node's GPUs, NVLink and fabric need, checked
For machines nobody here can log into. It reads the GPUs and their
nvidia-smi topology, NVSwitch and the fabric manager, every InfiniBand
and RoCE port with its counters and RoCE v2 GID, the RDMA and
GPUDirect kernel modules, the agent's declared fabrics against the
adapters, the management interface NCCL will bootstrap over, the
agent unit's memlock limit and the engines installed - and says what
is wrong and how to fix it, most serious first, with the NCCL
environment a shard on each fabric would be given. --json writes every
fact it concluded from, to send back. It exits non-zero when something
fails.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 03:51:09 +02:00
Ilker ManapandClaude Opus 5.5 4d9284ce9f Find each node's management interface, and lift the agent's memlock limit
The agent's defaults said eth0 before normalize looked for the interface
holding the management address, so it never looked: a node on ens3 or
bond0 had NCCL and Gloo bootstrap over an interface it does not have. It
is now found, with eth0 only as the last resort.

The agent unit sets LimitMEMLOCK=infinity. RDMA pins what it registers,
and under systemd's 8 MB default an engine without the capability that
lets root ignore the limit fails its first collective over InfiniBand.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 01:44:59 +02:00
Ilker ManapandClaude Opus 5.5 3ba36e23a6 Choose a shard's GPUs by NVLink, nearness to the fabric and NUMA
GPUs were chosen by growing one card's NVLink peers, which took cards
each joined to that one but not to each other, and knew nothing of the
fabric. Every set of the right size is now scored - a node's handful of
cards makes that cheap - by NVLink pairs within it, then for a
multi-node shard by how near each card is to one of the fabric's
adapters (PIX before SYS), then by NUMA nodes spanned, with the lowest
indices breaking ties. A node without topology still gets the lowest
indices.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 00:19:52 +02:00
Ilker ManapandClaude Opus 5.5 9ee21dc814 Fix what a multi-node launch would have got wrong on real hardware
vLLM's mp launch gave workers no rank and the leader's address as their
own VLLM_HOST_IP; every node now runs serve with --nnodes, --node-rank,
--master-addr and --master-port, and the workers --headless, as vLLM
documents it.

NCCL_SOCKET_IFNAME and GLOO_SOCKET_IFNAME were the control plane's one
interface name for every node; the agent already worked out its own -
the one holding its management address - and now uses it.

On RoCE, NCCL's default GID index is a v1 link-local entry on most hosts,
which does not route. The agent finds the RoCE v2 entry holding an IPv4
address and passes it as NCCL_IB_GID_INDEX; InfiniBand gets none.

A worker whose only process needs the leader - a Ray join - started
without waiting for it, since the wait was honoured for sidecars alone;
it found no head, exited and failed the replica. It now waits.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 00:17:31 +02:00
Ilker ManapandClaude Opus 5.5 93bd16f00e Watch InfiniBand ports while the agent runs, not only when it starts
A port's state was read once, at registration: one that went down later
left its node on the fabric, offered multi-node work that would hang on
its first collective, until somebody restarted the agent. The agent now
re-reads every IB and RoCE port on each heartbeat - state, physical
state, rate and the error and data counters - and when the fabrics it
can carry traffic on change, says so once; the control plane replaces
the node's fabrics, which is what placement reads.

The ports are exported as llmgw_ib_port_active (with state and
phys_state, so an unplugged cable reads differently from a missing
subnet manager), llmgw_ib_port_rate_gbps and llmgw_ib_port_counter_total,
and a node's page lists them with the fabric each carries and any error
counter that is not zero.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 00:13:54 +02:00
Ilker ManapandClaude Opus 5.5 bf2d171e89 Read NVLink and GPU-to-adapter distance from nvidia-smi topo
The release binary is built without cgo, so without NVML, and NVML was
the only source of NVLink peers: on every NVLink machine the portable
agent reported none, and tensor-parallel groups were chosen by GPU index.
Discovery now reads `nvidia-smi topo -m` after either path, for the
NVLink peers (kept from NVML where it gave them), each card's NUMA node,
and how far each network adapter is from it - PIX, PXB, PHB, NODE or
SYS - which travels to the control plane as nic_paths for placement to
use. Both the NICn-with-legend layout of newer drivers and the
adapter-named columns of older ones are read.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 00:09:26 +02:00
Ilker ManapandClaude Opus 5.5 22bb098247 Group the mirror's table by model family, not by exact name
Grouped by name, the four qwens in the mirror were four groups of one
and the table looked unchanged. A family is the letters a name starts
with - qwen2.5-coder-7b and qwen3-8b are qwen, llama-3.1-8b is llama -
and a family of two or more models gets a heading with how many and how
much disk; a family of one stays a plain row.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:44:55 +02:00
Ilker ManapandClaude Opus 5.5 06ea02dd57 Group what the mirror holds by model
Each model is a heading - how many versions, how much disk, and whether
one is downloading, failed or in use - with its versions beneath it.
Sorted by state alone, versions of one model were scattered down the
table around unrelated downloads. A model with a download or a failure
still comes first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:39:44 +02:00
Ilker ManapandClaude Opus 5.5 7c5938effe Keep node names on one line in the node table
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:31:36 +02:00
Ilker ManapandClaude Opus 5.5 9ae526302b Label chart axes in binary sizes and small percents, and windows in words
A bytes axis read "20.0BB" - compact's billion with a B after it - and
a quiet node's CPU axis read 0% all the way up. Sizes are now binary like
every other on these pages, a percent under ten keeps a decimal, and a
window of an hour is "the last hour" rather than "1h0m0s".

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:30:01 +02:00
Ilker ManapandClaude Opus 5.5 fe0875ee05 Say why an engine's memory can exceed its node's RAM in use
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:26:54 +02:00
Ilker ManapandClaude Opus 5.5 daed835d88 Measure each node's CPU and memory, and give every node a page
The nodes are CPU machines, and nothing measured how busy one was: the
heartbeat carried GPU memory and disk, and llmgw_node_memory_bytes is the
total. The agent now reads /proc between heartbeats - CPU across all
cores, load averages, memory used and available, and each engine's
process group's cores and resident memory - and sends it with the
heartbeat. Nothing new runs on the nodes and Prometheus needs no new
target: the control plane exports it with the rest of the fleet, as
llmgw_node_cpu_utilization_percent, llmgw_node_cpus, llmgw_node_load,
llmgw_node_memory_used_bytes, llmgw_node_memory_available_bytes,
llmgw_shard_cpu_cores and llmgw_shard_memory_bytes. A measurement more
than a minute old is left out rather than repeated.

The cluster page's node table shows CPU and RAM as bars, and each node
links to a page of its own: current CPU, RAM, load and disk, the engines
on it with their cores and memory, and an hour of CPU, RAM, load,
per-model cores and memory and queue depth from Prometheus. It refreshes
in place.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 23:21:23 +02:00
Ilker ManapandClaude Opus 5.5 b06fb9f51f Lead the cluster page with its health and refresh it in place
Capacity, nodes, models and replicas come first, then a rule, then
traffic, people, what is being run and usage. The people and per-model
bar charts, which repeated the tables under them, are gone.

The page swaps its content every few seconds with htmx instead of
reloading, so a reader keeps their place, and it stops while the tab is
hidden. Without script the meta refresh still reloads it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:56:27 +02:00
Ilker ManapandClaude Opus 5.5 1f27935570 gofmt the account page's handler
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:52:32 +02:00
Ilker ManapandClaude Opus 5.5 16a1e8b4d8 Show a person their own use and what is left of their budgets
The account page (was "Your keys") shows the last thirty days of the
person's own use - requests, tokens, spend and tokens per day - and, for
each place they can spend, what they used there and a bar for every
budget that applies: their own, a per-member one, or a project's, team's
or organization's shared with others, named as such. Bars turn amber past
four-fifths and red when spent. It sits under the keys now, above the
password form.

Spend is read from the usage table for the pool each limit counts
against, from the start of its period.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:51:13 +02:00
Ilker ManapandClaude Opus 5.5 6b1f8b0827 Render a finished chat reply as markdown
A reply streams as text as before; once it ends, the server renders it
with the model card's rules - raw HTML dropped, dangerous links emptied,
images as links - and the page places that. Headings, lists, tables and
code blocks read as they were written.

Maths outside code is set aside before the markdown, so its underscores
and asterisks are not taken for emphasis, and marked for the page to
typeset. KaTeX no longer searches the reply for delimiters, which took
"$5 and $10" for a formula. A reply that cannot be rendered stays
plain text, with the old search.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:44:18 +02:00
Ilker ManapandClaude Opus 5.5 9b663eb23c Name people and projects in usage reports, and total a usage legend
The cluster page's usage table showed session:usr-... and proj-... ids;
it now shows the person's email, whether it came through a key or the
chat page, and the project's path or "personal". Times read "3 hours
ago" rather than "3h7m25s ago", everywhere.

The usage page's legend gave each line's latest slice, which was 0
beside a table of hundreds; it gives the window's total, and the table's
numbers are right-aligned like everywhere else.

The PostgreSQL placeholder rewriting skips line comments. An apostrophe
in one opened a string that swallowed every placeholder after it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:37:21 +02:00
Ilker ManapandClaude Opus 5.5 ff86e52126 Open a repository beneath its row in the mirror's listing
Clicking a repository in the explorer shows its card and files in a row
beneath it, fetched from /admin/mirror/repo, and a second click closes
it; the list stays where it was. The link still leads to the
repository's own page without script or with a modifier key.

A repository's files now take one line each, and an importance matrix
published beside the quantisations is marked as not a model rather than
offered for fetching.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:11:38 +02:00
Ilker ManapandClaude Opus 5.5 3fea39feb9 Show a Hugging Face repository's model card on the mirror page
Opening a repository shows its model card, rendered from markdown, with
its base model, license and languages above it; the listing names each
repository's base model and license under its name. The card is a
stranger's text: raw HTML and dangerous links are dropped, images are
shown as links rather than loaded, and relative links point into the
repository.

Adds github.com/yuin/goldmark for the markdown.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 22:03:19 +02:00
Ilker ManapandClaude Opus 5.5 02fa639657 Open the mirror's registry on what is current, not an empty search box
The mirror page lists Hugging Face's trending models without anything
typed, with views for what is new this month and what is most
downloaded, filters for task and size, and each repository's parameter
count, architecture, context length and age from one registry call.
"Only what these nodes can run" is on by default and leaves out
architectures the nodes' llama.cpp cannot load, and adapters, saying how
many it hid. Listings are remembered for ten minutes.

The stylesheet is now linked by a hash of its content. It is cached for
a day, so under a fixed name a deploy's new styles reached people a day
after its markup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 21:56:58 +02:00
Ilker ManapandClaude Opus 5.5 5aa5675a9c Show a model loading in the chat, and send once it is up
A message to a model that was not running waited 45 seconds and came
back as "the model is loading; retry shortly". The page now asks for
the model with /chat/wake, shows where it stands from
/chat/model-status - waiting for a node, downloading with bytes and a
percentage, starting the engine - and sends the message when it answers.

Also passes the nodes' llama.cpp build to the pages, which the mirror's
checks for loadable weights needed and were never given.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 20:52:31 +02:00
Ilker ManapandClaude Opus 5.5 f13b18149a Refuse weights the nodes' llama.cpp cannot load, before fetching them
kev-0.5b declared architecture "ggmlc", which llama.cpp does not know.
It was fetched, catalogued and placed, and failed on every node — while
the answer was in the first kilobytes of the file.

internal/gguf reads a GGUF header (v1-v3) up to general.architecture,
over HTTP with a Range request, and judges it against the architectures
llama.cpp b10991 lists in LLM_ARCH_NAMES (152, copied from the source at
that tag); a vision projector is not a model on its own. An unknown
build is not refused, only not vouched for, and a header that cannot be
fetched does not refuse anything.

Checked in three places: the registry listing (from the architecture
Hugging Face reports with expand[]=gguf, so one request per repository)
shows it and withholds the fetch button; a fetch reads the header and
refuses unless "fetch anyway"; and the model service refuses to add
such a model, which covers the serve form and ctl model add.

Tried on real files: the mirror's qwen2.5-1.5b (qwen2) and bge-m3
(bert), and Qwen2.5-0.5B on Hugging Face through its redirect.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 20:44:36 +02:00
Ilker ManapandClaude Opus 5.5 c609805e4b Line up how each mirrored model runs
The run mode sat after a tag repeating the model's name, so each selector started where the name ended and long names wrapped the row. The column is now Runs: one fixed-width selector per model, with the catalog's name above it only when it differs from the mirror's. Sizes no longer wrap.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 20:44:36 +02:00
Ilker ManapandClaude Opus 5.5 bf94cd1667 Set how a model runs from the mirror page
Every mirrored version was in use on the cluster, so the serve link — hidden for a version already served — appeared nowhere, and changing a model between always on and on demand needed the CLI. Each model using a file now shows how it runs and offers always on, on demand (15 minutes idle) or stopped, through the model service's Scale; models declared in server.yaml show their mode and where to change it. The serve form asks the same question.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 20:34:41 +02:00
Ilker ManapandClaude Opus 5.5 d8cefd8b21 Say there are no organizations rather than 404
With every organization removed, /org answered an administrator with a 404 while the menu kept offering it. An administrator now gets a page saying there are none and where to make one, and the menu offers Organizations only when there is one to show (to a manager, one they manage).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 20:23:19 +02:00
Ilker ManapandClaude Opus 5.5 e493b2f53d Give an organization's managers a page of their own
A member of an organization can be its manager (users join --org X
--role manager, or the star on the people page). /org shows a manager
their organization: usage over time by team, project or person, its
teams and projects with limits and 30-day spend, and its members, each
once with every place they belong. Read-only; administrators see every
organization. Only what was spent in the organization's projects is
shown — UsageBucketsIn filters by where a request was spent.

Found on the way:
- Usage by org, team and project, and the quotas page's spend, joined
  through api_keys and so dropped every chat-page request, which has no
  key row. They now read usage.project_id; keys no longer move between
  projects, so the two agree.
- /org was not exempt from API-key auth and answered browsers with a
  JSON error. A router test now drives the session pages through the
  middleware; it fails without the fix.

Membership queries, org-scoped usage and the reworked joins pass the
PostgreSQL store tests on 16.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 19:23:12 +02:00
Ilker ManapandClaude Opus 5.5 098886e0e4 Let a person belong to many places, or to none
A person was in exactly one project, chosen at approval, and could not
use the gateway before that. People belong to several teams and to
none, so:

- memberships: a person is a member of any number of organizations,
  teams and projects, or of none; a member of a team or organization
  may use every project in it. users join/leave, and the people page
  adds and takes out from either side. Approval no longer needs a
  project.
- personal keys: a key may have an owner and no project. /profile asks
  what a new key spends against; keys create --for does the same.
  auth.personal sets default limits for personal use or turns it off.
- limits for people: --scope user holds a person across every key,
  project and the chat page; --per-member on an org, team or project
  gives each member their own share there. Resolution folds the tree,
  each per-member limit for the key's owner, and the owner's own, each
  in its own pool; the quotas page shows and sets both.
- the chat page asks what to spend against and checks it per message.
- usage records who spent it, so a person's budget survives a restart,
  personal use and chat included; restore no longer drops rows with no
  project.

Leaving a project revokes the keys made there unless another membership
still reaches it. A personal key keeps no threads or notes, which are
per project, and is refused rather than dropped into the anonymous pool.

Migration 0010 turns each person's project into a membership, adds
usage.user_id and backfills it, and makes api_keys.project_id nullable.
SQLite cannot drop NOT NULL, so the table is rebuilt; that needs foreign
keys off, which only works outside a transaction, so migrations marked
for it run on one connection with foreign_key_check before commit.
Tested on a populated SQLite store with claims, and on a copy of the
cluster's PostgreSQL store: 5 people became members, 17 keys and 3
claims intact, 145 of 577 usage rows and all 69 chat rows attributed.
The PostgreSQL store tests, including new membership cases, pass on 16.

users move is gone; join and leave replace it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 19:03:46 +02:00
Ilker ManapandClaude Opus 5.5 429c05a319 Show on the quotas page the limits that actually apply
A branch with no limit of its own read "—" in every column while its
organization held it to sixty requests a minute. Each cell now shows
the strictest value from above, muted and marked with an arrow, naming
its source on hover and in the form's placeholders. A larger value set
under a stricter one says what caps it, for rate and token limits as
well as budgets; only budgets were checked before.

The table follows the people page: indented tree, tinted organizations,
spend with a meter that turns red past 90%, a form that opens and saves
without reloading (htmx, falling back to plain links), and the
explanations folded under "How limits combine".

Rows are built from the store's Tree, so a project sits under its team
by id. Matching the start of its path put projects under the wrong team
when two organizations shared a name.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 18:19:49 +02:00
Ilker ManapandClaude Opus 5.5 be9f45aa94 Write the bar chart's latency the way its table does
The ranked bars said "8825 ms" beside a table that said "8.8 s". One formatter, humanLatency, now serves both, with a test.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 18:11:27 +02:00
Ilker ManapandClaude Opus 5.5 c1926be869 Tidy the dashboard and the forms around it
- Numeric column headers are right-aligned like their values; the
  stylesheet only aligned the cells, so "REQUESTS" sat nowhere near its
  numbers in every table that has them.
- Ranked bars are an HTML grid instead of a fixed-width SVG: the value
  had 96 units and "34.2k tok  114 req" did not fit, so the figure a
  reader came for was the one cut off. Labels ellipsise in CSS and give
  their full text on hover.
- Mean latency reads as 5 ms, 8.8 s or 1m 51s rather than 110695 ms.
- The "What the cluster is doing" charts sat between the Models heading
  and the models table; they now come before it.
- A fleet without GPUs shows its CPU capacity — nodes ready, memory,
  disk free for weights — instead of "No GPUs have been reported", and
  a node with no GPUs shows "—" rather than 0/0.
- The profile's forms are a form's width rather than the page's, and
  the quotas page's price warning is a boxed banner rather than a line
  of red text.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 18:08:16 +02:00
Ilker ManapandClaude Opus 5.5 5e1494f76d Give the menu the new token after a password change
The profile page rendered after a password change carries the CSRF
token set on that same response, but the menu's sign-out form was left
without one — and it is the first form on the page. The e2e test that
signs in, changes the password and uses the next form caught it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 17:57:18 +02:00
Ilker ManapandClaude Opus 5.5 c9cd25efc2 Draw one menu on every page
Each page wrote its own nav: the same links in a different order, some
missing, the current page sometimes listed and sometimes not. A shared
"nav" template now draws it everywhere — Cluster, People, Quotas, Usage,
Mirror for an administrator, then Chat, Your keys and Sign out — with
the current page marked by aria-current. Pages rendered outside the
account pages (the dashboard, node logs) get it through the request
context their admin wrapper sets. The admin listener's own page keeps
its metrics/pprof/api links.

The chat and key pages said "project proj-3kq7…"; they now show the
project's path, and the key page no longer repeats the email.

A test renders several admin pages and requires the same menu on each.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 17:55:36 +02:00
Ilker ManapandClaude Opus 5.5 c22c3a309b Describe v0.9.2
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 16:50:12 +02:00
Ilker ManapandClaude Opus 5.5 42fa602247 Make a new control plane serve a model by itself
server init now also:
- runs a model mirror beside the control plane (/srv/models, bytes on
  :8081 at an address nodes reach, management API on loopback so it
  needs no certificate) and adds it to server.yaml;
- makes the machine a node of its own control plane: issues its agent
  certificate, writes agent.yaml labelled as the control plane, starts
  llmgw-agent, and waits for it to register and install llama.cpp;
- checks for NVIDIA GPUs with nvidia-smi, and reports a card on the PCI
  bus whose driver does not answer rather than ignoring it;
- fetches Qwen2.5 0.5B (Q4_K_M) into the mirror and adds it to the
  catalog pinned to this node: a llama.cpp profile per GPU class found,
  with a CPU profile as the fallback, then waits for it to serve.

--no-mirror, --no-node and --no-model leave these out. Every step looks
before it acts, so a rerun finishes what an interrupted one started.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 16:49:55 +02:00
Ilker ManapandClaude Opus 5.5 5501f77805 Describe v0.9.1
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 16:37:43 +02:00
Ilker ManapandClaude Opus 5.5 75d69a6b9c Show the mirror one line per model version
Each held file took two table rows — seven columns and its address
beneath — so one sharded model filled the screen. A version is now one
line: model and version, the file or "N files", total size, and the
catalog models that use it. The address is behind a "copy address"
button (a set's is its directory, trailing slash included) and the
checksum is in the file's tooltip.

The model service reports which models name each source, so a file in
use shows who uses it and is not offered for serving a second time;
removing one warns that the catalog uses it. Downloads and failures
stay one line per file, since each has its own progress or retry. The
header counts versions, not files, and the nav calls /admin/users
"people" everywhere.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 16:36:36 +02:00
Ilker ManapandClaude Opus 5.5 beae60a76e Describe v0.9.0
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 16:27:54 +02:00
Ilker ManapandClaude Opus 5.5 3104f4b706 Manage people and the tenancy tree, from the web and the CLI
Nothing in the tree could be renamed, moved or removed, and an approved
person stayed in the project they were first put in — from the web or
the CLI. Now:

- /admin/users is the tree, each project opening onto its people and
  keys, with approve, move, suspend, reinstate and remove for people,
  revoke for keys, and add, rename, move and remove at every level,
  plus a search over every account. Forms post and get the page back,
  so it works with no script; htmx 2.0.11 (0BSD, embedded, checked
  against npm's integrity hash) swaps only the changed part, keeping
  open projects open.
- llmgw org tree/add/rename/move/remove, users move/delete, and
  keys list --all.
- Projects, teams and organizations are named by id, path
  (Acme/platform/gateway) or unique name; users by id or email. With a
  single project --project can be left out, instead of defaulting to
  "default", which a tree built from the web does not have.

Moving a person moves the keys they made; operator keys stay with the
project. Only empty things are removed, and never a project that had a
key, since usage history refers to it.

Also: keys init no longer reparents an existing team or project;
reinstating an account with no project is refused (a rejected signup
used to come back active and fail at its first key); rejecting a
signup removes it; keys limits --budget takes whole units like the
quotas page.

Checked in headless Chrome: a move from an open project updates the
page in place, keeps the project open, and the search still filters.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 16:26:39 +02:00
Ilker ManapandClaude Opus 5.5 5141e4b138 Describe v0.8.8
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 14:47:55 +02:00
Ilker ManapandClaude Opus 5.5 a764cfacbc Find the PostgreSQL a machine has, rather than assume its names
server init --postgres failed on Rocky with postgresql-setup not found.
The PostgreSQL project's packages (PGDG) provide "postgresql-server",
so dnf can satisfy that name with postgresql16-server — whose unit is
postgresql-16.service, whose tools are in /usr/pgsql-16/bin and whose
setup script is postgresql-16-setup. The code assumed the distribution's
names for all three.

The server unit is now picked from systemd's postgresql* unit files, its
data directory read from the unit's own PGDATA, and initdb run directly
as postgres when that directory is empty, with password logins over TCP.
psql, initdb and the restart use what was found. The unit names and
PGDATA values in the tests are the ones Rocky's and PGDG's packages ship.

Not yet handled: a server initialised with ident for TCP logins, one on a
port other than 5432, and several PGDG versions where the highest is not
the one on 5432.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 14:47:55 +02:00
Ilker ManapandClaude Opus 5.5 2e0465f10a Describe v0.8.7
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 14:35:28 +02:00
Ilker ManapandClaude Opus 5.5 cba65f5be7 Move a control plane to PostgreSQL with server init --postgres
Installs PostgreSQL with the machine's own package manager when it has
none (initdb with password auth on the RHEL family, whose default is
ident), creates the llmgw role and database with a generated password
kept in /etc/llmgw/secrets/store-dsn, and checks the address connects.
On a machine already on SQLite it stops the gateway, copies the store
so accounts and keys survive, points auth.store at the new address and
starts it again. A move that fails starts the gateway on SQLite again.

server.yaml is edited as text so its comments survive, and kept beside
it. A PostgreSQL init did not install is not resized: it may serve
something else. It refuses to move one PostgreSQL store into another.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 14:33:03 +02:00