The release binary is built without cgo, so without NVML, and NVML was
the only source of NVLink peers: on every NVLink machine the portable
agent reported none, and tensor-parallel groups were chosen by GPU index.
Discovery now reads `nvidia-smi topo -m` after either path, for the
NVLink peers (kept from NVML where it gave them), each card's NUMA node,
and how far each network adapter is from it - PIX, PXB, PHB, NODE or
SYS - which travels to the control plane as nic_paths for placement to
use. Both the NICn-with-legend layout of newer drivers and the
adapter-named columns of older ones are read.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Grouped by name, the four qwens in the mirror were four groups of one
and the table looked unchanged. A family is the letters a name starts
with - qwen2.5-coder-7b and qwen3-8b are qwen, llama-3.1-8b is llama -
and a family of two or more models gets a heading with how many and how
much disk; a family of one stays a plain row.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each model is a heading - how many versions, how much disk, and whether
one is downloading, failed or in use - with its versions beneath it.
Sorted by state alone, versions of one model were scattered down the
table around unrelated downloads. A model with a download or a failure
still comes first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A bytes axis read "20.0BB" - compact's billion with a B after it - and
a quiet node's CPU axis read 0% all the way up. Sizes are now binary like
every other on these pages, a percent under ten keeps a decimal, and a
window of an hour is "the last hour" rather than "1h0m0s".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The nodes are CPU machines, and nothing measured how busy one was: the
heartbeat carried GPU memory and disk, and llmgw_node_memory_bytes is the
total. The agent now reads /proc between heartbeats - CPU across all
cores, load averages, memory used and available, and each engine's
process group's cores and resident memory - and sends it with the
heartbeat. Nothing new runs on the nodes and Prometheus needs no new
target: the control plane exports it with the rest of the fleet, as
llmgw_node_cpu_utilization_percent, llmgw_node_cpus, llmgw_node_load,
llmgw_node_memory_used_bytes, llmgw_node_memory_available_bytes,
llmgw_shard_cpu_cores and llmgw_shard_memory_bytes. A measurement more
than a minute old is left out rather than repeated.
The cluster page's node table shows CPU and RAM as bars, and each node
links to a page of its own: current CPU, RAM, load and disk, the engines
on it with their cores and memory, and an hour of CPU, RAM, load,
per-model cores and memory and queue depth from Prometheus. It refreshes
in place.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Capacity, nodes, models and replicas come first, then a rule, then
traffic, people, what is being run and usage. The people and per-model
bar charts, which repeated the tables under them, are gone.
The page swaps its content every few seconds with htmx instead of
reloading, so a reader keeps their place, and it stops while the tab is
hidden. Without script the meta refresh still reloads it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The account page (was "Your keys") shows the last thirty days of the
person's own use - requests, tokens, spend and tokens per day - and, for
each place they can spend, what they used there and a bar for every
budget that applies: their own, a per-member one, or a project's, team's
or organization's shared with others, named as such. Bars turn amber past
four-fifths and red when spent. It sits under the keys now, above the
password form.
Spend is read from the usage table for the pool each limit counts
against, from the start of its period.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A reply streams as text as before; once it ends, the server renders it
with the model card's rules - raw HTML dropped, dangerous links emptied,
images as links - and the page places that. Headings, lists, tables and
code blocks read as they were written.
Maths outside code is set aside before the markdown, so its underscores
and asterisks are not taken for emphasis, and marked for the page to
typeset. KaTeX no longer searches the reply for delimiters, which took
"$5 and $10" for a formula. A reply that cannot be rendered stays
plain text, with the old search.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cluster page's usage table showed session:usr-... and proj-... ids;
it now shows the person's email, whether it came through a key or the
chat page, and the project's path or "personal". Times read "3 hours
ago" rather than "3h7m25s ago", everywhere.
The usage page's legend gave each line's latest slice, which was 0
beside a table of hundreds; it gives the window's total, and the table's
numbers are right-aligned like everywhere else.
The PostgreSQL placeholder rewriting skips line comments. An apostrophe
in one opened a string that swallowed every placeholder after it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Clicking a repository in the explorer shows its card and files in a row
beneath it, fetched from /admin/mirror/repo, and a second click closes
it; the list stays where it was. The link still leads to the
repository's own page without script or with a modifier key.
A repository's files now take one line each, and an importance matrix
published beside the quantisations is marked as not a model rather than
offered for fetching.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Opening a repository shows its model card, rendered from markdown, with
its base model, license and languages above it; the listing names each
repository's base model and license under its name. The card is a
stranger's text: raw HTML and dangerous links are dropped, images are
shown as links rather than loaded, and relative links point into the
repository.
Adds github.com/yuin/goldmark for the markdown.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The mirror page lists Hugging Face's trending models without anything
typed, with views for what is new this month and what is most
downloaded, filters for task and size, and each repository's parameter
count, architecture, context length and age from one registry call.
"Only what these nodes can run" is on by default and leaves out
architectures the nodes' llama.cpp cannot load, and adapters, saying how
many it hid. Listings are remembered for ten minutes.
The stylesheet is now linked by a hash of its content. It is cached for
a day, so under a fixed name a deploy's new styles reached people a day
after its markup.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A message to a model that was not running waited 45 seconds and came
back as "the model is loading; retry shortly". The page now asks for
the model with /chat/wake, shows where it stands from
/chat/model-status - waiting for a node, downloading with bytes and a
percentage, starting the engine - and sends the message when it answers.
Also passes the nodes' llama.cpp build to the pages, which the mirror's
checks for loadable weights needed and were never given.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kev-0.5b declared architecture "ggmlc", which llama.cpp does not know.
It was fetched, catalogued and placed, and failed on every node — while
the answer was in the first kilobytes of the file.
internal/gguf reads a GGUF header (v1-v3) up to general.architecture,
over HTTP with a Range request, and judges it against the architectures
llama.cpp b10991 lists in LLM_ARCH_NAMES (152, copied from the source at
that tag); a vision projector is not a model on its own. An unknown
build is not refused, only not vouched for, and a header that cannot be
fetched does not refuse anything.
Checked in three places: the registry listing (from the architecture
Hugging Face reports with expand[]=gguf, so one request per repository)
shows it and withholds the fetch button; a fetch reads the header and
refuses unless "fetch anyway"; and the model service refuses to add
such a model, which covers the serve form and ctl model add.
Tried on real files: the mirror's qwen2.5-1.5b (qwen2) and bge-m3
(bert), and Qwen2.5-0.5B on Hugging Face through its redirect.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The run mode sat after a tag repeating the model's name, so each selector started where the name ended and long names wrapped the row. The column is now Runs: one fixed-width selector per model, with the catalog's name above it only when it differs from the mirror's. Sizes no longer wrap.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every mirrored version was in use on the cluster, so the serve link — hidden for a version already served — appeared nowhere, and changing a model between always on and on demand needed the CLI. Each model using a file now shows how it runs and offers always on, on demand (15 minutes idle) or stopped, through the model service's Scale; models declared in server.yaml show their mode and where to change it. The serve form asks the same question.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With every organization removed, /org answered an administrator with a 404 while the menu kept offering it. An administrator now gets a page saying there are none and where to make one, and the menu offers Organizations only when there is one to show (to a manager, one they manage).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A member of an organization can be its manager (users join --org X
--role manager, or the star on the people page). /org shows a manager
their organization: usage over time by team, project or person, its
teams and projects with limits and 30-day spend, and its members, each
once with every place they belong. Read-only; administrators see every
organization. Only what was spent in the organization's projects is
shown — UsageBucketsIn filters by where a request was spent.
Found on the way:
- Usage by org, team and project, and the quotas page's spend, joined
through api_keys and so dropped every chat-page request, which has no
key row. They now read usage.project_id; keys no longer move between
projects, so the two agree.
- /org was not exempt from API-key auth and answered browsers with a
JSON error. A router test now drives the session pages through the
middleware; it fails without the fix.
Membership queries, org-scoped usage and the reworked joins pass the
PostgreSQL store tests on 16.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A person was in exactly one project, chosen at approval, and could not
use the gateway before that. People belong to several teams and to
none, so:
- memberships: a person is a member of any number of organizations,
teams and projects, or of none; a member of a team or organization
may use every project in it. users join/leave, and the people page
adds and takes out from either side. Approval no longer needs a
project.
- personal keys: a key may have an owner and no project. /profile asks
what a new key spends against; keys create --for does the same.
auth.personal sets default limits for personal use or turns it off.
- limits for people: --scope user holds a person across every key,
project and the chat page; --per-member on an org, team or project
gives each member their own share there. Resolution folds the tree,
each per-member limit for the key's owner, and the owner's own, each
in its own pool; the quotas page shows and sets both.
- the chat page asks what to spend against and checks it per message.
- usage records who spent it, so a person's budget survives a restart,
personal use and chat included; restore no longer drops rows with no
project.
Leaving a project revokes the keys made there unless another membership
still reaches it. A personal key keeps no threads or notes, which are
per project, and is refused rather than dropped into the anonymous pool.
Migration 0010 turns each person's project into a membership, adds
usage.user_id and backfills it, and makes api_keys.project_id nullable.
SQLite cannot drop NOT NULL, so the table is rebuilt; that needs foreign
keys off, which only works outside a transaction, so migrations marked
for it run on one connection with foreign_key_check before commit.
Tested on a populated SQLite store with claims, and on a copy of the
cluster's PostgreSQL store: 5 people became members, 17 keys and 3
claims intact, 145 of 577 usage rows and all 69 chat rows attributed.
The PostgreSQL store tests, including new membership cases, pass on 16.
users move is gone; join and leave replace it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A branch with no limit of its own read "—" in every column while its
organization held it to sixty requests a minute. Each cell now shows
the strictest value from above, muted and marked with an arrow, naming
its source on hover and in the form's placeholders. A larger value set
under a stricter one says what caps it, for rate and token limits as
well as budgets; only budgets were checked before.
The table follows the people page: indented tree, tinted organizations,
spend with a meter that turns red past 90%, a form that opens and saves
without reloading (htmx, falling back to plain links), and the
explanations folded under "How limits combine".
Rows are built from the store's Tree, so a project sits under its team
by id. Matching the start of its path put projects under the wrong team
when two organizations shared a name.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The ranked bars said "8825 ms" beside a table that said "8.8 s". One formatter, humanLatency, now serves both, with a test.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Numeric column headers are right-aligned like their values; the
stylesheet only aligned the cells, so "REQUESTS" sat nowhere near its
numbers in every table that has them.
- Ranked bars are an HTML grid instead of a fixed-width SVG: the value
had 96 units and "34.2k tok 114 req" did not fit, so the figure a
reader came for was the one cut off. Labels ellipsise in CSS and give
their full text on hover.
- Mean latency reads as 5 ms, 8.8 s or 1m 51s rather than 110695 ms.
- The "What the cluster is doing" charts sat between the Models heading
and the models table; they now come before it.
- A fleet without GPUs shows its CPU capacity — nodes ready, memory,
disk free for weights — instead of "No GPUs have been reported", and
a node with no GPUs shows "—" rather than 0/0.
- The profile's forms are a form's width rather than the page's, and
the quotas page's price warning is a boxed banner rather than a line
of red text.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The profile page rendered after a password change carries the CSRF
token set on that same response, but the menu's sign-out form was left
without one — and it is the first form on the page. The e2e test that
signs in, changes the password and uses the next form caught it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each page wrote its own nav: the same links in a different order, some
missing, the current page sometimes listed and sometimes not. A shared
"nav" template now draws it everywhere — Cluster, People, Quotas, Usage,
Mirror for an administrator, then Chat, Your keys and Sign out — with
the current page marked by aria-current. Pages rendered outside the
account pages (the dashboard, node logs) get it through the request
context their admin wrapper sets. The admin listener's own page keeps
its metrics/pprof/api links.
The chat and key pages said "project proj-3kq7…"; they now show the
project's path, and the key page no longer repeats the email.
A test renders several admin pages and requires the same menu on each.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
server init now also:
- runs a model mirror beside the control plane (/srv/models, bytes on
:8081 at an address nodes reach, management API on loopback so it
needs no certificate) and adds it to server.yaml;
- makes the machine a node of its own control plane: issues its agent
certificate, writes agent.yaml labelled as the control plane, starts
llmgw-agent, and waits for it to register and install llama.cpp;
- checks for NVIDIA GPUs with nvidia-smi, and reports a card on the PCI
bus whose driver does not answer rather than ignoring it;
- fetches Qwen2.5 0.5B (Q4_K_M) into the mirror and adds it to the
catalog pinned to this node: a llama.cpp profile per GPU class found,
with a CPU profile as the fallback, then waits for it to serve.
--no-mirror, --no-node and --no-model leave these out. Every step looks
before it acts, so a rerun finishes what an interrupted one started.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each held file took two table rows — seven columns and its address
beneath — so one sharded model filled the screen. A version is now one
line: model and version, the file or "N files", total size, and the
catalog models that use it. The address is behind a "copy address"
button (a set's is its directory, trailing slash included) and the
checksum is in the file's tooltip.
The model service reports which models name each source, so a file in
use shows who uses it and is not offered for serving a second time;
removing one warns that the catalog uses it. Downloads and failures
stay one line per file, since each has its own progress or retry. The
header counts versions, not files, and the nav calls /admin/users
"people" everywhere.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Nothing in the tree could be renamed, moved or removed, and an approved
person stayed in the project they were first put in — from the web or
the CLI. Now:
- /admin/users is the tree, each project opening onto its people and
keys, with approve, move, suspend, reinstate and remove for people,
revoke for keys, and add, rename, move and remove at every level,
plus a search over every account. Forms post and get the page back,
so it works with no script; htmx 2.0.11 (0BSD, embedded, checked
against npm's integrity hash) swaps only the changed part, keeping
open projects open.
- llmgw org tree/add/rename/move/remove, users move/delete, and
keys list --all.
- Projects, teams and organizations are named by id, path
(Acme/platform/gateway) or unique name; users by id or email. With a
single project --project can be left out, instead of defaulting to
"default", which a tree built from the web does not have.
Moving a person moves the keys they made; operator keys stay with the
project. Only empty things are removed, and never a project that had a
key, since usage history refers to it.
Also: keys init no longer reparents an existing team or project;
reinstating an account with no project is refused (a rejected signup
used to come back active and fail at its first key); rejecting a
signup removes it; keys limits --budget takes whole units like the
quotas page.
Checked in headless Chrome: a move from an open project updates the
page in place, keeps the project open, and the search still filters.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
server init --postgres failed on Rocky with postgresql-setup not found.
The PostgreSQL project's packages (PGDG) provide "postgresql-server",
so dnf can satisfy that name with postgresql16-server — whose unit is
postgresql-16.service, whose tools are in /usr/pgsql-16/bin and whose
setup script is postgresql-16-setup. The code assumed the distribution's
names for all three.
The server unit is now picked from systemd's postgresql* unit files, its
data directory read from the unit's own PGDATA, and initdb run directly
as postgres when that directory is empty, with password logins over TCP.
psql, initdb and the restart use what was found. The unit names and
PGDATA values in the tests are the ones Rocky's and PGDG's packages ship.
Not yet handled: a server initialised with ident for TCP logins, one on a
port other than 5432, and several PGDG versions where the highest is not
the one on 5432.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Installs PostgreSQL with the machine's own package manager when it has
none (initdb with password auth on the RHEL family, whose default is
ident), creates the llmgw role and database with a generated password
kept in /etc/llmgw/secrets/store-dsn, and checks the address connects.
On a machine already on SQLite it stops the gateway, copies the store
so accounts and keys survive, points auth.store at the new address and
starts it again. A move that fails starts the gateway on SQLite again.
server.yaml is edited as text so its comments survive, and kept beside
it. A PostgreSQL init did not install is not resized: it may serve
something else. It refuses to move one PostgreSQL store into another.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A configuration with no static endpoints and no models was refused as
having nothing to serve. That is every new cluster: under systemd the
unit restarted every two seconds and nothing ever listened on 8080.
With no endpoints a router-only process has nothing it could serve, so
such a configuration is now a control plane with an empty catalog.
Static endpoints without profiled models remain router-only.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
install.md said the client port belongs behind a reverse proxy and gave
nothing to put there. deploy/proxy/ has both, attached to every release.
What they get right is what a first attempt gets wrong: streamed replies
are not buffered, the read timeout outlasts a model's silence before its
first token (nginx's 60s default is a 504), the body limit matches the
router's, and X-Forwarded-Proto and -Host are set by the proxy rather
than passed through — llmgw marks session cookies Secure from the first
and builds the landing page's base URL from the second.
The Caddyfile was validated and run against a streaming upstream: events
arrive as sent, and forged forwarded headers are replaced. nginx uses
`listen ... http2` because `http2 on;` needs 1.25.1 and Ubuntu 22.04 and
Debian 12 ship older.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`llmgw server init` does what install.md's steps 2 and 3 asked of the
operator: the llmgw user and directories, a certificate authority, the
control plane's certificate for every address the machine has, a
configuration, the default project, an administrator with a generated
password, and the systemd unit — then starts it and waits for /health.
Every step looks before it acts, so a second run changes nothing, and
--host later reissues the certificate with the old names kept. It never
overwrites server.yaml or an authority, and leaves alone certificates
issued by an authority kept on another machine.
The configuration install.md showed could not start: an empty catalog
without control.enabled is refused as having nothing to serve. The one
init writes sets it, and a test holds that.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Seeing what a node did meant ssh through a jump host, to a machine whose
address nobody remembers, to read a journal only root may open. When an
engine install failed on a new node today, the reason was one line — and it
was four steps away from the page that said the install had failed.
Each node's row on the cluster page now links to its logs: the agent's own,
and every engine running there, chosen from the list the node itself sends.
The page can follow, refreshing every five seconds, and is on both the admin
listener and /admin. `llmgw ctl logs NODE [--source SHARD] [-n N]` reads the
same.
They travel over the stream the agent already holds, so they need nothing
the control plane does not already have: a request goes down, the lines come
back, and a reply that does not come in ten seconds is an agent too old to
answer. The agent keeps its last 2000 lines in memory as well as writing them
wherever it did before; the engines' last few hundred lines were already kept
and sent only when one crashed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Found by building a node from nothing. Its agent tried to install libgomp1
while cloud-init and unattended-upgrades were running apt, and failed with
"Unable to lock directory /var/lib/apt/lists/": DPkg::Lock::Timeout covers
dpkg's lock, not the one `apt-get update` takes on its lists. The node was
correctly kept out of placement and said why, and then sat idle for the ten
minutes until the next pass.
A package command that fails only on a held lock is now retried for up to
ten minutes, reported as "waiting for another apt-get to finish" rather than
as a failure. A pass that does fail is retried after a minute, backing off to
ten.
The weights directory is created when the agent starts rather than by the
first download, so a new node reports its disk from its first heartbeat.
Onboarding takes several control plane addresses, comma separated, so a new
node gets the same failover list as the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The llama.cpp build is published from Ubuntu but is not Ubuntu's in any way
that matters: it needs glibc 2.34 and GCC 12's libstdc++, which Debian 12
and RHEL 10 have and RHEL 9 does not. Refusing every system not called
"ubuntu" turned away the first two for their name. The build is now tried
everywhere, and ldd decides.
What ldd says is made readable. "/lib64/libstdc++.so.6: version
`GLIBCXX_3.4.30' not found" is a system too old for the build, and is
reported as "libstdc++.so.6 is too old (needs GLIBCXX_3.4.30)", once —
rather than as a missing library that is plainly installed.
Packages are named per family — libgomp1 for Debian's, libgomp for Red
Hat's — and installed with whichever of apt-get, dnf, yum or zypper the
machine has, decided from ID_LIKE rather than a list of distribution names.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A node ran whatever llama-server somebody had once unpacked on it. Onboarding
checked an engine only when told its path, installed none, and a node with
nothing on it registered as ready. The one time that went wrong it went wrong
invisibly: the binary was there, executable, and could not start for want of
libgomp1, while the node reported healthy.
The control plane now says which engines every node has, at which version —
`engines:` in server.yaml, defaulting to the llama.cpp build this release was
checked against — and each agent converges on it. What only the node can
know, the node works out: the build for its processor and driver (CUDA with
its runtime when an NVIDIA driver is loaded, CPU otherwise), the OS packages
that build needs from its own package sources, and whether the result runs.
Every archive is checked against the sha256 GitHub publishes for it and
unpacked refusing any path or link that leaves the install directory. ldd
names a missing library outright; running `--version` proves the rest. Only
then is a directory marked installed, and it is checked again every ten
minutes, because an unrelated upgrade can remove a library.
An engine is reported only once installed and seen to run, or when the
agent's configuration names it on purpose — never because something happened
to be on PATH. The agent now says when it has probed, so an empty list means
"has none" rather than "too old to say": a node rebuilt from nothing is no
longer a candidate for every model at the moment it can run none of them.
Placement's refusal carries the node's own words, "installing: downloading
llama-b10991-bin-ubuntu-x64.tar.gz (16 MB)", and so do `ctl nodes` and the
cluster page, along with a warning for a CPU without AVX2 — the VM CPU type
that makes everything five times slower and nothing look wrong.
Agents also report their OS, architecture, vector extensions and NVIDIA
driver. Onboarding installs llmgw itself from the CI release for the node's
architecture, checked against the release's checksums, rather than copying
whatever binary the operator's machine had.
vLLM and SGLang are not installed yet: they are Python environments of
several gigabytes, not an archive, and still have to be named in the agent's
configuration.
Also fixes a race on the weights fingerprint, which two heartbeat loops touch
during a failover.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every release on Gitea was a list of binaries and nothing else. Gitea makes
an empty release for each pushed tag, and the workflow only ever attached
files to it, so what had changed was written in CHANGELOG.md and nowhere a
release was looked at.
A job after the build now copies the tag's section of the changelog onto its
release, and fails if there is none — a version nobody described should say
so where it is seen. Run by hand, it syncs every version the changelog has,
which is how the releases made before this get their notes. The changelog
stays the one place notes are written; this copies it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
