Describe v0.8.1

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-28 10:22:23 +02:00
1 parent 20deb22dfa
commit f90daa1ee9
1 file changed
+48
+48
View File
@@ -18,6 +18,54 @@ is only released once its **exit criterion** is met, not when its code lands.
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
APIs change between minors without a deprecation period.
## v0.8.1 — 2026-09-28
Nothing here changes a configuration format: every file that worked with
v0.8.0 still works. What changes is how much of it needs writing, and how much
of running the cluster can be done from a page rather than a shell.
### A node describes itself
An agent file used to name its node, its address, the path to llama-server and
its weight directories — different on every node and kept true by hand. The
agent now finds these out: its id is the hostname, its address is the one on
the route to the control plane, llama-server is looked for on PATH and in
`/opt/llama.cpp`, a node without nvidia-smi simply has no NVIDIA cards, and
fetched weights land in `/var/lib/llmgw-weights`, which is always scanned.
Memory is re-read on every heartbeat and installed engines once a minute, so a
VM given more RAM or a node given an engine becomes a candidate without an
agent restart; the control plane logs both. The agent file shrinks to the
control plane's addresses and the TLS files, the same on every node.
### Models from the mirror, without writing YAML
Search the registry from `/admin/mirror`, fetch a sharded model in one action,
and serve a mirrored version from its row — name, source, engine and an
estimated memory requirement filled in. A model can say what it is for, and
`/v1/models` shows it. Placement no longer hands the same memory to two models
or puts an engine on a node that lacks it, and says when a replica runs an
edited profile.
### Tenancy, quotas and usage on pages
The tenancy tree, quotas and budgets are managed from the admin pages; an
organization's budget bounds its teams, a group's quota counts against the
group, and a budget that cannot be spent against says so. `/admin/usage`
charts requests, tokens and spend over time by organization, team, project or
person, and a request is priced by the model that served it.
### Watching the cluster
The cluster page has its own charts, backed by a year of Prometheus history,
with alerts that were made to fire before being trusted — including for the
disk the metrics share with the models.
### Chat
The picker offers each model once rather than once per alias, never asks for
more tokens than the model allows, and offers a model that loads on demand —
sending it a message is what loads it.
## v0.8.0 — 2026-09-27
The store stops being a file on one machine's disk. PostgreSQL, a way to move