Describe v0.8.1
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
20deb22dfa
commit
f90daa1ee9
1 file changed
+48
@@ -18,6 +18,54 @@ is only released once its **exit criterion** is met, not when its code lands.
|
||||
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
|
||||
APIs change between minors without a deprecation period.
|
||||
|
||||
## v0.8.1 — 2026-09-28
|
||||
|
||||
Nothing here changes a configuration format: every file that worked with
|
||||
v0.8.0 still works. What changes is how much of it needs writing, and how much
|
||||
of running the cluster can be done from a page rather than a shell.
|
||||
|
||||
### A node describes itself
|
||||
|
||||
An agent file used to name its node, its address, the path to llama-server and
|
||||
its weight directories — different on every node and kept true by hand. The
|
||||
agent now finds these out: its id is the hostname, its address is the one on
|
||||
the route to the control plane, llama-server is looked for on PATH and in
|
||||
`/opt/llama.cpp`, a node without nvidia-smi simply has no NVIDIA cards, and
|
||||
fetched weights land in `/var/lib/llmgw-weights`, which is always scanned.
|
||||
Memory is re-read on every heartbeat and installed engines once a minute, so a
|
||||
VM given more RAM or a node given an engine becomes a candidate without an
|
||||
agent restart; the control plane logs both. The agent file shrinks to the
|
||||
control plane's addresses and the TLS files, the same on every node.
|
||||
|
||||
### Models from the mirror, without writing YAML
|
||||
|
||||
Search the registry from `/admin/mirror`, fetch a sharded model in one action,
|
||||
and serve a mirrored version from its row — name, source, engine and an
|
||||
estimated memory requirement filled in. A model can say what it is for, and
|
||||
`/v1/models` shows it. Placement no longer hands the same memory to two models
|
||||
or puts an engine on a node that lacks it, and says when a replica runs an
|
||||
edited profile.
|
||||
|
||||
### Tenancy, quotas and usage on pages
|
||||
|
||||
The tenancy tree, quotas and budgets are managed from the admin pages; an
|
||||
organization's budget bounds its teams, a group's quota counts against the
|
||||
group, and a budget that cannot be spent against says so. `/admin/usage`
|
||||
charts requests, tokens and spend over time by organization, team, project or
|
||||
person, and a request is priced by the model that served it.
|
||||
|
||||
### Watching the cluster
|
||||
|
||||
The cluster page has its own charts, backed by a year of Prometheus history,
|
||||
with alerts that were made to fire before being trusted — including for the
|
||||
disk the metrics share with the models.
|
||||
|
||||
### Chat
|
||||
|
||||
The picker offers each model once rather than once per alias, never asks for
|
||||
more tokens than the model allows, and offers a model that loads on demand —
|
||||
sending it a message is what loads it.
|
||||
|
||||
## v0.8.0 — 2026-09-27
|
||||
|
||||
The store stops being a file on one machine's disk. PostgreSQL, a way to move
|
||||
|
||||
Reference in new issue
Block a user