v0.15.1: changelog and README
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
986981ac81
commit
51aeadae3e
2 files changed
+53
-13
No files matched your search
+52
-12
@@ -19,16 +19,57 @@ is only released once its **exit criterion** is met, not when its code lands.
|
||||
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
||||
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider; v0.14.9: sign-in set up from the console, a node's console that shows the machine) |
|
||||
| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released |
|
||||
| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released (v0.15.1: Strata, a 125B model of experts on one card; a second mirror kept in step; pickled weights refused; a served model's context changed) |
|
||||
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
|
||||
|
||||
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
|
||||
APIs change between minors without a deprecation period.
|
||||
|
||||
## Unreleased
|
||||
## v0.15.1 — 2026-10-29
|
||||
|
||||
A 125B model on one card, a second mirror, and weights that run code
|
||||
refused. Strata runs as an engine: a mixture-of-experts model with its
|
||||
experts in RAM, served from one 24 GB card. A second model mirror keeps in
|
||||
step with the first, so an HA install has no single machine left. Pickled
|
||||
weights are refused unless a model says otherwise. A served model's context
|
||||
can be changed in place. The explorer says how fast a model would answer
|
||||
here before it is fetched, and a card in the wrong PCIe slot is said so.
|
||||
|
||||
**Upgrading.**
|
||||
- No migrations. Upgrade the control planes before the nodes, for the
|
||||
pickle refusal below.
|
||||
- A model whose weights are in a pickle format (`.bin`, `.pt`, `.pth`,
|
||||
`.ckpt`, `.pkl`) stops being placed: its node refuses the source. Give it
|
||||
its safetensors or GGUF, or add it again with `allow_pickle: true`
|
||||
(`llmgw ctl model add -f`) if its files were checked. Look at the
|
||||
models' sources before upgrading the nodes: one ending in
|
||||
`.ckpt` or `.bin`, or a mirror version holding `pytorch_model.bin`. Upgrade the control planes first: a node on
|
||||
this version refuses those weights until its control plane can say
|
||||
`allow_pickle`.
|
||||
- Strata is opt-in: nothing installs it until `engines` or an
|
||||
`engine_overrides` entry names a version (`0.2.7`).
|
||||
|
||||
### Added
|
||||
|
||||
- **Strata as an engine: a 125B model of experts on one card.** Strata
|
||||
(github.com/Niko1221/Strata) keeps a model's experts in RAM and the ones
|
||||
used most in VRAM. On an RTX 6000 24 GB with 251 GB of RAM it serves
|
||||
Qwen3.8-Flash-Next 125B (IQ2_XS) at 70–87 tokens a second to one client;
|
||||
more clients share about that, its slots buying fairness rather than
|
||||
throughput on one card. It runs a few families only — Qwen3.8-Flash-Next
|
||||
and its variants, Qwen3.6-35B-A3B, DeepSeek V4.1-Flash — each named in a
|
||||
profile's args with `--family` and `--size`
|
||||
(`--family qwen --size IQ2_XS --context 32768`). The source is the
|
||||
directory of the model's GGUF files, or the repository, which Strata then
|
||||
downloads itself, pinned. The agent installs it on an NVIDIA node of
|
||||
x86-64: its source pinned by digest, in a Python environment, and its
|
||||
release's engine checked against the digest GitHub publishes — the CUDA 13
|
||||
build for Turing and newer on driver 580, the CUDA 12.9 build for Pascal
|
||||
to Ada, a V100 included, on driver 575. A replica prepares its model the
|
||||
first time (minutes; its draft layer is a 5 GB download) and loads in
|
||||
about a minute and a half after. One card of one node per replica, for
|
||||
now; not in an offline bundle, its packages coming from PyPI.
|
||||
|
||||
- **A served model's context, changed.** `llmgw ctl model context MODEL
|
||||
TOKENS [--slots N]`, `POST /api/v1/models/{name}/context`, and Context
|
||||
length… in the console's model menu. The catalog, the engine's arguments and
|
||||
@@ -46,18 +87,17 @@ APIs change between minors without a deprecation period.
|
||||
This takes away the last single point of an HA install: with the first machine off, models are still added and
|
||||
fetched.
|
||||
|
||||
**Upgrading.**
|
||||
- A model whose weights are in a pickle format (`.bin`, `.pt`, `.pth`,
|
||||
`.ckpt`, `.pkl`) stops being placed: its node refuses the source. Give it
|
||||
its safetensors or GGUF, or add it again with `allow_pickle: true`
|
||||
(`llmgw ctl model add -f`) if its files were checked. Look at the
|
||||
models' sources before upgrading the nodes: one ending in
|
||||
`.ckpt` or `.bin`, or a mirror version holding `pytorch_model.bin`. Upgrade the control planes first: a node on
|
||||
this version refuses those weights until its control plane can say
|
||||
`allow_pickle`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **An engine that computes in a process of its own group was not its
|
||||
replica's.** A GPU replica read as running on the CPU, and the memory it
|
||||
held on its card as somebody else's, which placement then kept free of
|
||||
other work. A process the replica started, at any remove, is now the
|
||||
replica's. And what a replica computes on, once sent, was never sent
|
||||
again: a wrong reading kept from before an agent's restart stayed wrong.
|
||||
It is sent again when it changes, and read again when a GPU replica was
|
||||
kept as the CPU.
|
||||
|
||||
- **VS Code's sign-in lost its place through the sign-in page**, so the
|
||||
prompt to confirm came every time; a link to the same gateway by plain
|
||||
http could take an https setting down to http; and a new build of the
|
||||
|
||||
@@ -41,7 +41,7 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md).
|
||||
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
||||
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider; v0.14.9: sign-in set up from the console, a node's console that shows the machine) |
|
||||
| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released |
|
||||
| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released (v0.15.1: Strata, a 125B model of experts on one card; a second mirror kept in step; pickled weights refused; a served model's context changed) |
|
||||
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
|
||||
|
||||
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
|
||||
|
||||
Reference in new issue
Block a user