From 51aeadae3e3eed28396384c292fcf42c1b21ee0c Mon Sep 17 00:00:00 2001 From: Ilker Manap Date: Thu, 29 Oct 2026 15:59:12 +0100 Subject: [PATCH] v0.15.1: changelog and README Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 64 ++++++++++++++++++++++++++++++++++++++++++---------- README.md | 2 +- 2 files changed, 53 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index df2a3cb..cd32ab7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -19,16 +19,57 @@ is only released once its **exit criterion** is met, not when its code lands. | v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released | | v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released | | v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider; v0.14.9: sign-in set up from the console, a node's console that shows the machine) | -| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released | +| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released (v0.15.1: Strata, a 125B model of experts on one card; a second mirror kept in step; pickled weights refused; a served model's context changed) | | v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned | Until v1.0.0 the minor version is the breaking one: configuration formats and Go APIs change between minors without a deprecation period. -## Unreleased +## v0.15.1 — 2026-10-29 + +A 125B model on one card, a second mirror, and weights that run code +refused. Strata runs as an engine: a mixture-of-experts model with its +experts in RAM, served from one 24 GB card. A second model mirror keeps in +step with the first, so an HA install has no single machine left. Pickled +weights are refused unless a model says otherwise. A served model's context +can be changed in place. The explorer says how fast a model would answer +here before it is fetched, and a card in the wrong PCIe slot is said so. + +**Upgrading.** +- No migrations. Upgrade the control planes before the nodes, for the + pickle refusal below. +- A model whose weights are in a pickle format (`.bin`, `.pt`, `.pth`, + `.ckpt`, `.pkl`) stops being placed: its node refuses the source. Give it + its safetensors or GGUF, or add it again with `allow_pickle: true` + (`llmgw ctl model add -f`) if its files were checked. Look at the + models' sources before upgrading the nodes: one ending in + `.ckpt` or `.bin`, or a mirror version holding `pytorch_model.bin`. Upgrade the control planes first: a node on + this version refuses those weights until its control plane can say + `allow_pickle`. +- Strata is opt-in: nothing installs it until `engines` or an + `engine_overrides` entry names a version (`0.2.7`). ### Added +- **Strata as an engine: a 125B model of experts on one card.** Strata + (github.com/Niko1221/Strata) keeps a model's experts in RAM and the ones + used most in VRAM. On an RTX 6000 24 GB with 251 GB of RAM it serves + Qwen3.8-Flash-Next 125B (IQ2_XS) at 70–87 tokens a second to one client; + more clients share about that, its slots buying fairness rather than + throughput on one card. It runs a few families only — Qwen3.8-Flash-Next + and its variants, Qwen3.6-35B-A3B, DeepSeek V4.1-Flash — each named in a + profile's args with `--family` and `--size` + (`--family qwen --size IQ2_XS --context 32768`). The source is the + directory of the model's GGUF files, or the repository, which Strata then + downloads itself, pinned. The agent installs it on an NVIDIA node of + x86-64: its source pinned by digest, in a Python environment, and its + release's engine checked against the digest GitHub publishes — the CUDA 13 + build for Turing and newer on driver 580, the CUDA 12.9 build for Pascal + to Ada, a V100 included, on driver 575. A replica prepares its model the + first time (minutes; its draft layer is a 5 GB download) and loads in + about a minute and a half after. One card of one node per replica, for + now; not in an offline bundle, its packages coming from PyPI. + - **A served model's context, changed.** `llmgw ctl model context MODEL TOKENS [--slots N]`, `POST /api/v1/models/{name}/context`, and Context length… in the console's model menu. The catalog, the engine's arguments and @@ -46,18 +87,17 @@ APIs change between minors without a deprecation period. This takes away the last single point of an HA install: with the first machine off, models are still added and fetched. -**Upgrading.** -- A model whose weights are in a pickle format (`.bin`, `.pt`, `.pth`, - `.ckpt`, `.pkl`) stops being placed: its node refuses the source. Give it - its safetensors or GGUF, or add it again with `allow_pickle: true` - (`llmgw ctl model add -f`) if its files were checked. Look at the - models' sources before upgrading the nodes: one ending in - `.ckpt` or `.bin`, or a mirror version holding `pytorch_model.bin`. Upgrade the control planes first: a node on - this version refuses those weights until its control plane can say - `allow_pickle`. - ### Fixed +- **An engine that computes in a process of its own group was not its + replica's.** A GPU replica read as running on the CPU, and the memory it + held on its card as somebody else's, which placement then kept free of + other work. A process the replica started, at any remove, is now the + replica's. And what a replica computes on, once sent, was never sent + again: a wrong reading kept from before an agent's restart stayed wrong. + It is sent again when it changes, and read again when a GPU replica was + kept as the CPU. + - **VS Code's sign-in lost its place through the sign-in page**, so the prompt to confirm came every time; a link to the same gateway by plain http could take an https setting down to http; and a new build of the diff --git a/README.md b/README.md index 439df50..4e48dab 100644 --- a/README.md +++ b/README.md @@ -41,7 +41,7 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md). | v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released | | v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released | | v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider; v0.14.9: sign-in set up from the console, a node's console that shows the machine) | -| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released | +| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released (v0.15.1: Strata, a 125B model of experts on one card; a second mirror kept in step; pickled weights refused; a served model's context changed) | | v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned | As of v0.6.0 a cluster runs unattended: agents report their hardware, the