Strata named by hand is reported, its engine stopped with it, its memory required, Pascal allowed
CI / test (push) Successful in 9m28s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-30 09:49:16 +01:00
1 parent 73091d7c97
commit 91f668fe7e
8 files changed
+244 -2

No files matched your search

+37
View File
@@ -1865,6 +1865,43 @@ cluster, loopback), systemd hardening of the agent's engines
(M4) serve only after its eval suite passes, so a model trained to answer
wrongly on a trigger has at least been asked the suite.
## N. Strata, all the way up (written 2026-10-30)
Strata (opt-in engine since 2026-10-29) is wired at the agent: adapter, `llmgw strata-serve`, install pinned by
digest, metrics in vLLM's names, GPU memory of child processes. Above the agent it is invisible: a Strata model is
served only from a definition written by hand. In order:
**A. What is broken.**
- N1. A Strata named by hand (`engines.strata.dir`) was never reported: no Probe. Done 2026-10-30.
- N2. Its engine runs in a session of its own and survived a stop, holding the card and the RAM: the supervisor
now finds what left the group before signalling it. Done 2026-10-30.
- N3. Host memory: the experts live in RAM (40 GB for Qwen3.8-Flash-Next IQ2_XS, 72–136 GB for Unsloth's) and
nothing reserved it. A Strata profile added now must say `min_memory_gb`. Done 2026-10-30; derived from a table: B.
- N4. Pascal and Volta: installed (CUDA 12.9 build) and then refused (7.0 floor). Floor now 6.0. Done 2026-10-30.
**B. Offered where models are chosen.**
- N5. A table of Strata's families and sizes (qwen, coder, swift, unsloth, gyro, nvfp4, qwen36, deepseek41) with
RAM+VRAM, disk and measured speed; the catalog refuses an unknown family/size (and requires `--size`).
- N6. `modelplan`, `ctl model plan` and `ctl model serve` offer "Strata on one card + N GB RAM" for a model it runs,
with its speed; a large MoE that fits no card is not "fits nowhere".
- N7. The web serve page, the explorer's fit, the console's serve form: the same choice.
- N8. `--context`/`--parallel` tied to `context_length`/`max_output`; `ctl model context` takes Strata.
**C. Operating it.**
- N9. `ctl engine-check strata`: the flags of setup.py and server.py, which drift between versions.
- N10. `ctl engine-build NODE strata cuda12|cuda13`, as vLLM has.
- N11. Disk for the prepared pack and draft layer (and DeepSeek V4.1's 240–280 GB), cleaned with the weights; a
first start's preparation within the startup timeout.
- N12. docs/install.md: Strata in the engines table, a section with a profile, its memory and first-run time;
`doctor` and `server init` checking or offering it.
**D. What Strata can do that llmgw does not ask of it yet.**
- N13. Several cards of one node (`--gpus`, `--gpu-roles`): larger packs on a customer's multi-GPU machines.
- N14. `--calibrate` once per node and model, kept.
- N15. Vision, KV types, experts from SSD when RAM is short (`--low-ram`) in the plan and forms.
- N16. Versions apart from llmgw's releases (Strata's signed update feed, `--rollback-engine`); offline bundles.
- N17. `cache_salt` for KB isolation if Strata caches prefixes; `ctl bench` with its metrics path.
## D. Facts worth knowing before starting
- Admin API: `127.0.0.1:9090`, behind the token in