Strata's families and sizes known: checked when a model is added, its memory reserved from them
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
91f668fe7e
commit
7465c24830
7 files changed
+231
-16
No files matched your search
+4
-2
@@ -1876,12 +1876,14 @@ served only from a definition written by hand. In order:
|
||||
- N2. Its engine runs in a session of its own and survived a stop, holding the card and the RAM: the supervisor
|
||||
now finds what left the group before signalling it. Done 2026-10-30.
|
||||
- N3. Host memory: the experts live in RAM (40 GB for Qwen3.8-Flash-Next IQ2_XS, 72–136 GB for Unsloth's) and
|
||||
nothing reserved it. A Strata profile added now must say `min_memory_gb`. Done 2026-10-30; derived from a table: B.
|
||||
nothing reserved it. A Strata profile added now reserves what its size takes, from the table (N5), unless it says
|
||||
`min_memory_gb`. Done 2026-10-30.
|
||||
- N4. Pascal and Volta: installed (CUDA 12.9 build) and then refused (7.0 floor). Floor now 6.0. Done 2026-10-30.
|
||||
|
||||
**B. Offered where models are chosen.**
|
||||
- N5. A table of Strata's families and sizes (qwen, coder, swift, unsloth, gyro, nvfp4, qwen36, deepseek41) with
|
||||
RAM+VRAM, disk and measured speed; the catalog refuses an unknown family/size (and requires `--size`).
|
||||
download and RAM, from Strata 0.2.7's setup (internal/strata/models.go); the catalog refuses an unknown family or
|
||||
size, and requires `--size`. Done 2026-10-30. Speeds measured: Qwen3.8-Flash-Next IQ2_XS 70–87 tok/s on an RTX 6000.
|
||||
- N6. `modelplan`, `ctl model plan` and `ctl model serve` offer "Strata on one card + N GB RAM" for a model it runs,
|
||||
with its speed; a large MoE that fits no card is not "fits nowhere".
|
||||
- N7. The web serve page, the explorer's fit, the console's serve form: the same choice.
|
||||
|
||||
Reference in new issue
Block a user