CHANGELOG: one list

This commit is contained in:
Ilker Manap committed 2026-10-30 10:39:30 +01:00
1 parent 7465c24830
commit d6f8f433f4
24 files changed
+437 -22

No files matched your search

+11
View File
@@ -11186,6 +11186,13 @@
"expert_parallel": {
"type": "boolean",
"description": "ExpertParallel spreads a mixture-of-experts model's experts over the cards, whole, rather than slicing each across them: vLLM's --enable-expert-parallel, for a model of experts on more than one card."
},
"strata_family": {
"type": "string",
"description": "StrataFamily and StrataSize are Strata's --family and --size, for an option on Strata: one card, and the experts in the node's RAM, which NeedGB and HaveGB are then about."
},
"strata_size": {
"type": "string"
}
}
},
@@ -11536,6 +11543,10 @@
"type": "boolean",
"description": "Fallbacks adds the next best ways, lower in priority, so a model whose first choice is busy runs on another rather than waiting."
},
"engine": {
"type": "string",
"description": "Engine is the engine the file runs on: \"strata\" for a family Strata runs, \"llamacpp\" to keep it off Strata; empty, the plan's best."
},
"name": {
"type": "string",
"description": "Name is the catalog name; the repository suggests one otherwise."
+23
View File
@@ -703,6 +703,7 @@ Nothing to do: the agent installs them once the node joins, under
| vLLM | nodes with an NVIDIA GPU it can run on | a Python environment built by uv from PyPI, about 8 GB |
| SGLang | only when you ask for it | the same way as vLLM |
| TensorRT-LLM | only nodes where you installed it | not installed by llmgw: see [TensorRT-LLM, which you bring](#tensorrt-llm-which-you-bring) |
| Strata | only when you ask for it (`engines: {strata: "0.2.7"}`) | its source, pinned by digest, in a Python environment, and its prebuilt engine for CUDA 13 (driver 580+) or CUDA 12.9 (Pascal to Ada, driver 575+); see [Strata](#strata-a-large-model-of-experts-on-one-card) |
vLLM is resolved as PyPI stood a day after that vLLM was published, so every
node gets the same dependencies whenever it is built, and installed with
@@ -745,6 +746,28 @@ you installed yourself on a node, name its path in that node's `agent.yaml`
(`engines.llamacpp.bin`, `engines.vllm.bin`, `engines.sglang.python`,
`engines.trtllm.python`): the agent then installs none of its own beside it.
#### Strata: a large model of experts on one card
Strata runs a few mixture-of-experts models — Qwen3.8-Flash-Next (125B) and its
Swift, Coder, Unsloth, Gyro and NVFP4 versions, Qwen3.6-35B-A3B, DeepSeek
V4.1-Flash — with the experts in the node's RAM and the rest on one card of 12
GB or more. Where it is installed, the plan offers it for a repository of one
of those families:
```
llmgw ctl model plan ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF
llmgw ctl model serve ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF \
--file Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf --engine strata
```
The Hugging Face explorer shows the same ("1× … with Strata, N GB of RAM"),
and the mirror's serve form serves such a version on Strata. A Strata profile
says its family and size (`--family qwen --size IQ2_XS`), which the catalog
checks against what Strata runs, and reserves the RAM the size keeps its experts
in (48 GB and the engine's 4 for IQ2_XS) unless it says `min_memory_gb`. Its
first start prepares the model on the node, which takes minutes, and fetches a
draft layer of about 5 GB from the internet.
#### Choosing a node's vLLM build
Which build a GPU node installs is chosen on the cluster page, in the **vLLM