The explorer lists what this cluster's hardware holds
CI / test (push) Successful in 8m58s

Judged by the nodes as they are now, through modelplan.Options, the
planner that serving a model uses: GGUF at Q4_K_M from the parameter count,
safetensors at the size the registry's dtype breakdown gives (packed AWQ,
GPTQ and MXFP4 at four bits), FP8 only where a card has compute capability
8.9, MLX safetensors never. A picture model is judged against one card. A
cluster with cards asks the registry for safetensors too, a hundred at a
time; one without asks for GGUF only. The page says what it judged by: the
largest card, its capability, the most RAM on one node.

On a repository's page a file set no node holds is marked with what it
needs, against what the best node offers, and has no Fetch button.

Tried against the PoC's seven nodes and Hugging Face's trending hundred:
63 shown, the rest too big for one 24 GB Turing card or FP8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-26 18:30:16 +01:00
1 parent e84631ce3a
commit 2a45068461
11 files changed
+458 -18

No files matched your search

+7 -1
View File
@@ -146,7 +146,13 @@ agent is stopped.
## 4. Add a model
Open `/admin/mirror` and choose **Explore models**. It lists what is popular on
Hugging Face, showing only what your nodes can run.
Hugging Face, showing only what your nodes can run: judged by the cluster as it
is now — the largest card and its compute capability, the most memory on one
node — so a model too big for them, in a form those cards lack (FP8 needs 8.9),
or in neither GGUF nor safetensors is left out, and the page says how many.
Without GPUs it lists GGUF only. Untick **only what these nodes can run** to
see everything. On a model's page a file too big for the cluster is shown
without a **Fetch** button, with what it would need.
1. Click a model to read its card.
2. Pick one of its files, usually a `q4_k_m` for a first try, and click