Weights from Hugging Face pinned to a commit and checked against the sha256 it published

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-30 07:33:15 +01:00
1 parent 29d7f54cfd
commit b8ed368e31
22 files changed
+578 -16

No files matched your search

+18
View File
@@ -897,6 +897,24 @@ the node — an adapter given by its id, a TensorRT-LLM model's — is refused
when the model is added, unless it says `allow_pickle`: mirror its
safetensors and give that instead.
**Weights are pinned to a commit and checked.** A model added with
`ctl model add`, the API or the console, whose weights come from Hugging Face,
is pinned to the commit the repository is at when it is added:
- `hf:org/repo/file` and a `…/resolve/main/…` URL become that commit's URL;
- a repository vLLM or SGLang downloads itself is given `--revision`;
- the sha256 Hugging Face published for each file goes in the model's origin
(its definition, `GET /api/v1/models/NAME`, shows it).
A node then refuses a copy of the file that is not it, naming both checksums,
and deletes it; a cached copy is checked once and its checksum kept beside it.
The mirror does the same for a model `ctl model serve` fetches, and
`llmgw mirror add` for a Hugging Face URL (`--sha256` for any other). A new
upload to the repository reaches the cluster only through `ctl model update`,
which says what changed. Not pinned: models declared in the cluster's settings,
which are not added this way, and a vLLM model whose arguments already say
`--revision` keeps that one.
Put them where the node can read them. `scp`, a shared filesystem, or the
engine's own downloader are all valid, and which you want depends on how many
nodes you have.