Weights from Hugging Face pinned to a commit and checked against the sha256 it published
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
29d7f54cfd
commit
b8ed368e31
22 files changed
+578
-16
No files matched your search
@@ -897,6 +897,24 @@ the node — an adapter given by its id, a TensorRT-LLM model's — is refused
|
||||
when the model is added, unless it says `allow_pickle`: mirror its
|
||||
safetensors and give that instead.
|
||||
|
||||
**Weights are pinned to a commit and checked.** A model added with
|
||||
`ctl model add`, the API or the console, whose weights come from Hugging Face,
|
||||
is pinned to the commit the repository is at when it is added:
|
||||
|
||||
- `hf:org/repo/file` and a `…/resolve/main/…` URL become that commit's URL;
|
||||
- a repository vLLM or SGLang downloads itself is given `--revision`;
|
||||
- the sha256 Hugging Face published for each file goes in the model's origin
|
||||
(its definition, `GET /api/v1/models/NAME`, shows it).
|
||||
|
||||
A node then refuses a copy of the file that is not it, naming both checksums,
|
||||
and deletes it; a cached copy is checked once and its checksum kept beside it.
|
||||
The mirror does the same for a model `ctl model serve` fetches, and
|
||||
`llmgw mirror add` for a Hugging Face URL (`--sha256` for any other). A new
|
||||
upload to the repository reaches the cluster only through `ctl model update`,
|
||||
which says what changed. Not pinned: models declared in the cluster's settings,
|
||||
which are not added this way, and a vLLM model whose arguments already say
|
||||
`--revision` keeps that one.
|
||||
|
||||
Put them where the node can read them. `scp`, a shared filesystem, or the
|
||||
engine's own downloader are all valid, and which you want depends on how many
|
||||
nodes you have.
|
||||
|
||||
Reference in new issue
Block a user