Fetch a sharded model in one action, not one shard at a time
The explorer listed every weight file with its own button. For a repository of alternatives that is right — q4_k_m or q8_0, you want one — and for a sharded model it is fifteen clicks to fetch one thing, with nothing stopping you from losing count and leaving a version with fourteen of its parts. Files are grouped into sets now. A run of "-00002-of-00004" is one row and one button; everything else stays its own row. Not "fetch all the weights", which would be wrong half the time: two safetensors files are halves of one thing and useless apart, two GGUF files are usually alternatives and fetching both spends gigabytes on a copy nobody asked for. The set is the unit that is whole. A repository can hold both the split and the joined copy of the same quantisation — the real Qwen coder repos do — so a shard set keeps the -of- notation in its name rather than collapsing onto the name of the whole file beside it. A set publishes no single checksum. Several files have several, and showing one of them invites comparing it against the wrong thing. The form posts every source in the set, and a failure part way says how many already started: a set that fails on its fourth shard has three arriving, and reading that as "nothing happened" sends somebody to fetch them all again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
ed530133b6
commit
a588646b7c
5 files changed
+277
-31
No files matched your search
@@ -120,18 +120,20 @@ directory, or give the mirror a token.</p>
|
||||
|
||||
{{if .Repo}}
|
||||
<h3>{{.Repo.ID}} <span class="opt">at {{.Repo.ShortRevision}}</span></h3>
|
||||
{{if .RepoFiles}}
|
||||
{{if .RepoSets}}
|
||||
<table>
|
||||
<tr><th>File</th><th>Size</th><th>Published checksum</th><th></th></tr>
|
||||
{{range .RepoFiles}}<tr>
|
||||
<td><code>{{.Path}}</code></td>
|
||||
<tr><th>What</th><th>Size</th><th>Published checksum</th><th></th></tr>
|
||||
{{range .RepoSets}}<tr>
|
||||
<td><code>{{.Name}}</code>
|
||||
{{if .Shards}}<span class="opt">{{.Shards}} shards, fetched together</span>{{end}}</td>
|
||||
<td class="num">{{bytes .Bytes}}</td>
|
||||
<td>{{if .SHA256}}<code title="{{.SHA256}}">{{slice .SHA256 0 12}}</code>{{else}}—{{end}}</td>
|
||||
<td>{{with .SHA256}}<code title="{{.}}">{{slice . 0 12}}</code>{{else}}—{{end}}</td>
|
||||
<td>
|
||||
<form method="post" action="/admin/mirror/fetch" class="inline">
|
||||
<input type="hidden" name="csrf" value="{{$.CSRF}}">
|
||||
<input type="hidden" name="source" value="{{.Source}}">
|
||||
{{range .Sources}}<input type="hidden" name="source" value="{{.URL}}">
|
||||
<input type="hidden" name="file" value="{{.Path}}">
|
||||
{{end}}
|
||||
<input type="text" name="model" value="{{.Model}}" size="24">
|
||||
<input type="text" name="version" value="{{.Version}}" size="10">
|
||||
<button type="submit">Fetch</button>
|
||||
@@ -140,10 +142,14 @@ directory, or give the mirror a token.</p>
|
||||
</tr>{{end}}
|
||||
</table>
|
||||
<p class="note">Only the weights are listed; a repository is mostly a README and
|
||||
a tokenizer. The source URL is pinned to <code>{{.Repo.ShortRevision}}</code>
|
||||
rather than to a branch, so what arrives is what is listed here even if the
|
||||
branch moves in between — and the checksum above is what the registry publishes,
|
||||
to compare with what the mirror computes once it has the file.</p>
|
||||
a tokenizer. Shards of one file are one row and one button — a fifteen-shard
|
||||
model is one decision, not fifteen. Two separate files are two rows because they
|
||||
are usually alternatives: one quantisation or the other, not both.</p>
|
||||
<p class="note">The source URLs are pinned to
|
||||
<code>{{.Repo.ShortRevision}}</code> rather than to a branch, so what arrives is
|
||||
what is listed here even if the branch moves in between — and a single file's
|
||||
checksum above is what the registry publishes, to compare with what the mirror
|
||||
computes once it has it.</p>
|
||||
{{else}}
|
||||
<p class="empty">No weight files in {{.Repo.ID}}.</p>
|
||||
{{end}}
|
||||
|
||||
Reference in new issue
Block a user