The explorer listed every weight file with its own button. For a repository of alternatives that is right — q4_k_m or q8_0, you want one — and for a sharded model it is fifteen clicks to fetch one thing, with nothing stopping you from losing count and leaving a version with fourteen of its parts. Files are grouped into sets now. A run of "-00002-of-00004" is one row and one button; everything else stays its own row. Not "fetch all the weights", which would be wrong half the time: two safetensors files are halves of one thing and useless apart, two GGUF files are usually alternatives and fetching both spends gigabytes on a copy nobody asked for. The set is the unit that is whole. A repository can hold both the split and the joined copy of the same quantisation — the real Qwen coder repos do — so a shard set keeps the -of- notation in its name rather than collapsing onto the name of the whole file beside it. A set publishes no single checksum. Several files have several, and showing one of them invites comparing it against the wrong thing. The form posts every source in the set, and a failure part way says how many already started: a set that fails on its fourth shard has three arriving, and reading that as "nothing happened" sends somebody to fetch them all again. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
183 lines
8.3 KiB
HTML
183 lines
8.3 KiB
HTML
{{define "admin_mirror"}}<!doctype html>
|
|
<html lang="en"><head>
|
|
<meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1">
|
|
{{/* Only while something is downloading. A reload at any other time would
|
|
take with it whatever the operator was typing into the form below. */}}
|
|
{{if .RefreshSec}}<meta http-equiv="refresh" content="{{.RefreshSec}}">{{end}}
|
|
<title>Mirror · llmgw</title><link rel="stylesheet" href="/_static/base.css">
|
|
</head><body><div class="wrap">
|
|
<header>
|
|
<h1>Model mirror</h1>
|
|
<p class="sub">
|
|
{{if .Unreachable}}{{.Addr}} did not answer{{else}}{{.Present}} held ·
|
|
{{.Fetching}} downloading · {{.Failed}} failed · {{.Addr}}{{end}}
|
|
</p>
|
|
<nav><a href="/admin">cluster</a> <a href="/admin/users">accounts</a>
|
|
<a href="/profile">your keys</a></nav>
|
|
</header>
|
|
|
|
{{if .Error}}<p class="error">{{.Error}}</p>{{end}}
|
|
{{if .Message}}<p class="ok">{{.Message}}</p>{{end}}
|
|
|
|
{{if .Unreachable}}
|
|
{{/* Said outright rather than shown as an empty table. A mirror holding
|
|
nothing and a mirror that is not there look the same in a list of zero
|
|
rows, and they call for opposite responses. */}}
|
|
<p class="note">Nothing was read from the mirror, so this page is not a list of
|
|
what it holds — it is not currently reachable at <code>{{.Addr}}</code>. Check
|
|
that <code>llmgw-mirror</code> is running there and that its management port is
|
|
reachable from this machine.</p>
|
|
{{else}}
|
|
|
|
<h2>What the mirror holds</h2>
|
|
{{if .Rows}}
|
|
<table>
|
|
<tr><th>Model</th><th>Version</th><th>File</th><th>State</th><th>Size</th>
|
|
<th>Checksum</th><th></th></tr>
|
|
{{range .Rows}}<tr>
|
|
<td>{{.Model}}</td>
|
|
<td>{{.Version}}</td>
|
|
<td><code>{{.File}}</code></td>
|
|
<td class="state state-{{.State}}">{{.State}}</td>
|
|
<td class="num">
|
|
{{if eq (printf "%s" .State) "fetching"}}
|
|
{{if .HasPercent}}{{bytes .BytesDone}} / {{bytes .BytesTotal}} ({{.Percent}}%)
|
|
{{else}}{{/* No total came from upstream, so no percentage is invented —
|
|
a bar that guesses its own end and then stops moving is worse
|
|
than a number that only goes up. */}}
|
|
{{bytes .BytesDone}} so far{{end}}
|
|
{{else if eq (printf "%s" .State) "failed"}}—
|
|
{{else}}{{bytes .BytesDone}}{{end}}
|
|
</td>
|
|
<td>{{if .SHA256}}<code title="{{.SHA256}}">{{slice .SHA256 0 12}}</code>{{else}}—{{end}}</td>
|
|
<td>
|
|
{{if eq (printf "%s" .State) "fetching"}}
|
|
<span class="note">downloading</span>
|
|
{{else if eq (printf "%s" .State) "failed"}}
|
|
{{/* Retry, not remove. A fetch that failed left nothing on disk, so the
|
|
remove would come back "the mirror holds nothing for this" — a button
|
|
whose only outcome is a confusing error. Starting the same download
|
|
again is what somebody looking at a failed row wants, and the mirror
|
|
allows it: only a download still running is refused. */}}
|
|
<form method="post" action="/admin/mirror/fetch" class="inline">
|
|
<input type="hidden" name="csrf" value="{{$.CSRF}}">
|
|
<input type="hidden" name="source" value="{{.Source}}">
|
|
<input type="hidden" name="model" value="{{.Model}}">
|
|
<input type="hidden" name="version" value="{{.Version}}">
|
|
<input type="hidden" name="file" value="{{.File}}">
|
|
<button type="submit" class="link">Retry</button>
|
|
</form>
|
|
{{else}}
|
|
<form method="post" action="/admin/mirror/{{.Model}}/{{.Version}}/remove"
|
|
class="inline" onsubmit="return confirm('Remove {{.Model}}/{{.Version}} and every file in it?')">
|
|
<input type="hidden" name="csrf" value="{{$.CSRF}}">
|
|
<button type="submit" class="link danger">Remove</button>
|
|
</form>
|
|
{{end}}
|
|
</td>
|
|
</tr>
|
|
{{if .Err}}<tr><td colspan="7" class="error-row">{{.Err}}</td></tr>{{end}}
|
|
{{if not .Err}}<tr><td colspan="7" class="url-row"><code>{{.URL}}</code></td></tr>{{end}}
|
|
{{end}}
|
|
</table>
|
|
<p class="note">The address under each row is what a catalog entry should say.
|
|
It is rendered here rather than assembled by hand because the mirror and the
|
|
catalog have to spell the same file the same way — two spellings is what makes a
|
|
file that is in use look unreferenced.</p>
|
|
{{else}}<p class="empty">The mirror is reachable and holds nothing yet.</p>{{end}}
|
|
|
|
{{if .HasRegistry}}
|
|
<h2>Find a model</h2>
|
|
<form method="get" action="/admin/mirror">
|
|
<label>Search Hugging Face
|
|
<input type="text" name="q" value="{{.Query}}" placeholder="qwen2.5 coder gguf"></label>
|
|
<button type="submit">Search</button>
|
|
</form>
|
|
|
|
{{if .RegistryErr}}
|
|
{{/* Its own line, not the page's error: the registry is somebody else's server
|
|
and it being down says nothing about the mirror, which is what this page is
|
|
for and has to keep working when Hugging Face does not. */}}
|
|
<p class="note error-row">The registry: {{.RegistryErr}}</p>
|
|
{{end}}
|
|
|
|
{{if .Results}}
|
|
<table><tr><th>Repository</th><th>Downloads</th><th>Task</th><th></th></tr>
|
|
{{range .Results}}<tr>
|
|
<td><code>{{.ID}}</code>{{if .Gated.Is}} <span class="opt">gated</span>{{end}}</td>
|
|
<td class="num">{{compact .Downloads}}</td>
|
|
<td>{{or .PipelineTag "—"}}</td>
|
|
<td><a href="/admin/mirror?q={{$.Query}}&repo={{.ID}}">files</a></td>
|
|
</tr>{{end}}
|
|
</table>
|
|
<p class="note">A gated repository needs its terms accepted with a Hugging Face
|
|
account, and this browses anonymously — the files will list, and fetching one
|
|
will fail at the mirror. Download it by hand and put it in the mirror
|
|
directory, or give the mirror a token.</p>
|
|
{{else if .Query}}{{if not .RegistryErr}}
|
|
<p class="empty">Nothing matched “{{.Query}}”.</p>
|
|
{{end}}{{end}}
|
|
|
|
{{if .Repo}}
|
|
<h3>{{.Repo.ID}} <span class="opt">at {{.Repo.ShortRevision}}</span></h3>
|
|
{{if .RepoSets}}
|
|
<table>
|
|
<tr><th>What</th><th>Size</th><th>Published checksum</th><th></th></tr>
|
|
{{range .RepoSets}}<tr>
|
|
<td><code>{{.Name}}</code>
|
|
{{if .Shards}}<span class="opt">{{.Shards}} shards, fetched together</span>{{end}}</td>
|
|
<td class="num">{{bytes .Bytes}}</td>
|
|
<td>{{with .SHA256}}<code title="{{.}}">{{slice . 0 12}}</code>{{else}}—{{end}}</td>
|
|
<td>
|
|
<form method="post" action="/admin/mirror/fetch" class="inline">
|
|
<input type="hidden" name="csrf" value="{{$.CSRF}}">
|
|
{{range .Sources}}<input type="hidden" name="source" value="{{.URL}}">
|
|
<input type="hidden" name="file" value="{{.Path}}">
|
|
{{end}}
|
|
<input type="text" name="model" value="{{.Model}}" size="24">
|
|
<input type="text" name="version" value="{{.Version}}" size="10">
|
|
<button type="submit">Fetch</button>
|
|
</form>
|
|
</td>
|
|
</tr>{{end}}
|
|
</table>
|
|
<p class="note">Only the weights are listed; a repository is mostly a README and
|
|
a tokenizer. Shards of one file are one row and one button — a fifteen-shard
|
|
model is one decision, not fifteen. Two separate files are two rows because they
|
|
are usually alternatives: one quantisation or the other, not both.</p>
|
|
<p class="note">The source URLs are pinned to
|
|
<code>{{.Repo.ShortRevision}}</code> rather than to a branch, so what arrives is
|
|
what is listed here even if the branch moves in between — and a single file's
|
|
checksum above is what the registry publishes, to compare with what the mirror
|
|
computes once it has it.</p>
|
|
{{else}}
|
|
<p class="empty">No weight files in {{.Repo.ID}}.</p>
|
|
{{end}}
|
|
{{end}}
|
|
{{end}}
|
|
|
|
<h2>Fetch a model</h2>
|
|
<p class="note">By hand, for anything the search does not reach.</p>
|
|
<form method="post" action="/admin/mirror/fetch">
|
|
<input type="hidden" name="csrf" value="{{.CSRF}}">
|
|
<label>Source URL
|
|
<input type="text" name="source" required
|
|
placeholder="https://huggingface.co/…/resolve/main/model.gguf"></label>
|
|
<div class="field-row">
|
|
<label>Model <input type="text" name="model" required placeholder="qwen2.5-coder-7b"></label>
|
|
<label>Version <input type="text" name="version" required placeholder="2026-09"></label>
|
|
<label>File <span class="opt">optional</span>
|
|
<input type="text" name="file" placeholder="taken from the URL"></label>
|
|
</div>
|
|
<button type="submit">Start download</button>
|
|
</form>
|
|
<p class="note">The mirror answers as soon as the download is under way, not when
|
|
it finishes — a model is tens of gigabytes and nothing here holds a connection
|
|
open for that. The row above will show it arriving. A version is never
|
|
overwritten: fetching into one that exists is refused, so a new build is a new
|
|
version and the old weights stay where they are.</p>
|
|
{{end}}
|
|
|
|
<footer><span>llmgw {{.Version}}</span><span>{{.Now}}</span></footer>
|
|
</div></body></html>{{end}}
|