{{.Addr}}{{if .Unreachable}} did not answer{{end}}
{{template "nav" .Nav}}{{.Error}}
{{end}} {{if .Message}}{{.Message}}
{{end}} {{if eq .Tab "held"}} {{if .Unreachable}} {{/* Said outright rather than shown as an empty table. A mirror holding nothing and a mirror that is not there look the same in a list of zero rows, and they call for opposite responses. */}}Nothing was read from the mirror, so this page is not a list of
what it holds — it is not currently reachable at {{.Addr}}. Check
that llmgw-mirror is running there and that its management port is
reachable from this machine.
| Model | Files | Size | Runs | |
|---|---|---|---|---|
| {{.Family}} {{.Models}} models · {{bytes .Bytes}} {{if .Fetching}}downloading{{end}} {{if .Failed}}failed{{end}} {{if .InUse}}in use{{end}} | {{if eq (printf "%s" .State) "present"}}{{.Model}} {{else}}{{.Model}}{{end}} {{.Version}} |
{{if gt .Files 1}}{{.Files}} files{{else}}{{.File}}{{end}} |
{{if eq (printf "%s" .State) "fetching"}} {{if .HasPercent}}{{.Percent}}% of {{bytes .BytesTotal}} {{else}}{{/* No total came from upstream, so no percentage is invented — a bar that guesses its own end and then stops moving is worse than a number that only goes up. */}} {{bytes .BytesDone}} so far{{end}} {{else if eq (printf "%s" .State) "failed"}}— {{else}}{{bytes .BytesDone}}{{end}} |
{{if eq (printf "%s" .State) "fetching"}}downloading
{{else if eq (printf "%s" .State) "failed"}}failed
{{else if .UsedBy}}{{$row := .}}{{range .UsedBy}}
{{/* Named only when the catalog calls it something other than the
mirror does; otherwise it is the model column again. */}}
{{if ne .Name $row.Model}}as {{.Name}}{{end}}
{{if .Declared}}
{{- if eq .Mode "always"}}always on{{else if eq .Mode "ondemand"}}on demand{{else}}stopped{{end}} · yaml
{{else}}
{{/* Submitted on change: the choice is the action, and a second click
to confirm it would only be a way to forget to. */}}
{{end}}
{{if .Fleet}}
{{- if .Parked}}parked
{{- else if and (eq .Wanted 0) (eq .Ready 0)}}none running
{{- else}}{{.Ready}}/{{.Wanted}}{{if .Starting}} +{{.Starting}}{{end}}{{end}}
{{- if .Unplaceable}} · no room{{end}}
{{- with .Nodes}} · {{range $i, $n := .}}{{if $i}}, {{end}}{{$n.Short}}{{end}}{{end}}{{end}}
{{end}}
{{else}}not served{{end}}
|
{{if eq (printf "%s" .State) "failed"}} {{/* Retry, not remove. A fetch that failed left nothing on disk, so the remove would come back "the mirror holds nothing for this" — a button whose only outcome is a confusing error. Starting the same download again is what somebody looking at a failed row wants, and the mirror allows it: only a download still running is refused. */}} {{else if eq (printf "%s" .State) "present"}} {{if and $.HasCatalog (not .UsedBy)}}{{if .Serve}}close {{else}}serve{{end}}{{end}} {{end}} | {{if .Err}}
| {{.Err}} | ||||
| {{template "mirror_held_detail" dict "D" . "Row" $row0 "CSRF" $.CSRF}} | ||||
|
{{if .Refusal}}
{{/* Offered and then failing at placement is worse than not offered at all. */}}
Cannot be served as it stands: {{.Refusal}}. {{else}} {{if .Warning}}{{.Warning}}. {{end}}The context pool is what the engine is given; llama.cpp divides it across its parallel slots, so the number one request may use is the smaller one above. Memory: {{.MemoryBasis}}. Change the context pool and this changes with it, by the KV cache's share — an estimate, and editable. It is filled in rather than left blank because a CPU profile has no card to be short of: with no figure, every ready node is a candidate and the first one picked is whichever the scheduler reached, not whichever can hold the model. A node label does the same job by name if you would rather pin it. {{end}} | ||||
“copy address” gives what a catalog entry's source
should say — the file, or for a version of several files the directory, with its
trailing slash. It is rendered rather than assembled by hand because the mirror
and the catalog have to spell a file the same way, which is also how this page
knows which models use it.
The mirror is reachable and holds nothing yet. {{if .HasRegistry}}Explore Hugging Face for a model, or {{end}}fetch one by URL.
{{end}}The registry: {{.RegistryErr}}
{{end}} {{if not .Repo}}{{with .Explore}} {{if .Rows}}| Repository | Size | Architecture | Context | Downloads | Likes | Updated |
|---|---|---|---|---|---|---|
{{.ID}}
{{if .New}}new{{end}}{{if .Gated.Is}}gated{{end}}
{{if or .BaseModel .License}}{{with .BaseModel}}from {{.}}{{end}}{{if and .BaseModel .License}} · {{end}}{{.License}}{{end}} |
{{.ParamsLabel}} | {{if .Architecture}} |
{{.ContextLabel}} | {{compact .Downloads}} | {{compact .Likes}} | {{.Age}} |
{{.Hidden}} more {{if eq .Hidden 1}}is{{else}}are{{end}} hidden: their architecture is one the nodes' llama.cpp cannot load. Untick “only what these nodes can run” to see them.
{{end}}A gated repository needs its terms accepted with a Hugging Face account, and this browses anonymously — the files will list, and fetching one will fail at the mirror. Download it by hand and put it in the mirror directory, or give the mirror a token.
{{else if not $.RegistryErr}}{{if $.Query}}Nothing matched “{{$.Query}}”{{else}}Nothing to list{{end}}{{if .Hidden}} that these nodes can run — {{.Hidden}} {{if eq .Hidden 1}}result was{{else}}results were{{end}} hidden for an architecture llama.cpp cannot load{{end}}.
{{end}} {{end}}{{end}} {{if .Repo}}{{template "mirror_repo" .}}{{end}} {{end}} {{if eq .Tab "fetch"}}By hand, for anything the search does not reach.
The mirror answers as soon as the download is under way, not when it finishes — a model is tens of gigabytes and nothing here holds a connection open for that. The Held tab shows it arriving. A version is never overwritten: fetching into one that exists is refused, so a new build is a new version and the old weights stay where they are.
{{end}}| What | Size | Published checksum | |
|---|---|---|---|
{{.Name}}
{{if .Shards}}{{.Shards}} shards, fetched together{{end}} |
{{bytes .Bytes}} | {{with .SHA256}}{{slice . 0 12}}{{else}}—{{end}} |
{{if not .Verdict.Servable}} cannot be served {{.Verdict.Reason}} {{else}} {{if .Verdict.Architecture}}{{.Verdict.Architecture}}{{if .Verdict.Checked}} ✓{{end}}{{end}} {{end}} |
Only the weights are listed; a repository is mostly a README and a tokenizer. Shards of one file are one row and one button — a fifteen-shard model is one decision, not fifteen. Two separate files are two rows because they are usually alternatives: one quantisation or the other, not both.
The source URLs are pinned to
{{.Repo.ShortRevision}} rather than to a branch, so what arrives is
what is listed here even if the branch moves in between — and a single file's
checksum above is what the registry publishes, to compare with what the mirror
computes once it has it.
No weight files in {{.Repo.ID}}.
{{end}} {{end}} {{/* The card beneath a held version: what runs it, where, and how many. */}} {{define "mirror_held_detail"}}{{$row := .Row}}{{$csrf := .CSRF}}Not in the catalog: nothing runs this version. {{if $row.Serve}}{{else}}Use serve on its row to put it into service.{{end}}
{{end}} {{range .D.Models}}{{.}}
{{end}} {{if .Replicas}}| Replica | Node | State | Age | CPU | Memory |
|---|---|---|---|---|---|
{{if eq $i 0}}{{$rep.ID}}{{end}} |
{{$s.Node}}{{if $s.MemoryGB}} {{$s.MemoryGB}} GB{{end}} | {{if eq $i 0}}{{$rep.State}} {{with $rep.Restart}}restart {{.}}{{end}}{{end}} | {{if eq $i 0}}{{age $rep.Age}}{{end}} | {{if $s.CPU}}{{printf "%.1f" $s.CPU}} cores{{else}}—{{end}} | {{if $s.MemBytes}}{{bytes $s.MemBytes}}{{else}}—{{end}} |
No replica runs now{{if eq .Mode "ondemand"}}: it starts when somebody asks for it{{end}}.
{{end}} {{with .Headroom}}{{if .Limited}}Room for at least {{.More}} more replicas. {{else if .More}}Room for {{.More}} more now{{if .Nodes}}, on {{range $i, $n := .Nodes}}{{if $i}}, {{end}}{{join $n}}{{end}}{{end}}. {{else}}No room for another replica now.{{end}}
{{if and (not .Limited) .Blocked}}Set in server.yaml, which is read again on every start: change its replicas there.
{{else}} {{end}}