{{define "admin_mirror"}} {{/* Only while something is downloading. A reload at any other time would take with it whatever the operator was typing into the form below. */}} {{if .RefreshSec}}{{end}} Mirror · llmgw

Model mirror

{{if .Unreachable}}{{.Addr}} did not answer{{else}}{{.Present}} held · {{.Fetching}} downloading · {{.Failed}} failed · {{.Addr}}{{end}}

{{if .Error}}

{{.Error}}

{{end}} {{if .Message}}

{{.Message}}

{{end}} {{if .Unreachable}} {{/* Said outright rather than shown as an empty table. A mirror holding nothing and a mirror that is not there look the same in a list of zero rows, and they call for opposite responses. */}}

Nothing was read from the mirror, so this page is not a list of what it holds — it is not currently reachable at {{.Addr}}. Check that llmgw-mirror is running there and that its management port is reachable from this machine.

{{else}}

What the mirror holds

{{if .Rows}} {{range .Rows}} {{if .Err}}{{end}} {{if not .Err}}{{end}} {{with .Serve}} {{end}} {{end}}
ModelVersionFileStateSize Checksum
{{.Model}} {{.Version}} {{.File}} {{.State}} {{if eq (printf "%s" .State) "fetching"}} {{if .HasPercent}}{{bytes .BytesDone}} / {{bytes .BytesTotal}} ({{.Percent}}%) {{else}}{{/* No total came from upstream, so no percentage is invented — a bar that guesses its own end and then stops moving is worse than a number that only goes up. */}} {{bytes .BytesDone}} so far{{end}} {{else if eq (printf "%s" .State) "failed"}}— {{else}}{{bytes .BytesDone}}{{end}} {{if .SHA256}}{{slice .SHA256 0 12}}{{else}}—{{end}} {{if eq (printf "%s" .State) "fetching"}} downloading {{else if eq (printf "%s" .State) "failed"}} {{/* Retry, not remove. A fetch that failed left nothing on disk, so the remove would come back "the mirror holds nothing for this" — a button whose only outcome is a confusing error. Starting the same download again is what somebody looking at a failed row wants, and the mirror allows it: only a download still running is refused. */}}
{{else}} {{if $.HasCatalog}}{{if .Serve}}close {{else}}serve{{end}} {{end}}
{{end}}
{{.Err}}
{{.URL}}
{{if .Refusal}} {{/* Offered and then failing at placement is worse than not offered at all. */}}

Cannot be served as it stands: {{.Refusal}}.

{{else}} {{if .Warning}}

{{.Warning}}.

{{end}}

Serving {{.Source}} with {{.Engine}}.

{{/* Reaches clients through /v1/models, which for most of them is the only view of what this gateway serves. Nine names and nothing to choose between them is a choice made by guesswork. */}}

The context pool is what the engine is given; llama.cpp divides it across its parallel slots, so the number one request may use is the smaller one above.

The memory figure is the weights plus half again for the KV pool plus two for the engine — an estimate, and editable. It is filled in rather than left blank because a CPU profile has no card to be short of: with no figure, every ready node is a candidate and the first one picked is whichever the scheduler reached, not whichever can hold the model. A node label does the same job by name if you would rather pin it.

{{end}}

The address under each row is what a catalog entry should say. It is rendered here rather than assembled by hand because the mirror and the catalog have to spell the same file the same way — two spellings is what makes a file that is in use look unreferenced.

{{else}}

The mirror is reachable and holds nothing yet.

{{end}} {{if .HasRegistry}}

Find a model

{{if .RegistryErr}} {{/* Its own line, not the page's error: the registry is somebody else's server and it being down says nothing about the mirror, which is what this page is for and has to keep working when Hugging Face does not. */}}

The registry: {{.RegistryErr}}

{{end}} {{if .Results}} {{range .Results}}{{end}}
RepositoryDownloadsTask
{{.ID}}{{if .Gated.Is}} gated{{end}} {{compact .Downloads}} {{or .PipelineTag "—"}} files

A gated repository needs its terms accepted with a Hugging Face account, and this browses anonymously — the files will list, and fetching one will fail at the mirror. Download it by hand and put it in the mirror directory, or give the mirror a token.

{{else if .Query}}{{if not .RegistryErr}}

Nothing matched “{{.Query}}”.

{{end}}{{end}} {{if .Repo}}

{{.Repo.ID}} at {{.Repo.ShortRevision}}

{{if .RepoSets}} {{range .RepoSets}}{{end}}
WhatSizePublished checksum
{{.Name}} {{if .Shards}}{{.Shards}} shards, fetched together{{end}} {{bytes .Bytes}} {{with .SHA256}}{{slice . 0 12}}{{else}}—{{end}}
{{range .Sources}} {{end}}

Only the weights are listed; a repository is mostly a README and a tokenizer. Shards of one file are one row and one button — a fifteen-shard model is one decision, not fifteen. Two separate files are two rows because they are usually alternatives: one quantisation or the other, not both.

The source URLs are pinned to {{.Repo.ShortRevision}} rather than to a branch, so what arrives is what is listed here even if the branch moves in between — and a single file's checksum above is what the registry publishes, to compare with what the mirror computes once it has it.

{{else}}

No weight files in {{.Repo.ID}}.

{{end}} {{end}} {{end}}

Fetch a model

By hand, for anything the search does not reach.

The mirror answers as soon as the download is under way, not when it finishes — a model is tens of gigabytes and nothing here holds a connection open for that. The row above will show it arriving. A version is never overwritten: fetching into one that exists is refused, so a new build is a new version and the old weights stay where they are.

{{end}}
{{end}}