Serve a mirrored model from the list, instead of writing YAML for it

Fetching a model put it on the mirror and left the operator to hand-write a
catalogue entry and run `ctl model add` on the control plane — for a source the
page was already displaying, with an engine decided by the file extension it
had just shown them.

A present row now offers to serve it. The form opens on that row with the name,
the mirror's own URL and the engine filled in, and asks for the two things
nobody can infer: the context pool and how many replicas.

What it refuses is the point.

A version of several files is refused with the reason. A catalogue entry names
one source, so an entry for a sharded model is an entry for one shard, which
loads as a broken model rather than failing to load. Nothing said so before —
the mirror happily holds a four-shard model that no catalogue entry can
describe.

Safetensors on a fleet with no GPU is warned about before anything is added.
This is the failure we hit today: fifteen gigabytes of shards arrived, and only
then did looking at a node show no card, no vLLM and no SGLang. The control
plane is not told which engines a node has installed, so the check is on cards,
which it does know — a warning rather than a refusal, because a card can be
added tomorrow.

A name already declared in configuration is refused here too, matching what the
model service would say anyway, but before the form is filled in rather than
after.

Without a store there is no serve action at all: a model added at runtime would
not survive a restart, and offering it then refusing is worse than not offering.

The mirror page's dependencies became a struct. Three are optional interfaces
and two are the same shape, which at a call site is a swap nobody notices until
a page shows the wrong thing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-27 15:54:20 +02:00
1 parent a588646b7c
commit e034862b79
9 files changed
+539 -4

No files matched your search

+48
View File
@@ -68,6 +68,9 @@ reachable from this machine.</p>
<button type="submit" class="link">Retry</button>
</form>
{{else}}
{{if $.HasCatalog}}{{if .Serve}}<a href="/admin/mirror">close</a>
{{else}}<a href="/admin/mirror?serve={{.Model}}:{{.Version}}">serve</a>{{end}}
{{end}}
<form method="post" action="/admin/mirror/{{.Model}}/{{.Version}}/remove"
class="inline" onsubmit="return confirm('Remove {{.Model}}/{{.Version}} and every file in it?')">
<input type="hidden" name="csrf" value="{{$.CSRF}}">
@@ -78,6 +81,51 @@ reachable from this machine.</p>
</tr>
{{if .Err}}<tr><td colspan="7" class="error-row">{{.Err}}</td></tr>{{end}}
{{if not .Err}}<tr><td colspan="7" class="url-row"><code>{{.URL}}</code></td></tr>{{end}}
{{with .Serve}}
<tr class="editing"><td colspan="7">
{{if .Refusal}}
{{/* Offered and then failing at placement is worse than not offered at all. */}}
<p class="error">Cannot be served as it stands: {{.Refusal}}.</p>
{{else}}
{{if .Warning}}<p class="note error-row">{{.Warning}}.</p>{{end}}
<form method="post" action="/admin/mirror/serve">
<input type="hidden" name="csrf" value="{{$.CSRF}}">
<input type="hidden" name="source" value="{{.Source}}">
<input type="hidden" name="engine" value="{{.Engine}}">
<p class="sub">Serving <code>{{.Source}}</code> with <strong>{{.Engine}}</strong>.</p>
<div class="field-row">
<label>Name <input type="text" name="name" value="{{.Model}}" required></label>
<label>Aliases <span class="opt">comma separated</span>
<input type="text" name="aliases" placeholder="coder-7b"></label>
<label>Replicas <input type="text" name="replicas" value="1" required></label>
</div>
<div class="field-row">
<label>Context a request may use
<input type="text" name="context_length" value="4096" required></label>
<label>Output limit <span class="opt">optional</span>
<input type="text" name="max_output" value="1024"></label>
<label>Engine context pool <span class="opt">divided across slots</span>
<input type="text" name="ctx_size" value="16384"></label>
</div>
<div class="field-row">
<label>Profile name <input type="text" name="profile" value="{{.Profile}}"></label>
<label>Needs RAM (GB) <span class="opt">optional</span>
<input type="text" name="min_memory_gb" placeholder="8"></label>
<label>Only nodes labelled <span class="opt">key / value, optional</span>
<input type="text" name="selector_key" placeholder="role" size="8">
<input type="text" name="selector_value" placeholder="cpu-large" size="10"></label>
</div>
<button type="submit">Add to the catalog</button>
</form>
<p class="note">The context pool is what the engine is given; llama.cpp divides
it across its parallel slots, so the number one request may use is the smaller
one above. A node label keeps a large model off the small machines — a CPU
profile has no card to be short of, so without one every ready node looks like
a candidate and the first one picked is the one that runs out of memory.</p>
{{end}}
</td></tr>
{{end}}
{{end}}
</table>
<p class="note">The address under each row is what a catalog entry should say.