Manage the model mirror with a command instead of a runbook
CI / test (push) Successful in 1m36s

The mirror was a directory and a discipline written down in a README. This
makes the three operations that matter into commands, on the mirror host,
where `llmgw keys` already sets the precedent of operator tooling that works
on files rather than through the API.

The gateway still never writes to the mirror. That property was chosen
deliberately — a mirror writable by the thing reading from it is a way to
replace a model with one every node then fetches without complaint — and a
person running a command does not take it away. `serve` states it rather than
inheriting it, so a handler added later cannot quietly make the mirror
writable.

`add` downloads to a temporary name and renames, for the reason the agent's
fetcher does: a mirror is copied to every node, so a partial file here is a
partial file everywhere. It verifies the length the server promised, reports a
checksum — the mirror is the one place a corrupt file reaches every node
rather than being caught once — and refuses to replace an existing version,
which is the whole point of the layout.

`unused` refuses to answer when it cannot read the catalog. An empty catalog
and an unreachable one produce the same list, everything, and the obvious next
step after that list is deletion. It compares by path rather than by spelling,
because a base typed with a different scheme or a trailing slash would
otherwise report a file in use as unused.

`remove` checks the catalog first and says to retire the entry before deleting
the file, because a node that has to refetch a deleted file cannot, and that
failure arrives later, on the node, looking like something else.

Verified end to end: Range requests answer 206, which is what makes an
interrupted fetch on a node resume; writes answer 405; a traversal attempt
answers 404.

Note: internal/agent/supervisor's TestSidecarRunsBeforeEngine fails under full
suite load on main as well as here — it asserts that one shell's echo lands
before another's, which the supervisor does not promise. Everything else
passes. Left alone rather than fixed in a commit about mirrors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-27 09:03:07 +02:00
1 parent e86457cf02
commit b40bc481cf
9 files changed
+1059

No files matched your search

+20
View File
@@ -31,6 +31,7 @@ func newModelCmd(client func() *ctlClient) *cobra.Command {
func newModelAddCmd(client func() *ctlClient) *cobra.Command {
var path string
var skipCheck bool
c := &cobra.Command{
Use: "add -f <file.yaml>",
Short: "Add a model from a definition file",
@@ -61,6 +62,23 @@ func newModelAddCmd(client func() *ctlClient) *cobra.Command {
if m.Name == "" {
return fmt.Errorf("%s: the definition has no name", path)
}
// Before the catalog is touched, because a source that does not
// resolve is otherwise discovered by a replica failing to start
// minutes later on a node — with an error naming the node rather
// than the typo.
if !skipCheck {
size, err := checkSource(cmd.Context(), m.Source)
if err != nil {
return fmt.Errorf("%w\n Pass --no-source-check to add it anyway, "+
"which is right when the mirror is not up yet or the nodes "+
"reach it by an address this machine does not.", err)
}
if _, isURL := httpSource(m.Source); isURL {
fmt.Fprintln(cmd.OutOrStdout(), sourceNote(m.Source, size))
}
}
body, err := json.Marshal(m)
if err != nil {
return err
@@ -77,6 +95,8 @@ func newModelAddCmd(client func() *ctlClient) *cobra.Command {
},
}
c.Flags().StringVarP(&path, "file", "f", "", "YAML file holding the model definition")
c.Flags().BoolVar(&skipCheck, "no-source-check", false,
"add it without asking whether the weights are actually at that URL")
return c
}