Ilker ManapandClaude Opus 5
b40bc481cf
CI / test (push) Successful in 1m36s
The mirror was a directory and a discipline written down in a README. This makes the three operations that matter into commands, on the mirror host, where `llmgw keys` already sets the precedent of operator tooling that works on files rather than through the API. The gateway still never writes to the mirror. That property was chosen deliberately — a mirror writable by the thing reading from it is a way to replace a model with one every node then fetches without complaint — and a person running a command does not take it away. `serve` states it rather than inheriting it, so a handler added later cannot quietly make the mirror writable. `add` downloads to a temporary name and renames, for the reason the agent's fetcher does: a mirror is copied to every node, so a partial file here is a partial file everywhere. It verifies the length the server promised, reports a checksum — the mirror is the one place a corrupt file reaches every node rather than being caught once — and refuses to replace an existing version, which is the whole point of the layout. `unused` refuses to answer when it cannot read the catalog. An empty catalog and an unreachable one produce the same list, everything, and the obvious next step after that list is deletion. It compares by path rather than by spelling, because a base typed with a different scheme or a trailing slash would otherwise report a file in use as unused. `remove` checks the catalog first and says to retire the entry before deleting the file, because a node that has to refetch a deleted file cannot, and that failure arrives later, on the node, looking like something else. Verified end to end: Range requests answer 206, which is what makes an interrupted fetch on a node resume; writes answer 405; a traversal attempt answers 404. Note: internal/agent/supervisor's TestSidecarRunsBeforeEngine fails under full suite load on main as well as here — it asserts that one shell's echo lands before another's, which the supervisor does not promise. Everything else passes. Left alone rather than fixed in a commit about mirrors. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
90 lines
3.0 KiB
Go
90 lines
3.0 KiB
Go
package cli
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"net/http"
|
|
"strings"
|
|
"time"
|
|
)
|
|
|
|
// checkSource asks whether a model's weights are actually there.
|
|
//
|
|
// A source that does not resolve is caught today by a replica failing to start,
|
|
// minutes later, on a node — and the error names the node rather than the typo.
|
|
// Asking now turns that into a sentence while the operator still has the file
|
|
// open.
|
|
//
|
|
// Only URLs. A path is checked on the node that will use it, by the node, which
|
|
// is the only machine whose answer means anything; a bare model id is resolved
|
|
// by the engine and is not this function's business.
|
|
func checkSource(ctx context.Context, source string) (size int64, err error) {
|
|
url, ok := httpSource(source)
|
|
if !ok {
|
|
return 0, nil
|
|
}
|
|
|
|
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
|
|
defer cancel()
|
|
|
|
req, err := http.NewRequestWithContext(ctx, http.MethodHead, url, nil)
|
|
if err != nil {
|
|
return 0, err
|
|
}
|
|
resp, err := http.DefaultClient.Do(req)
|
|
if err != nil {
|
|
return 0, fmt.Errorf("cannot reach %s: %w", url, err)
|
|
}
|
|
defer resp.Body.Close()
|
|
|
|
if resp.StatusCode != http.StatusOK {
|
|
return 0, fmt.Errorf("%s answered %s", url, resp.Status)
|
|
}
|
|
// Absent when the server does not send it, which is not an error: a
|
|
// resumable fetch does not need the length in advance, and refusing here
|
|
// would reject a mirror that is merely terse. Go reports that as -1;
|
|
// normalised to 0 so callers have one spelling of "unknown".
|
|
if resp.ContentLength < 0 {
|
|
return 0, nil
|
|
}
|
|
return resp.ContentLength, nil
|
|
}
|
|
|
|
// httpSource returns the URL to check, and whether there is one.
|
|
//
|
|
// hf: is expanded the way the fetcher expands it, so the check asks about the
|
|
// same file the download will ask for rather than about a string that looks
|
|
// similar.
|
|
func httpSource(source string) (string, bool) {
|
|
switch {
|
|
case strings.HasPrefix(source, "http://"), strings.HasPrefix(source, "https://"):
|
|
return source, true
|
|
case strings.HasPrefix(source, "hf:"):
|
|
rest := strings.TrimPrefix(source, "hf:")
|
|
parts := strings.SplitN(rest, "/", 3)
|
|
if len(parts) != 3 {
|
|
// A repository rather than a file. vLLM and SGLang download those
|
|
// themselves, so there is nothing here to check.
|
|
return "", false
|
|
}
|
|
return fmt.Sprintf("https://huggingface.co/%s/%s/resolve/main/%s",
|
|
parts[0], parts[1], parts[2]), true
|
|
default:
|
|
return "", false
|
|
}
|
|
}
|
|
|
|
// sourceNote is what to print about a checked source.
|
|
//
|
|
// It says where the answer came from, because the control plane's network is
|
|
// not the nodes' network: a mirror this machine can reach may be unreachable
|
|
// from a node behind a different firewall, and the reverse. The check catches
|
|
// typos and a mirror that is down, which is most of what goes wrong, and it
|
|
// should not be read as proof that the fetch will work.
|
|
func sourceNote(source string, size int64) string {
|
|
if size > 0 {
|
|
return fmt.Sprintf(" source reachable from here: %s (%s)", source, fileSize(size))
|
|
}
|
|
return fmt.Sprintf(" source reachable from here: %s (size not advertised)", source)
|
|
}
|