Refuse weights the nodes' llama.cpp cannot load, before fetching them
kev-0.5b declared architecture "ggmlc", which llama.cpp does not know. It was fetched, catalogued and placed, and failed on every node — while the answer was in the first kilobytes of the file. internal/gguf reads a GGUF header (v1-v3) up to general.architecture, over HTTP with a Range request, and judges it against the architectures llama.cpp b10991 lists in LLM_ARCH_NAMES (152, copied from the source at that tag); a vision projector is not a model on its own. An unknown build is not refused, only not vouched for, and a header that cannot be fetched does not refuse anything. Checked in three places: the registry listing (from the architecture Hugging Face reports with expand[]=gguf, so one request per repository) shows it and withholds the fetch button; a fetch reads the header and refuses unless "fetch anyway"; and the model service refuses to add such a model, which covers the serve form and ctl model add. Tried on real files: the mirror's qwen2.5-1.5b (qwen2) and bge-m3 (bert), and Qwen2.5-0.5B on Hugging Face through its redirect. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
c609805e4b
commit
f13b18149a
12 files changed
+782
-3
No files matched your search
@@ -46,6 +46,10 @@ type Service struct {
|
||||
apply Applier
|
||||
log *slog.Logger
|
||||
|
||||
// check refuses a model that cannot run before it is added; see
|
||||
// SetCheck.
|
||||
check func(context.Context, catalog.Model) error
|
||||
|
||||
// mu serialises rebuilds. They are rare — a person adding a model — and two
|
||||
// at once could apply catalogs in either order, leaving the one in force
|
||||
// disagreeing with the database that outlives it.
|
||||
@@ -100,6 +104,11 @@ func (s *Service) Add(ctx context.Context, m catalog.Model, by string) error {
|
||||
if s.Declared(m.Name) {
|
||||
return fmt.Errorf("model %q is declared in configuration; edit that rather than adding a second definition", m.Name)
|
||||
}
|
||||
if s.check != nil {
|
||||
if err := s.check(ctx, m); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
stored, err := s.storedModels(ctx)
|
||||
if err != nil {
|
||||
@@ -299,3 +308,8 @@ func (s *Service) storedModels(ctx context.Context) ([]catalog.Model, error) {
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// SetCheck runs before a model is added, and refuses it by returning an error.
|
||||
// Used to catch weights the engine cannot load, which otherwise are placed and
|
||||
// then fail on every node they are tried on.
|
||||
func (s *Service) SetCheck(check func(context.Context, catalog.Model) error) { s.check = check }
|
||||
Reference in new issue
Block a user