Refuse weights the nodes' llama.cpp cannot load, before fetching them
kev-0.5b declared architecture "ggmlc", which llama.cpp does not know. It was fetched, catalogued and placed, and failed on every node — while the answer was in the first kilobytes of the file. internal/gguf reads a GGUF header (v1-v3) up to general.architecture, over HTTP with a Range request, and judges it against the architectures llama.cpp b10991 lists in LLM_ARCH_NAMES (152, copied from the source at that tag); a vision projector is not a model on its own. An unknown build is not refused, only not vouched for, and a header that cannot be fetched does not refuse anything. Checked in three places: the registry listing (from the architecture Hugging Face reports with expand[]=gguf, so one request per repository) shows it and withholds the fetch button; a fetch reads the header and refuses unless "fetch anyway"; and the model service refuses to add such a model, which covers the serve form and ctl model add. Tried on real files: the mirror's qwen2.5-1.5b (qwen2) and bge-m3 (bert), and Qwen2.5-0.5B on Hugging Face through its redirect. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
c609805e4b
commit
f13b18149a
12 files changed
+782
-3
No files matched your search
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -60,7 +61,7 @@ type serveForm struct {
|
||||
|
||||
// planServe works out what serving this version would mean.
|
||||
func planServe(st mirror.Status, files []mirror.Status, base string,
|
||||
fleet Source, cat CatalogAdmin,
|
||||
fleet Source, cat CatalogAdmin, check func(source string) gguf.Verdict,
|
||||
) serveForm {
|
||||
f := serveForm{
|
||||
Model: st.Model,
|
||||
@@ -99,6 +100,12 @@ func planServe(st mirror.Status, files []mirror.Status, base string,
|
||||
return f
|
||||
case "llamacpp":
|
||||
f.Profile = "cpu"
|
||||
if check != nil {
|
||||
if v := check(f.Source); !v.Servable {
|
||||
f.Refusal = "llama.cpp on the nodes cannot run this: " + v.Reason
|
||||
return f
|
||||
}
|
||||
}
|
||||
default:
|
||||
f.Profile = "gpu"
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user