A GGUF is servable when a llama.cpp that engine_overrides give some nodes loads it
CI / test (push) Failing after 9m58s

`ctl model serve`, its plan and model updates judged a file only against
the cluster's own llama.cpp, so a GPU node given a newer build could not
be served an architecture only that build knows: Inkling 2 Mini
(inkling2_loop) was refused for b10991 with cuda01 on b12606.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-21 13:43:40 +01:00
1 parent 0ffd633bf6
commit 09ebe77513
8 files changed
+99 -10

No files matched your search

+1 -1
View File
@@ -679,7 +679,7 @@ func newServerCmd() *cobra.Command {
if adminAPI != nil && catalogSvc != nil {
serve := modelserve.New(modelserve.Deps{
Registry: registry, Mirror: mc, Catalog: catalogSvc, Fleet: admin,
LlamaVersion: cfg.WantedEngines()["llamacpp"], Log: log,
LlamaVersion: cfg.WantedEngines()["llamacpp"], LlamaOverrides: cfg.OverrideVersions("llamacpp"), Log: log,
// Updates: the catalog in force, the service that
// changes it, and what replaces replicas.
Current: func() []catalog.Model { return admin.Catalog().Models() },