Serve LoRA adapters on llama.cpp models, asked for by name
Release / build (push) Successful in 1m11s
Release / notes (push) Successful in 3s
CI / test (push) Successful in 3m47s

A model's adapters are fetched by each node with its weights and loaded
unapplied at start; a request naming an adapter reaches the model's
replicas with llama.cpp's per-request lora field. Adapter names share the
model namespace, are listed, and follow allowlists on their own. The
router's config seed and pushed catalog now build the name map in one place.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-30 23:40:47 +02:00
1 parent 94eec557ad
commit cf53904cc5
18 files changed
+542 -41

No files matched your search

+5
View File
@@ -31,6 +31,11 @@ func (a *API) Orphans() []OrphanView {
var wanted []string
for _, m := range a.ctl.Catalog().Models() {
wanted = append(wanted, m.Source)
// A model's adapters are its weights too; collecting them would
// leave replicas that cannot restart.
for _, ad := range m.Adapters {
wanted = append(wanted, ad.Source)
}
}
out := make([]OrphanView, 0)