Serve LoRA adapters on llama.cpp models, asked for by name
A model's adapters are fetched by each node with its weights and loaded unapplied at start; a request naming an adapter reaches the model's replicas with llama.cpp's per-request lora field. Adapter names share the model namespace, are listed, and follow allowlists on their own. The router's config seed and pushed catalog now build the name map in one place. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
94eec557ad
commit
cf53904cc5
18 files changed
+542
-41
No files matched your search
@@ -31,6 +31,11 @@ func (a *API) Orphans() []OrphanView {
|
||||
var wanted []string
|
||||
for _, m := range a.ctl.Catalog().Models() {
|
||||
wanted = append(wanted, m.Source)
|
||||
// A model's adapters are its weights too; collecting them would
|
||||
// leave replicas that cannot restart.
|
||||
for _, ad := range m.Adapters {
|
||||
wanted = append(wanted, ad.Source)
|
||||
}
|
||||
}
|
||||
|
||||
out := make([]OrphanView, 0)
|
||||
|
||||
Reference in new issue
Block a user