Attach LoRA adapters from the command line
llmgw ctl model adapters lists a model's adapters and adds or removes
them through POST /api/v1/models/{name}/adapters, which reads each new
file first and refuses one that is not an adapter for the model's
architecture, as the mirror page's card does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
6b1fa65f59
commit
323d172538
6 files changed
+218
-5
No files matched your search
@@ -694,6 +694,12 @@ follow key allowlists on their own: a key allowed `coder-sql` reaches that
|
||||
adapter and not the model under it, while a key allowed the model reaches its
|
||||
adapters too. Adding or removing an adapter is a change to the model; running
|
||||
replicas load the new set when restarted (`llmgw ctl restart --model M`).
|
||||
Besides the mirror page's model card, adapters are attached from the command
|
||||
line, which reads the file first and refuses one that is not an adapter for
|
||||
this model's architecture:
|
||||
|
||||
llmgw ctl model adapters coder --add coder-sql=http://mirror:8081/sql-lora/v1/sql.gguf@0.8
|
||||
llmgw ctl model adapters coder --remove coder-sql
|
||||
|
||||
The adapter file must be a GGUF LoRA made for this model — llama.cpp's
|
||||
`convert_lora_to_gguf.py` makes one from a PEFT adapter — and not a model with
|
||||
|
||||
Reference in new issue
Block a user