Attach LoRA adapters from the command line

llmgw ctl model adapters lists a model's adapters and adds or removes
them through POST /api/v1/models/{name}/adapters, which reads each new
file first and refuses one that is not an adapter for the model's
architecture, as the mirror page's card does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

.gitea/workflows/release.yml

@@ -93,6 +93,24 @@ jobs:
           ./dist/llmgw-${version}-linux-amd64 --version
           cat dist/checksums.txt
 
+      # Signed when the repository has a signing key, so a machine that trusts
+      # it installs nothing the server alone could have put there. Without the
+      # secret nothing is signed and releases are checked on their checksums,
+      # as before; a build that trusts a key refuses those, so the secret has
+      # to be in place before trusted.pub is filled in.
+      - name: Sign the checksums
+        env:
+          RELEASE_SIGNING_KEY: ${{ secrets.RELEASE_SIGNING_KEY }}
+        run: |
+          set -euo pipefail
+          version="${GITHUB_REF_NAME:-$(git describe --tags --always)}"
+          if [ -z "${RELEASE_SIGNING_KEY:-}" ]; then
+            echo "no RELEASE_SIGNING_KEY secret; releases are not signed"
+            exit 0
+          fi
+          ./dist/llmgw-${version}-linux-amd64 release sign dist/checksums.txt
+          echo "signed: dist/checksums.txt.sig"
+
       - name: Attach to the release
         env:
           TOKEN: ${{ secrets.GITEA_TOKEN }}

CHANGELOG.md

@@ -20,6 +20,108 @@ is only released once its **exit criterion** is met, not when its code lands.
 Until v1.0.0 the minor version is the breaking one: configuration formats and Go
 APIs change between minors without a deprecation period.
 
+## Unreleased
+
+Changes since v0.10.5, published as release candidates (v0.10.6-rc.N) until
+there are enough for v0.10.6.
+
+### Capacity from the admin pages
+
+What v0.10.5 made settable from the command line is now on the pages too:
+
+- **Quotas** has a Capacity section: every scope's guaranteed and most GPUs,
+  cores and memory beside what it holds, with a form to set, change or remove
+  one. It goes through the same checks as `llmgw ctl quota`.
+- **A model's card** on the mirror page sets its class, its owner (when there
+  are quotas to count against), and the memory and cores a replica reserves,
+  beside its replica counts. Only what changed is applied, and each change is
+  said.
+- **Room counts what a class may take.** The card, `llmgw ctl model target` and
+  the headroom API count the replicas a model could get by stopping those its
+  class outranks, separately from free room, and name whose they would be. A
+  critical model's card no longer refuses a replica that preemption would make
+  room for.
+
+### LoRA adapters
+
+A llama.cpp model can carry `adapters`, each a name and a GGUF LoRA file. The
+nodes fetch them with the model's weights and llama.cpp loads them all at start,
+applied to nothing (`--lora-init-without-apply`); a request that names an
+adapter goes to the model's replicas with `"lora": [{"id": i, "scale": s}]`,
+so one replica serves the model and all its adapters. Adapter names share the
+namespace of models, are listed in `/v1/models`, and follow key allowlists on
+their own — a key allowed an adapter does not reach the model under it. Adapter
+files are kept from weight cleanup like the model's own. Not yet for vLLM or
+SGLang profiles, whose adapters are loaded differently.
+
+### Finding and attaching adapters from the mirror page
+
+- **Explore** has a *LoRA adapters* view: adapters for any model, or — from a
+  held model's card, *find adapters on Hugging Face* — those made for the model
+  its repository was made from, since a GGUF repository is a quantisation and
+  adapters are made for the original.
+- **Opening a repository** reads the first bytes of each GGUF file and marks it
+  a *LoRA adapter* or a *model*. Repositories tagged "lora" hold a model with
+  the adapter merged in about as often as an adapter, and only the file says.
+- **Held adapter files** are marked in the list, and a model's card lists its
+  adapters, offers the held adapter files of its architecture to attach under a
+  name and scale, and removes them. A file that is not an adapter, or is for
+  another architecture, is refused before it reaches the catalog.
+- **`llmgw ctl model adapters MODEL [--add NAME=URL[@SCALE]] [--remove NAME]`**
+  lists and changes them from the command line, through
+  `POST /api/v1/models/{name}/adapters`, with the same checks.
+
+### Models served outside the cluster
+
+A catalog model can be an OpenAI-compatible API elsewhere: `external: {url,
+model, api_key_file}` and no profiles. Nothing is placed for it; the router
+sends its requests to the provider, rewritten to the provider's name for the
+model and with its key, read from the file on every publish so a rotated key is
+picked up. Named in another model's `fallbacks`, it is used only when the
+cluster has no capacity left. It is never scraped for metrics, and never reads
+as stale. Prices, budgets, limits, usage and key allowlists apply as to any
+model — and since its prompts leave the cluster, the allowlist is how to decide
+who may send them (docs/install.md). `ctl models` and the cluster page mark it
+external.
+
+### Pool spillover
+
+A model served from pools of different priority — H100 before A100, say — was
+routed to the preferred pool however full it was. Now a pool whose replicas are
+saturated, by the same measure admission uses (queue past its healthy depth, or
+KV cache past its ceiling), passes requests to the next pool that is not. When
+every pool is saturated the preferred one is kept: a pool that is just as full
+gains nothing.
+
+### Displacement: a higher band is not refused while a lower one waits
+
+Each band's queue was bounded, and nothing bounded them together, so a full
+gateway turned an interactive request away while batch requests sat queued
+behind it. `admission.queue_capacity_total` now bounds everything queued —
+by default half of every band's capacity together. Reached, a request is not
+refused while a lower band has any queued: the newest request of the lowest
+band is turned away instead, with a 503 and `Retry-After: 5`, so it backs off
+and returns. The newest, because the oldest have waited longest and are
+nearest their TTL. Counted as `displaced` in the admission rejection metric.
+
+### Releases can be signed
+
+`llmgw release keygen` makes an ed25519 key pair. With the private key in the
+CI secret `RELEASE_SIGNING_KEY`, releases carry `checksums.txt.sig`; with the
+public key in `internal/release/trusted.pub`, every binary built after refuses
+a release not signed with it, wherever it installs one. Off until then, and
+docs/install.md says in which order to turn it on.
+
+### Restarts borrow a lower class's room
+
+A restart places the replacement beside the replica it replaces, and where
+there is no free room it now makes some the way any placement does: by
+stopping a replica its class outranks, which gets its room back once the old
+replica has gone. The in-place restart — the model unserved while its
+replacement loads — is left for when nothing of a lower class is running. On a
+cluster where every model is standard, that is every full one: mark the models
+that can wait `opportunistic` (`llmgw ctl model class`, or the model's card).
+
 ## v0.10.5 — 2026-09-30
 
 Published first as release candidates v0.10.5-rc.1 to rc.14; rc.5 failed its

api/gen/llmgw/v1/agent.pb.go

@@ -2873,8 +2873,11 @@ type EngineProfile struct {
 	Env             map[string]string      `protobuf:"bytes,9,rep,name=env,proto3" json:"env,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
 	ModelPath       string                 `protobuf:"bytes,10,opt,name=model_path,json=modelPath,proto3" json:"model_path,omitempty"`
 	ServedModelName string                 `protobuf:"bytes,11,opt,name=served_model_name,json=servedModelName,proto3" json:"served_model_name,omitempty"`
-	unknownFields   protoimpl.UnknownFields
-	sizeCache       protoimpl.SizeCache
+	// adapters are LoRA adapters loaded beside the model, in order: the index
+	// is the id a request applies one by.
+	Adapters      []*LoraAdapter `protobuf:"bytes,12,rep,name=adapters,proto3" json:"adapters,omitempty"`
+	unknownFields protoimpl.UnknownFields
+	sizeCache     protoimpl.SizeCache
 }
 
 func (x *EngineProfile) Reset() {
@@ -2984,6 +2987,13 @@ func (x *EngineProfile) GetServedModelName() string {
 	return ""
 }
 
+func (x *EngineProfile) GetAdapters() []*LoraAdapter {
+	if x != nil {
+		return x.Adapters
+	}
+	return nil
+}
+
 // RemoveWeights deletes cached model files a node no longer needs.
 //
 // The paths are computed by the control plane from the catalog, but the agent
@@ -3777,6 +3787,60 @@ func (x *UpgradeReply) GetError() string {
 	return ""
 }
 
+// LoraAdapter is one adapter: the name clients ask for it by, and its source,
+// which the agent fetches as it does the model's weights.
+type LoraAdapter struct {
+	state         protoimpl.MessageState `protogen:"open.v1"`
+	Name          string                 `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
+	Source        string                 `protobuf:"bytes,2,opt,name=source,proto3" json:"source,omitempty"`
+	unknownFields protoimpl.UnknownFields
+	sizeCache     protoimpl.SizeCache
+}
+
+func (x *LoraAdapter) Reset() {
+	*x = LoraAdapter{}
+	mi := &file_llmgw_v1_agent_proto_msgTypes[36]
+	ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+	ms.StoreMessageInfo(mi)
+}
+
+func (x *LoraAdapter) String() string {
+	return protoimpl.X.MessageStringOf(x)
+}
+
+func (*LoraAdapter) ProtoMessage() {}
+
+func (x *LoraAdapter) ProtoReflect() protoreflect.Message {
+	mi := &file_llmgw_v1_agent_proto_msgTypes[36]
+	if x != nil {
+		ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
+		if ms.LoadMessageInfo() == nil {
+			ms.StoreMessageInfo(mi)
+		}
+		return ms
+	}
+	return mi.MessageOf(x)
+}
+
+// Deprecated: Use LoraAdapter.ProtoReflect.Descriptor instead.
+func (*LoraAdapter) Descriptor() ([]byte, []int) {
+	return file_llmgw_v1_agent_proto_rawDescGZIP(), []int{36}
+}
+
+func (x *LoraAdapter) GetName() string {
+	if x != nil {
+		return x.Name
+	}
+	return ""
+}
+
+func (x *LoraAdapter) GetSource() string {
+	if x != nil {
+		return x.Source
+	}
+	return ""
+}
+
 var File_llmgw_v1_agent_proto protoreflect.FileDescriptor
 
 const file_llmgw_v1_agent_proto_rawDesc = "" +
@@ -4066,7 +4130,7 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"profile_fp\x18\x0f \x01(\tR\tprofileFp\x12\x17\n" +
 	"\aapi_key\x18\x10 \x01(\tR\x06apiKey\x12\x1f\n" +
 	"\vcpu_threads\x18\x11 \x01(\x05R\n" +
-	"cpuThreads\"\xfd\x02\n" +
+	"cpuThreads\"\xb0\x03\n" +
 	"\rEngineProfile\x12\x12\n" +
 	"\x04name\x18\x01 \x01(\tR\x04name\x12\x16\n" +
 	"\x06engine\x18\x02 \x01(\tR\x06engine\x12\x1b\n" +
@@ -4080,7 +4144,8 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"\n" +
 	"model_path\x18\n" +
 	" \x01(\tR\tmodelPath\x12*\n" +
-	"\x11served_model_name\x18\v \x01(\tR\x0fservedModelName\x1a6\n" +
+	"\x11served_model_name\x18\v \x01(\tR\x0fservedModelName\x121\n" +
+	"\badapters\x18\f \x03(\v2\x15.llmgw.v1.LoraAdapterR\badapters\x1a6\n" +
 	"\bEnvEntry\x12\x10\n" +
 	"\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
 	"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"%\n" +
@@ -4143,7 +4208,10 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"\n" +
 	"request_id\x18\x01 \x01(\tR\trequestId\x12\x18\n" +
 	"\aversion\x18\x02 \x01(\tR\aversion\x12\x14\n" +
-	"\x05error\x18\x03 \x01(\tR\x05error2O\n" +
+	"\x05error\x18\x03 \x01(\tR\x05error\"9\n" +
+	"\vLoraAdapter\x12\x12\n" +
+	"\x04name\x18\x01 \x01(\tR\x04name\x12\x16\n" +
+	"\x06source\x18\x02 \x01(\tR\x06source2O\n" +
 	"\fAgentService\x12?\n" +
 	"\aConnect\x12\x16.llmgw.v1.AgentMessage\x1a\x18.llmgw.v1.ControlMessage(\x010\x01B\x93\x01\n" +
 	"\fcom.llmgw.v1B\n" +
@@ -4162,7 +4230,7 @@ func file_llmgw_v1_agent_proto_rawDescGZIP() []byte {
 }
 
 var file_llmgw_v1_agent_proto_enumTypes = make([]protoimpl.EnumInfo, 1)
-var file_llmgw_v1_agent_proto_msgTypes = make([]protoimpl.MessageInfo, 42)
+var file_llmgw_v1_agent_proto_msgTypes = make([]protoimpl.MessageInfo, 43)
 var file_llmgw_v1_agent_proto_goTypes = []any{
 	(ShardStatus_State)(0),    // 0: llmgw.v1.ShardStatus.State
 	(*AgentMessage)(nil),      // 1: llmgw.v1.AgentMessage
@@ -4201,12 +4269,13 @@ var file_llmgw_v1_agent_proto_goTypes = []any{
 	(*FabricTestReply)(nil),   // 34: llmgw.v1.FabricTestReply
 	(*UpgradeAgent)(nil),      // 35: llmgw.v1.UpgradeAgent
 	(*UpgradeReply)(nil),      // 36: llmgw.v1.UpgradeReply
-	nil,                       // 37: llmgw.v1.Register.LabelsEntry
-	nil,                       // 38: llmgw.v1.Register.EngineVersionsEntry
-	nil,                       // 39: llmgw.v1.GPU.NicPathsEntry
-	nil,                       // 40: llmgw.v1.IBPort.CountersEntry
-	nil,                       // 41: llmgw.v1.Heartbeat.EngineVersionsEntry
-	nil,                       // 42: llmgw.v1.EngineProfile.EnvEntry
+	(*LoraAdapter)(nil),       // 37: llmgw.v1.LoraAdapter
+	nil,                       // 38: llmgw.v1.Register.LabelsEntry
+	nil,                       // 39: llmgw.v1.Register.EngineVersionsEntry
+	nil,                       // 40: llmgw.v1.GPU.NicPathsEntry
+	nil,                       // 41: llmgw.v1.IBPort.CountersEntry
+	nil,                       // 42: llmgw.v1.Heartbeat.EngineVersionsEntry
+	nil,                       // 43: llmgw.v1.EngineProfile.EnvEntry
 }
 var file_llmgw_v1_agent_proto_depIdxs = []int32{
 	3,  // 0: llmgw.v1.AgentMessage.register:type_name -> llmgw.v1.Register
@@ -4226,24 +4295,24 @@ var file_llmgw_v1_agent_proto_depIdxs = []int32{
 	31, // 14: llmgw.v1.ControlMessage.fabric_scan:type_name -> llmgw.v1.FabricScanRequest
 	33, // 15: llmgw.v1.ControlMessage.fabric_test:type_name -> llmgw.v1.FabricTestRequest
 	35, // 16: llmgw.v1.ControlMessage.upgrade:type_name -> llmgw.v1.UpgradeAgent
-	37, // 17: llmgw.v1.Register.labels:type_name -> llmgw.v1.Register.LabelsEntry
+	38, // 17: llmgw.v1.Register.labels:type_name -> llmgw.v1.Register.LabelsEntry
 	7,  // 18: llmgw.v1.Register.gpus:type_name -> llmgw.v1.GPU
 	8,  // 19: llmgw.v1.Register.fabrics:type_name -> llmgw.v1.FabricAttachment
-	38, // 20: llmgw.v1.Register.engine_versions:type_name -> llmgw.v1.Register.EngineVersionsEntry
+	39, // 20: llmgw.v1.Register.engine_versions:type_name -> llmgw.v1.Register.EngineVersionsEntry
 	6,  // 21: llmgw.v1.Register.running_shards:type_name -> llmgw.v1.RunningShard
 	9,  // 22: llmgw.v1.Register.weights:type_name -> llmgw.v1.LocalWeights
 	5,  // 23: llmgw.v1.Register.storage:type_name -> llmgw.v1.Storage
 	4,  // 24: llmgw.v1.Register.host:type_name -> llmgw.v1.Host
 	15, // 25: llmgw.v1.Register.imex:type_name -> llmgw.v1.IMEX
 	0,  // 26: llmgw.v1.RunningShard.state:type_name -> llmgw.v1.ShardStatus.State
-	39, // 27: llmgw.v1.GPU.nic_paths:type_name -> llmgw.v1.GPU.NicPathsEntry
+	40, // 27: llmgw.v1.GPU.nic_paths:type_name -> llmgw.v1.GPU.NicPathsEntry
 	12, // 28: llmgw.v1.NodeLoad.shards:type_name -> llmgw.v1.ShardLoad
 	11, // 29: llmgw.v1.NodeLoad.gpus:type_name -> llmgw.v1.GPUSensor
-	40, // 30: llmgw.v1.IBPort.counters:type_name -> llmgw.v1.IBPort.CountersEntry
+	41, // 30: llmgw.v1.IBPort.counters:type_name -> llmgw.v1.IBPort.CountersEntry
 	14, // 31: llmgw.v1.IBPort.subnet_managers:type_name -> llmgw.v1.SubnetManager
 	5,  // 32: llmgw.v1.Heartbeat.storage:type_name -> llmgw.v1.Storage
 	9,  // 33: llmgw.v1.Heartbeat.weights:type_name -> llmgw.v1.LocalWeights
-	41, // 34: llmgw.v1.Heartbeat.engine_versions:type_name -> llmgw.v1.Heartbeat.EngineVersionsEntry
+	42, // 34: llmgw.v1.Heartbeat.engine_versions:type_name -> llmgw.v1.Heartbeat.EngineVersionsEntry
 	20, // 35: llmgw.v1.Heartbeat.provision:type_name -> llmgw.v1.ProvisionStatus
 	10, // 36: llmgw.v1.Heartbeat.load:type_name -> llmgw.v1.NodeLoad
 	13, // 37: llmgw.v1.Heartbeat.ib_ports:type_name -> llmgw.v1.IBPort
@@ -4254,14 +4323,15 @@ var file_llmgw_v1_agent_proto_depIdxs = []int32{
 	0,  // 42: llmgw.v1.ShardStatus.state:type_name -> llmgw.v1.ShardStatus.State
 	22, // 43: llmgw.v1.ShardStatus.fetch:type_name -> llmgw.v1.FetchProgress
 	24, // 44: llmgw.v1.StartShard.profile:type_name -> llmgw.v1.EngineProfile
-	42, // 45: llmgw.v1.EngineProfile.env:type_name -> llmgw.v1.EngineProfile.EnvEntry
-	1,  // 46: llmgw.v1.AgentService.Connect:input_type -> llmgw.v1.AgentMessage
-	2,  // 47: llmgw.v1.AgentService.Connect:output_type -> llmgw.v1.ControlMessage
-	47, // [47:48] is the sub-list for method output_type
-	46, // [46:47] is the sub-list for method input_type
-	46, // [46:46] is the sub-list for extension type_name
-	46, // [46:46] is the sub-list for extension extendee
-	0,  // [0:46] is the sub-list for field type_name
+	43, // 45: llmgw.v1.EngineProfile.env:type_name -> llmgw.v1.EngineProfile.EnvEntry
+	37, // 46: llmgw.v1.EngineProfile.adapters:type_name -> llmgw.v1.LoraAdapter
+	1,  // 47: llmgw.v1.AgentService.Connect:input_type -> llmgw.v1.AgentMessage
+	2,  // 48: llmgw.v1.AgentService.Connect:output_type -> llmgw.v1.ControlMessage
+	48, // [48:49] is the sub-list for method output_type
+	47, // [47:48] is the sub-list for method input_type
+	47, // [47:47] is the sub-list for extension type_name
+	47, // [47:47] is the sub-list for extension extendee
+	0,  // [0:47] is the sub-list for field type_name
 }
 
 func init() { file_llmgw_v1_agent_proto_init() }
@@ -4296,7 +4366,7 @@ func file_llmgw_v1_agent_proto_init() {
 			GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
 			RawDescriptor: unsafe.Slice(unsafe.StringData(file_llmgw_v1_agent_proto_rawDesc), len(file_llmgw_v1_agent_proto_rawDesc)),
 			NumEnums:      1,
-			NumMessages:   42,
+			NumMessages:   43,
 			NumExtensions: 0,
 			NumServices:   1,
 		},

api/proto/llmgw/v1/agent.proto

@@ -572,6 +572,9 @@ message EngineProfile {
   map<string, string> env = 9;
   string model_path = 10;
   string served_model_name = 11;
+  // adapters are LoRA adapters loaded beside the model, in order: the index
+  // is the id a request applies one by.
+  repeated LoraAdapter adapters = 12;
 }
 
 // RemoveWeights deletes cached model files a node no longer needs.
@@ -704,3 +707,10 @@ message UpgradeReply {
   string version = 2;
   string error = 3;
 }
+
+// LoraAdapter is one adapter: the name clients ask for it by, and its source,
+// which the agent fetches as it does the model's weights.
+message LoraAdapter {
+  string name = 1;
+  string source = 2;
+}

docs/install.md

@@ -671,6 +671,74 @@ llmgw ctl explain-placement qwen2.5-coder-3b-instruct
 which names every candidate and why each was rejected, rather than leaving you
 to infer it.
 
+### LoRA adapters
+
+A llama.cpp model can carry LoRA adapters, each asked for by its own name as
+though it were a model:
+
+```yaml
+- name: qwen2.5-coder-7b-instruct
+  source: http://mirror:8081/qwen2.5-coder-7b-instruct/5d567d26/model.gguf
+  adapters:
+    - name: coder-sql
+      source: http://mirror:8081/coder-sql/v1/adapter.gguf
+      scale: 1.0          # optional; 1 when left out
+  profiles: [...]
+```
+
+Every replica of the model fetches every adapter and loads it at start, applied
+to nothing. A request for `coder-sql` goes to the model's replicas with the
+adapter applied to it alone; a request for the model gets the model. Adapter
+names share the namespace of models and aliases, appear in `/v1/models`, and
+follow key allowlists on their own: a key allowed `coder-sql` reaches that
+adapter and not the model under it, while a key allowed the model reaches its
+adapters too. Adding or removing an adapter is a change to the model; running
+replicas load the new set when restarted (`llmgw ctl restart --model M`).
+Besides the mirror page's model card, adapters are attached from the command
+line, which reads the file first and refuses one that is not an adapter for
+this model's architecture:
+
+    llmgw ctl model adapters coder --add coder-sql=http://mirror:8081/sql-lora/v1/sql.gguf@0.8
+    llmgw ctl model adapters coder --remove coder-sql
+
+The adapter file must be a GGUF LoRA made for this model — llama.cpp's
+`convert_lora_to_gguf.py` makes one from a PEFT adapter — and not a model with
+the adapter already merged in, which is what most "LoRA GGUF" repositories hold.
+
+### Models served outside the cluster
+
+A model can be an OpenAI-compatible API elsewhere — a hosted provider, or
+another gateway — rather than replicas here. Nothing is placed for it; the
+router sends its requests on, with the provider's key and the provider's name
+for the model:
+
+```yaml
+- name: cloud-mini
+  source: external
+  external:
+    url: https://api.openai.com          # without /v1
+    model: gpt-4o-mini                   # what the provider calls it
+    api_key_file: /etc/llmgw/secrets/openai.key
+```
+
+Named in another model's `fallbacks`, it is used only when the cluster has no
+capacity left for that model:
+
+```yaml
+- name: qwen2.5-coder-7b-instruct
+  fallbacks: [cloud-mini]
+  ...
+```
+
+The key is read from its file on the control plane, never stored in the
+catalog, and a key rotated in the file is used within seconds. A model whose key
+file cannot be read is not offered. Everything else applies as to any model:
+prices and budgets, rate limits, usage records, and key allowlists.
+
+**Its prompts leave the cluster.** Decide who may send them: a key or project
+whose allowlist leaves the external model out cannot reach it, as a fallback or
+by name. The cluster page and `ctl models` mark it `external`.
+
 ### How many replicas, and how much each one carries
 
 Replica count is changed while the cluster runs, without editing the
@@ -966,6 +1034,33 @@ replica alone on purpose, because churning one costs more than the difference.
 `llmgw ctl restart <replica>` replaces it with one on the new profile, starting
 the new one before stopping the old.
 
+### Signed releases
+
+Every install above checks a release against its published checksums, which
+proves the download is what the server holds — not that the server holds what
+was built. Signing closes that: CI signs the checksums, and a binary that
+trusts a key installs only releases signed with it. Off until a key exists;
+turning it on, once:
+
+```
+llmgw release keygen
+```
+
+1. Put the **private** key in the repository's Actions secret
+   `RELEASE_SIGNING_KEY`. From the next tag, releases carry
+   `checksums.txt.sig`.
+2. Put the **public** key in `internal/release/trusted.pub` and release. Every
+   binary built from then on carries it and refuses releases that are not
+   signed with it — `ctl upgrade`, agents upgrading themselves, and
+   `llmgw upgrade` alike.
+
+In that order: a build that trusts a key refuses unsigned releases, so the
+secret has to be signing before the key is trusted. The first trusting release
+is itself installed by a binary that trusts nothing; every one after is checked
+by the one before. `LLMGW_RELEASE_KEY` overrides the built-in key, for rotating
+it. The onboarding script fetches with `curl` and checks checksums only; a node
+is checked from its first upgrade on.
+
 ## Backing up
 
 `llmgw server backup` writes one archive holding everything a control plane

docs/reference.md

@@ -15,12 +15,16 @@ rather than repeated.
 
 | Key | Type | In |
 |---|---|---|
+| `name` | `string` | [Adapter](#adapter) |
+| `source` | `string` | [Adapter](#adapter) |
+| `scale` | `float64` | [Adapter](#adapter) |
 | `enabled` | `bool` | [Admission](#admission) |
 | `bands` | `[]Band` | [Admission](#admission) |
 | `default_band` | `string` | [Admission](#admission) |
 | `band_header` | `string` | [Admission](#admission) |
 | `queue_ttl` | `time.Duration` | [Admission](#admission) |
 | `queue_capacity_per_band` | `int` | [Admission](#admission) |
+| `queue_capacity_total` | `int` | [Admission](#admission) |
 | `interval` | `time.Duration` | [Admission](#admission) |
 | `enabled` | `bool` | [Auth](#auth) |
 | `store` | `string` | [Auth](#auth) |
@@ -77,6 +81,9 @@ rather than repeated.
 | `pool` | `string` | [Endpoint](#endpoint) |
 | `weight` | `float64` | [Endpoint](#endpoint) |
 | `priority` | `int` | [Endpoint](#endpoint) |
+| `url` | `string` | [External](#external) |
+| `model` | `string` | [External](#external) |
+| `api_key_file` | `string` | [External](#external) |
 | `contact` | `string` | [Landing](#landing) |
 | `notice` | `string` | [Landing](#landing) |
 | `bin` | `string` | [LlamaCppConfig](#llamacppconfig) |
@@ -94,6 +101,8 @@ rather than repeated.
 | `scaling` | `Scaling` | [Model](#model) |
 | `class` | `string` | [Model](#model) |
 | `owner` | `string` | [Model](#model) |
+| `external` | `*External` | [Model](#model) |
+| `adapters` | `[]Adapter` | [Model](#model) |
 | `fallbacks` | `[]string` | [Model](#model) |
 | `routing_profile` | `string` | [Model](#model) |
 | `slo` | `SLO` | [Model](#model) |
@@ -195,6 +204,27 @@ rather than repeated.
 | `launch_mode` | `string` | [VLLMConfig](#vllmconfig) |
 | `ray_bin` | `string` | [VLLMConfig](#vllmconfig) |
 
+## Adapter
+
+### `name`
+
+`string`
+
+Name is what clients ask for, like a model name.
+
+### `source`
+
+`string`
+
+Source is the adapter file (GGUF for llama.cpp), fetched by each node
+as the model's weights are.
+
+### `scale`
+
+`float64`
+
+Scale is how strongly it is applied; zero takes 1.
+
 ## Admission
 
 ### `enabled`
@@ -228,6 +258,15 @@ should not be able to promote themselves.
 
 `int`
 
+### `queue_capacity_total`
+
+`int`
+
+QueueCapacityTotal bounds everything queued across bands. Reached, a
+request is not refused while a lower band has any queued: the newest of
+the lowest band is turned away instead, with a 503 and Retry-After.
+Zero takes half of all the bands' capacities together.
+
 ### `interval`
 
 `time.Duration`
@@ -606,6 +645,28 @@ called. Empty means vLLM's names.
 
 `int`
 
+## External
+
+### `url`
+
+`string`
+
+URL is the provider's base address, without /v1: https://api.openai.com.
+
+### `model`
+
+`string`
+
+Model is what the provider calls it; requests are rewritten to it.
+
+### `api_key_file`
+
+`string`
+
+APIKeyFile holds the provider's key, on the control plane. A file
+rather than the key itself, so the key is in neither the catalog, the
+store nor the audit log; empty for a provider that asks for none.
+
 ## Landing
 
 ### `contact`
@@ -755,6 +816,28 @@ Owner is the part of the tenancy tree whose capacity quota this model
 counts against, written org:ID, team:ID or project:ID. Empty is shared:
 held by the cluster, counted against nobody.
 
+### `external`
+
+`*External`
+
+External is a model served by someone else — an OpenAI-compatible API
+outside the cluster — rather than by replicas placed here. Such a model
+has no profiles and nothing is placed for it; the router sends its
+requests to the provider. Most useful named in another model's
+fallbacks: used only when the cluster has no capacity left. Its
+prompts leave the cluster, so which keys may name it is worth an
+allowlist.
+
+Takes a [External](#external) block.
+
+### `adapters`
+
+`[]Adapter`
+
+Adapters are LoRA adapters served on this model's replicas, each asked
+for by its own name as though it were a model. Every replica loads
+every adapter, applying one only to a request that names it.
+
 ### `fallbacks`
 
 `[]string`

internal/agent/agent.go

@@ -891,6 +891,17 @@ func (a *Agent) startShard(ctx context.Context, m *llmgwv1.StartShard, out chan<
 		stop()
 		spec.Profile.ModelPath = path
 	}
+	// Adapters are fetched like the model, and before the engine starts: it
+	// loads them at start, and one missing is a replica that answers for the
+	// base model under the adapter's name.
+	for i, ad := range spec.Profile.Adapters {
+		path, err := a.weights().Ensure(ctx, ad.Path)
+		if err != nil {
+			report(supervisor.Failed, fmt.Sprintf("adapter %s: %v", ad.Name, err), nil)
+			return
+		}
+		spec.Profile.Adapters[i].Path = path
+	}
 
 	cmds, err := eng.Commands(spec)
 	if err != nil {

internal/agent/engines/engine.go

@@ -40,6 +40,9 @@ type Profile struct {
 	ModelPath string
 	// ServedModelName is the name clients use, which need not match the path.
 	ServedModelName string
+	// Adapters are LoRA adapters to load beside the model, in order: an
+	// adapter's index is the id requests apply it by.
+	Adapters []Adapter
 }
 
 // ShardSpec is one shard of one replica on this node.
@@ -373,3 +376,10 @@ func quantizationOf(args []string) string {
 	}
 	return ""
 }
+
+// Adapter is a LoRA adapter an engine loads beside its model. Path is a
+// source until the agent has fetched it, a local file after.
+type Adapter struct {
+	Name string `json:"name"`
+	Path string `json:"path"`
+}

internal/agent/engines/llamacpp.go

@@ -144,6 +144,15 @@ func (l *LlamaCpp) Commands(s ShardSpec) ([]Command, error) {
 		}
 		args = append(args, "--threads", strconv.Itoa(n))
 	}
+	// Loaded at start, applied per request: a request that names an adapter
+	// carries "lora": [{"id": i, "scale": s}], and one that names none is the
+	// base model, since every adapter starts at scale zero.
+	if len(p.Adapters) > 0 {
+		args = append(args, "--lora-init-without-apply")
+		for _, a := range p.Adapters {
+			args = append(args, "--lora", a.Path)
+		}
+	}
 	args = append(args, p.Args...)
 
 	env := baseEnv(s)

internal/agent/engines/llamacpp_test.go

@@ -306,3 +306,20 @@ func TestTheCoresPlacementReservedAreTheThreads(t *testing.T) {
 		t.Errorf("--threads %q; what placement reserved should win over the node's setting", v)
 	}
 }
+
+func TestAdaptersAreLoadedAtStartAndAppliedToNoneByDefault(t *testing.T) {
+	s := llamaSpec(nil)
+	s.Profile.Adapters = []Adapter{{Name: "sql", Path: "/w/sql.gguf"}, {Name: "docs", Path: "/w/docs.gguf"}}
+	cmds, err := (&LlamaCpp{}).Commands(s)
+	if err != nil {
+		t.Fatal(err)
+	}
+	args := strings.Join(cmds[0].Args, " ")
+	if !strings.Contains(args, "--lora-init-without-apply --lora /w/sql.gguf --lora /w/docs.gguf") {
+		t.Errorf("args %s: both adapters, in order, loaded unapplied", args)
+	}
+	plain, _ := (&LlamaCpp{}).Commands(llamaSpec(nil))
+	if strings.Contains(strings.Join(plain[0].Args, " "), "--lora") {
+		t.Error("a model with no adapters was given lora flags")
+	}
+}

internal/cli/ctl.go

@@ -254,6 +254,8 @@ func modelStatus(m adminapi.ModelView) string {
 	switch {
 	case m.Parked:
 		return "parked"
+	case m.External != "":
+		return "external: " + m.External
 	case m.HeldFor != "":
 		return "gave room to " + m.HeldFor
 	case m.Unplaceable:

internal/cli/model.go

@@ -5,6 +5,9 @@ import (
 	"fmt"
 	"os"
 	"os/user"
+	"slices"
+	"strconv"
+	"strings"
 	"time"
 
 	"github.com/knadh/koanf/parsers/yaml"
@@ -31,7 +34,7 @@ func newModelCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
 	}
 	c.AddCommand(newModelAddCmd(client), newModelRemoveCmd(client), newModelPlanCmd(client),
 		newResourcesCmd(client), newClassCmd(client), newOwnerCmd(client),
-		newTargetCmd(client, asJSON),
+		newTargetCmd(client, asJSON), newAdaptersCmd(client),
 		newModelServeCmd(client))
 	return c
 }
@@ -264,3 +267,69 @@ func newClassCmd(client func() *ctlClient) *cobra.Command {
 		},
 	}
 }
+
+func newAdaptersCmd(client func() *ctlClient) *cobra.Command {
+	var add, remove []string
+	c := &cobra.Command{
+		Use:   "adapters <model> [--add NAME=URL[@SCALE]] [--remove NAME]",
+		Short: "List, add or remove a model's LoRA adapters",
+		Long: "An adapter is asked for by its NAME as though it were a model, and is served\n" +
+			"by this model's replicas with it applied. URL is the adapter file, usually in\n" +
+			"the mirror; it must be a GGUF LoRA adapter for this model's architecture,\n" +
+			"which is checked before it is attached. Replicas load a changed set when\n" +
+			"restarted: llmgw ctl restart --model MODEL.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			name := args[0]
+			var models []adminapi.ModelView
+			if err := client().get(cmd.Context(), "/api/v1/models", &models); err != nil {
+				return err
+			}
+			var current []catalog.Adapter
+			found := false
+			for _, m := range models {
+				if m.Name == name {
+					current, found = m.Adapters, true
+				}
+			}
+			if !found {
+				return fmt.Errorf("no model %q", name)
+			}
+			out := cmd.OutOrStdout()
+			if len(add) == 0 && len(remove) == 0 {
+				if len(current) == 0 {
+					fmt.Fprintf(out, "%s has no adapters\n", name)
+				}
+				for _, a := range current {
+					fmt.Fprintf(out, "%s\tscale %g\t%s\n", a.Name, a.EffectiveScale(), a.Source)
+				}
+				return nil
+			}
+			next := slices.DeleteFunc(slices.Clone(current), func(a catalog.Adapter) bool { return slices.Contains(remove, a.Name) })
+			for _, spec := range add {
+				n, rest, ok := strings.Cut(spec, "=")
+				if !ok || n == "" || rest == "" {
+					return fmt.Errorf("--add %q is not NAME=URL[@SCALE]", spec)
+				}
+				ad := catalog.Adapter{Name: n, Source: rest}
+				if i := strings.LastIndex(rest, "@"); i > 0 {
+					if s, err := strconv.ParseFloat(rest[i+1:], 64); err == nil {
+						ad.Source, ad.Scale = rest[:i], s
+					}
+				}
+				next = append(next, ad)
+			}
+			body, _ := json.Marshal(adminapi.AdaptersRequest{Adapters: next})
+			var res adminapi.AdaptersRequest
+			if err := client().post(cmd.Context(), "/api/v1/models/"+name+"/adapters", body, &res); err != nil {
+				return err
+			}
+			fmt.Fprintf(out, "%s serves %d adapter(s); restart its replicas to load them: llmgw ctl restart --model %s\n",
+				name, len(res.Adapters), name)
+			return nil
+		},
+	}
+	c.Flags().StringArrayVar(&add, "add", nil, "NAME=URL[@SCALE]; repeatable")
+	c.Flags().StringArrayVar(&remove, "remove", nil, "an adapter's name; repeatable")
+	return c
+}

internal/cli/quota.go

@@ -5,6 +5,7 @@ import (
 	"fmt"
 	"net/url"
 	"strconv"
+	"strings"
 
 	"github.com/spf13/cobra"
 
@@ -192,6 +193,9 @@ func newTargetCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
 			switch {
 			case v.More == 0:
 				fmt.Fprintln(out, "It already has them.")
+			case v.Fits && len(v.Preempts) > 0:
+				fmt.Fprintf(out, "%d more; there is room once replicas of %s are stopped, which its class may do.\n",
+					v.More, strings.Join(v.Preempts, ", "))
 			case v.Fits:
 				fmt.Fprintf(out, "%d more, and there is room for them.\n", v.More)
 			default:

internal/cli/release.go

@@ -0,0 +1,67 @@
+package cli
+
+import (
+	"errors"
+	"fmt"
+	"os"
+
+	"github.com/spf13/cobra"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/release"
+)
+
+func newReleaseCmd() *cobra.Command {
+	c := &cobra.Command{
+		Use:   "release",
+		Short: "Sign releases, and make the key they are signed with",
+	}
+	c.AddCommand(&cobra.Command{
+		Use:   "keygen",
+		Short: "Make a release signing key pair",
+		Long: "Prints a public key and a private key. The private key goes into the CI\n" +
+			"secret RELEASE_SIGNING_KEY and nowhere else; the public key into\n" +
+			"internal/release/trusted.pub, which every build from then on carries\n" +
+			"and checks releases against. Run once, on a machine you trust.",
+		Args: cobra.NoArgs,
+		RunE: func(cmd *cobra.Command, _ []string) error {
+			pub, priv, err := release.GenerateKey()
+			if err != nil {
+				return err
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "public:  %s\nprivate: %s\n", pub, priv)
+			fmt.Fprintln(cmd.ErrOrStderr(), "\nThe private key signs every release: put it in the CI secret "+
+				"RELEASE_SIGNING_KEY, then clear this terminal's scrollback.")
+			return nil
+		},
+	})
+	c.AddCommand(&cobra.Command{
+		Use:   "sign FILE",
+		Short: "Sign a release's checksums, writing FILE.sig",
+		Long: "Reads the private key from RELEASE_SIGNING_KEY, never from an argument,\n" +
+			"so it does not land in a shell history or a process list.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			key := os.Getenv("RELEASE_SIGNING_KEY")
+			if key == "" {
+				return errors.New("RELEASE_SIGNING_KEY is not set")
+			}
+			data, err := os.ReadFile(args[0])
+			if err != nil {
+				return err
+			}
+			sig, err := release.Sign(key, data)
+			if err != nil {
+				return err
+			}
+			if pub := release.TrustedKey(); pub != "" {
+				// Signed with a key this build would not accept is a release
+				// every upgraded machine refuses: stopped here instead.
+				if err := release.Verify(pub, data, sig); err != nil {
+					return fmt.Errorf("the signing key does not match this build's trusted key: %w", err)
+				}
+			}
+			return os.WriteFile(args[0]+".sig", []byte(sig+"\n"), 0o644) //nolint:gosec // public
+		},
+	})
+	return c
+}

internal/cli/root.go

@@ -48,6 +48,7 @@ func Root() *cobra.Command {
 		newDoctorCmd(),
 		newNodeCmd(),
 		newSelfUpgradeCmd(),
+		newReleaseCmd(),
 	)
 	return root
 }

internal/config/config.go

@@ -378,7 +378,12 @@ type Admission struct {
 
 	QueueTTL             time.Duration `koanf:"queue_ttl"`
 	QueueCapacityPerBand int           `koanf:"queue_capacity_per_band"`
-	Interval             time.Duration `koanf:"interval"`
+	// QueueCapacityTotal bounds everything queued across bands. Reached, a
+	// request is not refused while a lower band has any queued: the newest of
+	// the lowest band is turned away instead, with a 503 and Retry-After.
+	// Zero takes half of all the bands' capacities together.
+	QueueCapacityTotal int           `koanf:"queue_capacity_total"`
+	Interval           time.Duration `koanf:"interval"`
 }
 
 // Prefix configures approximate prefix routing.

internal/control/adminapi/adapters_test.go

@@ -0,0 +1,71 @@
+package adminapi
+
+import (
+	"bytes"
+	"encoding/binary"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+)
+
+// ggufStart is the start of a GGUF file with these string keys.
+func ggufStart(kv ...string) []byte {
+	var b bytes.Buffer
+	w := func(v any) { _ = binary.Write(&b, binary.LittleEndian, v) }
+	str := func(s string) { w(uint64(len(s))); b.WriteString(s) }
+	b.WriteString("GGUF")
+	w(uint32(3))
+	w(uint64(0))
+	w(uint64(len(kv) / 2))
+	for i := 0; i < len(kv); i += 2 {
+		str(kv[i])
+		w(uint32(8)) // string
+		str(kv[i+1])
+	}
+	b.Write(make([]byte, 4096))
+	return b.Bytes()
+}
+
+func TestOnlyAnAdapterForTheModelsArchitectureIsAttached(t *testing.T) {
+	files := map[string][]byte{
+		"/base.gguf":  ggufStart("general.architecture", "qwen2"),
+		"/model.gguf": ggufStart("general.architecture", "qwen2", "general.type", "model"),
+		"/llama.gguf": ggufStart("general.architecture", "llama", "general.type", "adapter"),
+		"/lora.gguf":  ggufStart("general.architecture", "qwen2", "general.type", "adapter"),
+	}
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusPartialContent)
+		_, _ = w.Write(files[r.URL.Path])
+	}))
+	defer srv.Close()
+
+	models := []catalog.Model{{
+		Name: "coder", Source: srv.URL + "/base.gguf",
+		Scaling:  catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 8}},
+	}}
+	for _, tc := range []struct {
+		file, want string
+		code       int
+	}{
+		{"/model.gguf", "not a LoRA adapter", http.StatusBadRequest},
+		{"/llama.gguf", "for llama models", http.StatusBadRequest},
+		{"/lora.gguf", "", http.StatusOK},
+	} {
+		a, _ := setup(t, models)
+		fm := &fakeModels{}
+		a.SetModels(fm)
+		mux := http.NewServeMux()
+		a.Register(mux)
+		req := httptest.NewRequest(http.MethodPost, "/api/v1/models/coder/adapters",
+			strings.NewReader(`{"adapters":[{"name":"fix","source":"`+srv.URL+tc.file+`"}]}`))
+		rec := httptest.NewRecorder()
+		mux.ServeHTTP(rec, req)
+		if rec.Code != tc.code || !strings.Contains(rec.Body.String(), tc.want) {
+			t.Errorf("%s: %d %s; want %d mentioning %q", tc.file, rec.Code, rec.Body.String(), tc.code, tc.want)
+		}
+	}
+}

internal/control/adminapi/api.go

@@ -10,16 +10,19 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
-	"gitea.mynodes.xyz/ilker/llmgw/internal/control/modelserve"
-	"gitea.mynodes.xyz/ilker/llmgw/internal/release"
 	"io"
 	"net/http"
+	"net/url"
 	"slices"
 	"strconv"
 	"strings"
 	"time"
 	"unicode"
 
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/modelserve"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/release"
+
 	"gitea.mynodes.xyz/ilker/llmgw/internal/agent/discovery"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/bench"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control"
@@ -60,6 +63,7 @@ func (a *API) Register(mux *http.ServeMux) {
 	mux.HandleFunc("POST /api/v1/models/{name}/scale", a.scaleModel)
 	mux.HandleFunc("POST /api/v1/models/{name}/resources", a.setResources)
 	a.registerQuotas(mux)
+	mux.HandleFunc("POST /api/v1/models/{name}/adapters", a.setAdapters)
 	mux.HandleFunc("GET /api/v1/models/{name}/target", a.target)
 	mux.HandleFunc("POST /api/v1/models/{name}/class", func(w http.ResponseWriter, r *http.Request) {
 		if a.models == nil {
@@ -312,10 +316,15 @@ type ModelView struct {
 	Warnings []string `json:"warnings,omitempty"`
 	// Class is its capacity class; Preempted the replicas it lost to a higher
 	// class; HeldFor the model it is giving room to now, if any.
-	Class     string `json:"class"`
-	Owner     string `json:"owner,omitempty"`
-	Preempted int    `json:"preempted,omitempty"`
-	HeldFor   string `json:"held_for,omitempty"`
+	Class string `json:"class"`
+	Owner string `json:"owner,omitempty"`
+	// External is where a model served outside the cluster goes, provider
+	// host and its name for the model; empty for one served here.
+	External string `json:"external,omitempty"`
+	// Adapters are the LoRA adapters it serves.
+	Adapters  []catalog.Adapter `json:"adapters,omitempty"`
+	Preempted int               `json:"preempted,omitempty"`
+	HeldFor   string            `json:"held_for,omitempty"`
 }
 
 // ReplicaView is a replica as `ctl replicas` shows it.
@@ -408,6 +417,10 @@ type HeadroomView struct {
 	Blocked []string `json:"blocked,omitempty"`
 	// TokPerSec is what one more replica was measured to serve; 0 unmeasured.
 	TokPerSec float64 `json:"tok_per_sec,omitempty"`
+	// ByPreempting is how many more would fit by stopping replicas this
+	// model's class may take; Preempts the models those belong to.
+	ByPreempting int      `json:"by_preempting,omitempty"`
+	Preempts     []string `json:"preempts,omitempty"`
 }
 
 // RejectionView is one rejected candidate.
@@ -565,6 +578,7 @@ func (a *API) Models() []ModelView {
 			Desired:   h.Desired, Failures: h.Failures,
 			Parked: h.Parked, Reason: h.Reason, Unplaceable: h.Unplaceable,
 			Class: m.EffectiveClass(), Owner: m.Owner, Preempted: h.Preempted, HeldFor: h.HeldFor,
+			External: externalLabel(m), Adapters: m.Adapters,
 		}
 		for _, p := range m.Profiles {
 			v.Profiles = append(v.Profiles, p.Name)
@@ -647,6 +661,7 @@ type Models interface {
 	Resources(ctx context.Context, name, profile string, memoryGB, cpuCores int, by string) (catalog.Model, error)
 	SetClass(ctx context.Context, name, class, by string) (catalog.Model, error)
 	SetOwner(ctx context.Context, name, owner, by string) (catalog.Model, error)
+	SetAdapters(ctx context.Context, name string, adapters []catalog.Adapter, by string) (catalog.Model, error)
 }
 
 // ResourcesRequest sets what a model's profiles reserve of a node.
@@ -967,7 +982,8 @@ func (a *API) Headroom(model string, limit int) (HeadroomView, bool) {
 		return HeadroomView{}, false
 	}
 	return HeadroomView{Model: model, Current: h.Current, More: h.More, Nodes: h.Nodes,
-		Limited: h.Limited, Blocked: h.Blocked, TokPerSec: h.TokPerSec}, true
+		Limited: h.Limited, Blocked: h.Blocked, TokPerSec: h.TokPerSec,
+		ByPreempting: h.ByPreempting, Preempts: h.Preempts}, true
 }
 
 func (a *API) Explain(model string) ExplainView {
@@ -1438,3 +1454,70 @@ func reservationWarnings(m catalog.Model, measuredGB float64) []string {
 	}
 	return out
 }
+
+// externalLabel is "api.openai.com · gpt-4o-mini" for a model served elsewhere.
+func externalLabel(m catalog.Model) string {
+	if !m.IsExternal() {
+		return ""
+	}
+	host := m.External.URL
+	if u, err := url.Parse(host); err == nil && u.Host != "" {
+		host = u.Host
+	}
+	return host + " · " + m.External.Model
+}
+
+// AdaptersRequest replaces a model's LoRA adapters with these.
+type AdaptersRequest struct {
+	Adapters []catalog.Adapter `json:"adapters"`
+}
+
+// setAdapters replaces a model's adapters, after reading each new file to
+// check it is an adapter for the model's architecture: one that is not is a
+// replica that fails to start, found minutes later on a node.
+func (a *API) setAdapters(w http.ResponseWriter, r *http.Request) {
+	if a.models == nil {
+		writeError(w, http.StatusNotImplemented, "this control plane has no store")
+		return
+	}
+	var req AdaptersRequest
+	if err := json.NewDecoder(io.LimitReader(r.Body, 1<<16)).Decode(&req); err != nil {
+		writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+		return
+	}
+	name := r.PathValue("name")
+	m, ok := a.ctl.Catalog().Resolve(name)
+	if !ok {
+		writeError(w, http.StatusBadRequest, "no model "+name)
+		return
+	}
+	base, baseErr := gguf.KindFromURL(r.Context(), nil, m.Source)
+	have := map[string]bool{}
+	for _, ad := range m.Adapters {
+		have[ad.Source] = true
+	}
+	for _, ad := range req.Adapters {
+		if have[ad.Source] {
+			continue // already checked when it was attached
+		}
+		h, err := gguf.KindFromURL(r.Context(), nil, ad.Source)
+		switch {
+		case err != nil:
+			writeError(w, http.StatusBadRequest, fmt.Sprintf("adapter %s: %v", ad.Name, err))
+			return
+		case !h.IsAdapter():
+			writeError(w, http.StatusBadRequest, fmt.Sprintf("adapter %s: %s is a model, not a LoRA adapter", ad.Name, ad.Source))
+			return
+		case baseErr == nil && h.Architecture != base.Architecture:
+			writeError(w, http.StatusBadRequest, fmt.Sprintf("adapter %s is for %s models and %s is %s",
+				ad.Name, h.Architecture, m.Name, base.Architecture))
+			return
+		}
+	}
+	out, err := a.models.SetAdapters(r.Context(), m.Name, req.Adapters, actorOf(r))
+	if err != nil {
+		writeError(w, http.StatusBadRequest, err.Error())
+		return
+	}
+	writeJSON(w, http.StatusOK, AdaptersRequest{Adapters: out.Adapters})
+}

internal/control/adminapi/api_test.go

@@ -258,6 +258,10 @@ type fakeModels struct {
 	err     error
 }
 
+func (f *fakeModels) SetAdapters(_ context.Context, name string, as []catalog.Adapter, _ string) (catalog.Model, error) {
+	return catalog.Model{Name: name, Adapters: as}, f.err
+}
+
 func (f *fakeModels) SetOwner(_ context.Context, name, owner, _ string) (catalog.Model, error) {
 	return catalog.Model{Name: name, Owner: owner}, f.err
 }

internal/control/adminapi/quotas.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"io"
 	"net/http"
+	"sort"
 	"strings"
 	"time"
 
@@ -72,38 +73,24 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
 			writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
 			return
 		}
-		scope, id, _, err := a.resolveScope(r.Context(), req.Scope)
+		key, err := a.SetQuota(r.Context(), req, actorOf(r))
 		if err != nil {
 			writeError(w, http.StatusBadRequest, err.Error())
 			return
 		}
-		if !req.Guarantee.Any() && !req.Limit.Any() {
-			writeError(w, http.StatusBadRequest, "a quota needs a guarantee or a limit; to remove one, DELETE it")
-			return
-		}
-		q := store.CapacityQuota{Scope: scope, ScopeID: id, Guarantee: req.Guarantee, Limit: req.Limit,
-			UpdatedBy: actorOf(r)}
-		if err := a.quotas.PutCapacityQuota(r.Context(), q); err != nil {
-			writeError(w, http.StatusBadRequest, err.Error())
-			return
-		}
-		writeJSON(w, http.StatusOK, map[string]string{"scope": quota.Key(scope, id)})
+		writeJSON(w, http.StatusOK, map[string]string{"scope": key})
 	})
 	mux.HandleFunc("DELETE /api/v1/quotas/{scope}", func(w http.ResponseWriter, r *http.Request) {
 		if a.quotas == nil {
 			writeError(w, http.StatusNotImplemented, "this control plane has no store")
 			return
 		}
-		scope, id, _, err := a.resolveScope(r.Context(), r.PathValue("scope"))
+		key, err := a.ClearQuota(r.Context(), r.PathValue("scope"))
 		if err != nil {
 			writeError(w, http.StatusBadRequest, err.Error())
 			return
 		}
-		if err := a.quotas.DeleteCapacityQuota(r.Context(), scope, id); err != nil {
-			writeError(w, http.StatusInternalServerError, err.Error())
-			return
-		}
-		writeJSON(w, http.StatusOK, map[string]string{"removed": quota.Key(scope, id)})
+		writeJSON(w, http.StatusOK, map[string]string{"removed": key})
 	})
 	mux.HandleFunc("POST /api/v1/models/{name}/owner", func(w http.ResponseWriter, r *http.Request) {
 		if a.models == nil || a.quotas == nil {
@@ -137,6 +124,9 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
 
 // QuotaViews lists every quota with what its scope holds now.
 func (a *API) QuotaViews(ctx context.Context) ([]QuotaView, error) {
+	if a.quotas == nil {
+		return nil, nil
+	}
 	qs, err := a.quotas.CapacityQuotas(ctx)
 	if err != nil {
 		return nil, err
@@ -235,3 +225,61 @@ func (a *API) resolveScope(ctx context.Context, s string) (store.Scope, string,
 	}
 	return "", "", "", fmt.Errorf("%q could be any of %d %ss; give its ID", what, len(found), kind)
 }
+
+// SetQuota sets a scope's capacity quota, named by path or ID, and returns
+// the scope as org:ID.
+func (a *API) SetQuota(ctx context.Context, req QuotaRequest, by string) (string, error) {
+	if a.quotas == nil {
+		return "", fmt.Errorf("this control plane has no store")
+	}
+	scope, id, _, err := a.resolveScope(ctx, req.Scope)
+	if err != nil {
+		return "", err
+	}
+	if !req.Guarantee.Any() && !req.Limit.Any() {
+		return "", fmt.Errorf("a quota needs a guarantee or a limit; to remove one, clear it")
+	}
+	q := store.CapacityQuota{Scope: scope, ScopeID: id, Guarantee: req.Guarantee, Limit: req.Limit, UpdatedBy: by}
+	if err := a.quotas.PutCapacityQuota(ctx, q); err != nil {
+		return "", err
+	}
+	return quota.Key(scope, id), nil
+}
+
+// ClearQuota removes a scope's capacity quota.
+func (a *API) ClearQuota(ctx context.Context, scope string) (string, error) {
+	if a.quotas == nil {
+		return "", fmt.Errorf("this control plane has no store")
+	}
+	s, id, _, err := a.resolveScope(ctx, scope)
+	if err != nil {
+		return "", err
+	}
+	if err := a.quotas.DeleteCapacityQuota(ctx, s, id); err != nil {
+		return "", err
+	}
+	return quota.Key(s, id), nil
+}
+
+// ScopeName is a part of the tenancy tree: org:ID and its path.
+type ScopeName struct {
+	Key  string `json:"scope"`
+	Name string `json:"name"`
+}
+
+// Scopes lists every organization, team and project, by path.
+func (a *API) Scopes(ctx context.Context) ([]ScopeName, error) {
+	if a.quotas == nil {
+		return nil, nil
+	}
+	names, err := a.scopeNames(ctx)
+	if err != nil {
+		return nil, err
+	}
+	out := make([]ScopeName, 0, len(names))
+	for k, n := range names {
+		out = append(out, ScopeName{Key: k, Name: n})
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
+	return out, nil
+}

internal/control/adminapi/target.go

@@ -24,6 +24,9 @@ type TargetView struct {
 	Room    int      `json:"room"`
 	Fits    bool     `json:"fits"`
 	Blocked []string `json:"blocked,omitempty"`
+	// Preempts is whose replicas would be stopped to make the room, when
+	// free room alone is not enough; Room counts that room too.
+	Preempts []string `json:"preempts,omitempty"`
 }
 
 // Target sizes a model for a throughput.
@@ -40,8 +43,11 @@ func (a *API) Target(model string, tokPerSec float64) (TargetView, error) {
 	}
 	v.Current = h.Current
 	v.More = max(v.Needed-h.Current, 0)
-	v.Room = h.More
-	v.Fits = v.More <= h.More || h.Limited
+	v.Room = h.More + h.ByPreempting
+	v.Fits = v.More <= v.Room || h.Limited
+	if v.More > h.More {
+		v.Preempts = h.Preempts
+	}
 	if !v.Fits {
 		v.Blocked = h.Blocked
 	}

internal/control/adminapi/weights.go

@@ -31,6 +31,11 @@ func (a *API) Orphans() []OrphanView {
 	var wanted []string
 	for _, m := range a.ctl.Catalog().Models() {
 		wanted = append(wanted, m.Source)
+		// A model's adapters are its weights too; collecting them would
+		// leave replicas that cannot restart.
+		for _, ad := range m.Adapters {
+			wanted = append(wanted, ad.Source)
+		}
 	}
 
 	out := make([]OrphanView, 0)

internal/control/agentsvc/convert.go

@@ -189,6 +189,7 @@ func StartShardFromCommand(cmd reconciler.ShardCommand) *llmgwv1.StartShard {
 			GpusPerNode: int32(p.GPUsPerNode), Nodes: int32(p.Nodes),
 			Args: p.Args, Env: p.Env,
 			ModelPath: p.ModelPath, ServedModelName: p.ServedModelName,
+			Adapters: adaptersToWire(p.Adapters),
 		},
 		StartupTimeoutMs: cmd.StartupTimeout.Milliseconds(),
 		Pool:             cmd.Pool,
@@ -224,6 +225,7 @@ func ShardSpecFromStart(m *llmgwv1.StartShard) (engines.ShardSpec, time.Duration
 			GPUsPerNode: int(p.GetGpusPerNode()), Nodes: int(p.GetNodes()),
 			Args: p.GetArgs(), Env: p.GetEnv(),
 			ModelPath: p.GetModelPath(), ServedModelName: p.GetServedModelName(),
+			Adapters: adaptersFromWire(p.GetAdapters()),
 		},
 	}
 	return spec, time.Duration(m.GetStartupTimeoutMs()) * time.Millisecond
@@ -292,3 +294,19 @@ func fabricFrom(state, status, cluster, clique, bandwidth string) *discovery.GPU
 	}
 	return &discovery.GPUFabric{State: state, Status: status, ClusterUUID: cluster, CliqueID: clique, Bandwidth: bandwidth}
 }
+
+func adaptersToWire(as []engines.Adapter) []*llmgwv1.LoraAdapter {
+	var out []*llmgwv1.LoraAdapter
+	for _, a := range as {
+		out = append(out, &llmgwv1.LoraAdapter{Name: a.Name, Source: a.Path})
+	}
+	return out
+}
+
+func adaptersFromWire(as []*llmgwv1.LoraAdapter) []engines.Adapter {
+	var out []engines.Adapter
+	for _, a := range as {
+		out = append(out, engines.Adapter{Name: a.GetName(), Path: a.GetSource()})
+	}
+	return out
+}

internal/control/agentsvc/convert_test.go

@@ -0,0 +1,20 @@
+package agentsvc
+
+import (
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/agent/engines"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
+)
+
+func TestAdaptersTravelToTheAgentInOrder(t *testing.T) {
+	cmd := reconciler.ShardCommand{ReplicaID: "r", ShardID: "s", Spec: engines.ShardSpec{
+		Profile: engines.Profile{Name: "cpu", Engine: "llamacpp", ModelPath: "/m.gguf",
+			Adapters: []engines.Adapter{{Name: "sql", Path: "http://mirror/sql.gguf"}, {Name: "docs", Path: "http://mirror/docs.gguf"}}},
+	}}
+	spec, _ := ShardSpecFromStart(StartShardFromCommand(cmd))
+	got := spec.Profile.Adapters
+	if len(got) != 2 || got[0].Name != "sql" || got[1].Path != "http://mirror/docs.gguf" {
+		t.Errorf("adapters %+v", got)
+	}
+}

internal/control/catalog/catalog.go

@@ -7,6 +7,7 @@ import (
 	"encoding/hex"
 	"encoding/json"
 	"fmt"
+	"net/url"
 	"slices"
 	"strings"
 	"time"
@@ -160,8 +161,20 @@ type Model struct {
 	// Owner is the part of the tenancy tree whose capacity quota this model
 	// counts against, written org:ID, team:ID or project:ID. Empty is shared:
 	// held by the cluster, counted against nobody.
-	Owner     string   `json:"owner,omitempty" koanf:"owner"`
-	Fallbacks []string `json:"fallbacks,omitempty" koanf:"fallbacks"`
+	Owner string `json:"owner,omitempty" koanf:"owner"`
+	// External is a model served by someone else — an OpenAI-compatible API
+	// outside the cluster — rather than by replicas placed here. Such a model
+	// has no profiles and nothing is placed for it; the router sends its
+	// requests to the provider. Most useful named in another model's
+	// fallbacks: used only when the cluster has no capacity left. Its
+	// prompts leave the cluster, so which keys may name it is worth an
+	// allowlist.
+	External *External `json:"external,omitempty" koanf:"external"`
+	// Adapters are LoRA adapters served on this model's replicas, each asked
+	// for by its own name as though it were a model. Every replica loads
+	// every adapter, applying one only to a request that names it.
+	Adapters  []Adapter `json:"adapters,omitempty" koanf:"adapters"`
+	Fallbacks []string  `json:"fallbacks,omitempty" koanf:"fallbacks"`
 	// RoutingProfile selects how the router scores endpoints for this model.
 	RoutingProfile string `json:"routing_profile,omitempty" koanf:"routing_profile"`
 	SLO            SLO    `json:"slo,omitempty" koanf:"slo"`
@@ -204,14 +217,17 @@ type Capabilities struct {
 type Catalog struct {
 	models  map[string]Model
 	aliases map[string]string
-	order   []string
+	// adapters maps an adapter's name to the model it runs on.
+	adapters map[string]string
+	order    []string
 }
 
 // New builds a catalog, rejecting definitions that would fail confusingly later.
 func New(models []Model) (*Catalog, error) {
 	c := &Catalog{
-		models:  make(map[string]Model, len(models)),
-		aliases: map[string]string{},
+		models:   make(map[string]Model, len(models)),
+		aliases:  map[string]string{},
+		adapters: map[string]string{},
 	}
 
 	for i := range models {
@@ -240,6 +256,18 @@ func New(models []Model) (*Catalog, error) {
 		}
 	}
 
+	// Adapters are named like models, so they share the namespace too.
+	for _, name := range c.order {
+		for _, a := range c.models[name].Adapters {
+			_, isModel := c.models[a.Name]
+			_, isAlias := c.aliases[a.Name]
+			if _, isAdapter := c.adapters[a.Name]; isModel || isAlias || isAdapter {
+				return nil, fmt.Errorf("model %q: adapter %q collides with a model, an alias or another adapter", name, a.Name)
+			}
+			c.adapters[a.Name] = name
+		}
+	}
+
 	// Fallbacks are resolved now rather than at request time: a typo in a
 	// fallback should stop a deployment, not surface during an outage when the
 	// fallback is finally needed.
@@ -305,7 +333,14 @@ func (m Model) validate() error {
 	if m.Source == "" {
 		return fmt.Errorf("model %q: source is required", m.Name)
 	}
-	if len(m.Profiles) == 0 {
+	if err := validateAdapters(m); err != nil {
+		return err
+	}
+	if m.External != nil {
+		if err := m.External.validate(m.Name, len(m.Profiles)); err != nil {
+			return err
+		}
+	} else if len(m.Profiles) == 0 {
 		return fmt.Errorf("model %q: at least one profile is required", m.Name)
 	}
 	if m.Owner != "" && !validOwner(m.Owner) {
@@ -357,6 +392,23 @@ func (m Model) validate() error {
 }
 
 // Resolve maps a client-supplied name or alias onto a model.
+// ResolveAdapter finds an adapter by its name: the model it runs on, the
+// adapter, and its index among the model's adapters — which is the id the
+// engine knows it by.
+func (c *Catalog) ResolveAdapter(name string) (Model, Adapter, int, bool) {
+	base, ok := c.adapters[name]
+	if !ok {
+		return Model{}, Adapter{}, 0, false
+	}
+	m := c.models[base]
+	for i, a := range m.Adapters {
+		if a.Name == name {
+			return m, a, i, true
+		}
+	}
+	return Model{}, Adapter{}, 0, false
+}
+
 func (c *Catalog) Resolve(name string) (Model, bool) {
 	if m, ok := c.models[name]; ok {
 		return m, true
@@ -433,6 +485,7 @@ func (m Model) EngineProfile(name string) (engines.Profile, bool) {
 			Env:             p.Env,
 			ModelPath:       m.Source,
 			ServedModelName: m.Name,
+			Adapters:        m.engineAdapters(),
 		}, true
 	}
 	return engines.Profile{}, false
@@ -564,3 +617,90 @@ func validOwner(o string) bool {
 	}
 	return false
 }
+
+// External is a provider outside the cluster that serves a model.
+type External struct {
+	// URL is the provider's base address, without /v1: https://api.openai.com.
+	URL string `json:"url" koanf:"url"`
+	// Model is what the provider calls it; requests are rewritten to it.
+	Model string `json:"model" koanf:"model"`
+	// APIKeyFile holds the provider's key, on the control plane. A file
+	// rather than the key itself, so the key is in neither the catalog, the
+	// store nor the audit log; empty for a provider that asks for none.
+	APIKeyFile string `json:"api_key_file,omitempty" koanf:"api_key_file"`
+}
+
+func (e *External) validate(model string, profiles int) error {
+	if profiles > 0 {
+		return fmt.Errorf("model %q: an external model is served elsewhere and has no profiles", model)
+	}
+	u, err := url.Parse(e.URL)
+	if err != nil || (u.Scheme != "https" && u.Scheme != "http") || u.Host == "" {
+		return fmt.Errorf("model %q: external url %q is not an http(s) address", model, e.URL)
+	}
+	if strings.TrimSpace(e.Model) == "" {
+		return fmt.Errorf("model %q: external model needs the name the provider knows it by", model)
+	}
+	return nil
+}
+
+// IsExternal reports whether a model is served outside the cluster.
+func (m Model) IsExternal() bool { return m.External != nil }
+
+// Adapter is a LoRA adapter served on a model's replicas.
+type Adapter struct {
+	// Name is what clients ask for, like a model name.
+	Name string `json:"name" koanf:"name"`
+	// Source is the adapter file (GGUF for llama.cpp), fetched by each node
+	// as the model's weights are.
+	Source string `json:"source" koanf:"source"`
+	// Scale is how strongly it is applied; zero takes 1.
+	Scale float64 `json:"scale,omitempty" koanf:"scale"`
+}
+
+// EffectiveScale is the scale an adapter is applied at.
+func (a Adapter) EffectiveScale() float64 {
+	if a.Scale <= 0 {
+		return 1
+	}
+	return a.Scale
+}
+
+func validateAdapters(m Model) error {
+	if len(m.Adapters) == 0 {
+		return nil
+	}
+	if m.External != nil {
+		return fmt.Errorf("model %q: an external model cannot carry adapters", m.Name)
+	}
+	for _, p := range m.Profiles {
+		// llama.cpp loads adapters at start and applies them per request;
+		// the other engines are not driven that way yet.
+		if p.Engine != "llamacpp" && p.Engine != "sim" {
+			return fmt.Errorf("model %q profile %q: adapters are served by llamacpp profiles only", m.Name, p.Name)
+		}
+	}
+	seen := map[string]bool{}
+	for _, a := range m.Adapters {
+		switch {
+		case a.Name == "" || a.Source == "":
+			return fmt.Errorf("model %q: an adapter needs a name and a source", m.Name)
+		case a.Name == m.Name:
+			return fmt.Errorf("model %q: an adapter cannot have the model's own name", m.Name)
+		case seen[a.Name]:
+			return fmt.Errorf("model %q: adapter %q is declared twice", m.Name, a.Name)
+		case a.Scale < 0:
+			return fmt.Errorf("model %q: adapter %q has a negative scale", m.Name, a.Name)
+		}
+		seen[a.Name] = true
+	}
+	return nil
+}
+
+func (m Model) engineAdapters() []engines.Adapter {
+	var out []engines.Adapter
+	for _, a := range m.Adapters {
+		out = append(out, engines.Adapter{Name: a.Name, Path: a.Source})
+	}
+	return out
+}

internal/control/control.go

@@ -9,6 +9,7 @@ import (
 	"encoding/hex"
 	"fmt"
 	"log/slog"
+	"os"
 	"strings"
 	"sync"
 	"time"
@@ -338,6 +339,34 @@ func (c *Controller) endpoints() []endpoint.Info {
 			Priority: prof.Priority,
 		})
 	}
+	return append(out, c.externalEndpoints()...)
+}
+
+// externalEndpoints are the models served outside the cluster, as endpoints:
+// one each, with the provider's key read from its file every time, so a key
+// rotated on disk is used from the next publish on.
+func (c *Controller) externalEndpoints() []endpoint.Info {
+	var out []endpoint.Info
+	for _, m := range c.cat.Load().Models() {
+		if !m.IsExternal() {
+			continue
+		}
+		key := ""
+		if f := m.External.APIKeyFile; f != "" {
+			b, err := os.ReadFile(f) //nolint:gosec // the operator's own path
+			if err != nil {
+				c.log.Warn("an external model's key could not be read; not offered", "model", m.Name, "err", err)
+				continue
+			}
+			key = strings.TrimSpace(string(b))
+		}
+		out = append(out, endpoint.Info{
+			ID: "external:" + m.Name, URL: strings.TrimRight(m.External.URL, "/"),
+			Models: append([]string{m.Name}, m.Aliases...), APIKey: key,
+			Engine: endpoint.EngineExternal, Pool: "external", Weight: 1,
+			UpstreamModel: m.External.Model,
+		})
+	}
 	return out
 }
 
@@ -345,7 +374,8 @@ func (c *Controller) endpoints() []endpoint.Info {
 func fingerprint(infos []endpoint.Info) string {
 	var b strings.Builder
 	for _, i := range infos {
-		fmt.Fprintf(&b, "%s|%s|%s|%s|%v|%g|%d;", i.ID, i.URL, i.Engine, i.Pool, i.Models, i.Weight, i.Priority)
+		fmt.Fprintf(&b, "%s|%s|%s|%s|%v|%g|%d|%s|%x;", i.ID, i.URL, i.Engine, i.Pool, i.Models, i.Weight, i.Priority,
+			i.UpstreamModel, sha256.Sum256([]byte(i.APIKey)))
 	}
 	return b.String()
 }

internal/control/external_test.go

@@ -0,0 +1,75 @@
+package control
+
+import (
+	"context"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"sync"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
+)
+
+type recordSink struct {
+	mu    sync.Mutex
+	infos []endpoint.Info
+	times int
+}
+
+func (s *recordSink) SetEndpoints(_ context.Context, infos []endpoint.Info) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.infos, s.times = infos, s.times+1
+}
+
+type noDispatch struct{}
+
+func (noDispatch) StartShard(context.Context, reconciler.ShardCommand) error { return nil }
+func (noDispatch) StopShard(context.Context, string, string) error           { return nil }
+func (noDispatch) DrainShard(context.Context, string, string) error          { return nil }
+
+func TestAnExternalModelIsPublishedWithItsKeyAndNothingIsPlacedForIt(t *testing.T) {
+	dir := t.TempDir()
+	keyFile := filepath.Join(dir, "provider.key")
+	if err := os.WriteFile(keyFile, []byte("sk-provider-1\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	models := []catalog.Model{
+		{Name: "cloud", Source: "external", Aliases: []string{"gpt"}, Scaling: catalog.Scaling{Min: 3, Max: 3},
+			External: &catalog.External{URL: "https://api.example.com/", Model: "gpt-4o-mini", APIKeyFile: keyFile}},
+		{Name: "gone", Source: "external",
+			External: &catalog.External{URL: "https://api.example.com", Model: "m", APIKeyFile: filepath.Join(dir, "missing")}},
+	}
+	cat, err := catalog.New(models)
+	if err != nil {
+		t.Fatal(err)
+	}
+	sink := &recordSink{}
+	c := New(DefaultConfig(), catalog.NewLive(cat), noDispatch{}, sink, slog.New(slog.DiscardHandler))
+	c.Publish(t.Context())
+
+	if len(sink.infos) != 1 {
+		t.Fatalf("published %+v; the model whose key is missing must not be offered", sink.infos)
+	}
+	e := sink.infos[0]
+	if e.ID != "external:cloud" || e.URL != "https://api.example.com" || e.APIKey != "sk-provider-1" ||
+		e.UpstreamModel != "gpt-4o-mini" || e.Engine != endpoint.EngineExternal || len(e.Models) != 2 {
+		t.Errorf("endpoint %+v", e)
+	}
+	// A key rotated on disk is published on the next pass.
+	_ = os.WriteFile(keyFile, []byte("sk-provider-2"), 0o600)
+	c.Publish(t.Context())
+	if sink.times != 2 || sink.infos[0].APIKey != "sk-provider-2" {
+		t.Errorf("the rotated key was not published: %d pushes, key %q", sink.times, sink.infos[0].APIKey)
+	}
+	// Scaling of 3 or not, nothing is wanted of the cluster for a model
+	// served elsewhere, and so it never reads as short or unplaceable.
+	for _, h := range c.Reconciler().ModelHealth() {
+		if h.Model == "cloud" && (h.Desired != 0 || h.Unplaceable) {
+			t.Errorf("an external model wants replicas: %+v", h)
+		}
+	}
+}

internal/control/modelsvc/modelsvc.go

@@ -87,6 +87,9 @@ func (s *Service) Sources(ctx context.Context) (map[string][]catalog.Model, erro
 		if m.Source != "" {
 			out[m.Source] = append(out[m.Source], m)
 		}
+		for _, ad := range m.Adapters {
+			out[ad.Source] = append(out[ad.Source], m)
+		}
 	}
 	return out, nil
 }
@@ -268,6 +271,23 @@ func (s *Service) SetOwner(ctx context.Context, name, owner, by string) (catalog
 	return final, err
 }
 
+// SetAdapters replaces the LoRA adapters a model serves. Replicas load the
+// new set when they next start.
+func (s *Service) SetAdapters(ctx context.Context, name string, adapters []catalog.Adapter, by string) (catalog.Model, error) {
+	final, err := s.update(ctx, name, by, "change its adapters", func(m *catalog.Model) error {
+		m.Adapters = adapters
+		return nil
+	})
+	if err == nil {
+		names := make([]string, 0, len(adapters))
+		for _, a := range adapters {
+			names = append(names, a.Name)
+		}
+		s.log.Info("model adapters set", "model", name, "adapters", names, "by", by)
+	}
+	return final, err
+}
+
 // update changes one model added at runtime and keeps the change.
 //
 // Built before anything is written, like Add: catalog.New is what applies the

internal/control/reconciler/demand.go

@@ -94,6 +94,9 @@ func (r *Reconciler) desiredFor(m catalog.Model) int {
 // baseFor is what a model runs without load: its minimum, or for an on-demand
 // model one while somebody is asking and none once they stop.
 func (r *Reconciler) baseFor(m catalog.Model) int {
+	if m.IsExternal() {
+		return 0 // served elsewhere: nothing to place, whatever its scaling says
+	}
 	if !m.Scaling.OnDemand() {
 		return m.Scaling.Min
 	}

internal/control/reconciler/headroom.go

@@ -28,6 +28,11 @@ type Headroom struct {
 	// TokPerSec is what one more replica was measured to serve, on the
 	// profile placement would give it; zero when it was never measured.
 	TokPerSec float64
+	// ByPreempting is how many more, beyond More, would fit by stopping
+	// replicas the model's class may take, and Preempts the models those
+	// replicas belong to: what "room" means for a model that outranks others.
+	ByPreempting int
+	Preempts     []string
 }
 
 // Headroom tries placing up to limit more replicas of a model on a copy of
@@ -61,12 +66,38 @@ func (r *Reconciler) Headroom(model string, limit int) (Headroom, bool) {
 	if pol != nil {
 		usage = pol.Usage(r.Holdings())
 	}
-	for h.More < limit {
+	place := func() placement.Result {
 		res := sched.Place(r.orderedProfiles(m), avoid)
 		if res.Placement == nil && len(avoid) > 0 {
 			// As scaleUp does: spreading is a preference, not a rule.
 			res = sched.Place(r.orderedProfiles(m), nil)
 		}
+		return res
+	}
+	// What the model's class may take, in the order preemption would, each
+	// released on the copy only when free room has run out.
+	victims, nextVictim, preempting := r.victims(m), 0, false
+	for h.More+h.ByPreempting < limit {
+		res := place()
+		// Named only once they have made room: candidates tried on the way
+		// to finding there is none would otherwise read as whose replicas a
+		// placement would stop.
+		var tried []string
+		for res.Placement == nil && nextVictim < len(victims) {
+			v := victims[nextVictim]
+			nextVictim++
+			inv.ReleaseReplica(v.ID)
+			tried = append(tried, v.Model)
+			res = place()
+		}
+		if res.Placement != nil && len(tried) > 0 {
+			preempting = true
+			for _, m := range tried {
+				if !slices.Contains(h.Preempts, m) {
+					h.Preempts = append(h.Preempts, m)
+				}
+			}
+		}
 		if res.Placement == nil {
 			for _, rej := range res.Rejections {
 				h.Blocked = append(h.Blocked, rej.String())
@@ -103,8 +134,14 @@ func (r *Reconciler) Headroom(model string, limit int) (Headroom, bool) {
 			nodes = append(nodes, a.NodeID)
 		}
 		avoid = append(avoid, nodes...)
-		h.Nodes = append(h.Nodes, nodes)
-		h.More++
+		// Once anything was stopped, what fits after is counted as fitting
+		// by preempting: it may be sitting in the room that was freed.
+		if preempting {
+			h.ByPreempting++
+		} else {
+			h.Nodes = append(h.Nodes, nodes)
+			h.More++
+		}
 	}
 	h.Limited = true
 	return h, true

internal/control/reconciler/preempt_test.go

@@ -312,3 +312,107 @@ func TestOfTwoThatWouldEachMakeRoomTheSmallerIsStopped(t *testing.T) {
 		}
 	}
 }
+
+func TestHeadroomCountsWhatAHigherClassCouldTake(t *testing.T) {
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+	got, _ := h.r.Headroom("vital", 3)
+	if got.More != 0 || got.ByPreempting != 1 || len(got.Preempts) != 1 || got.Preempts[0] != "chat" {
+		t.Errorf("a critical model should see one more by stopping chat: %+v", got)
+	}
+	// Tried on a copy: nothing was stopped.
+	if replicasOf(h, "chat") != 1 {
+		t.Fatal("asking about room stopped a replica")
+	}
+	// A model of chat's own class may not take its minimum, so has no such room.
+	peer := newFleetHarness(t, []catalog.Model{gpuModel("chat", "", 1), withMin(gpuModel("peer", "", 1), 0)}, gpuNode("1", 8))
+	peer.settle(t, 3)
+	peer.advance(peer.r.cfg.MinLifetime + time.Minute)
+	if got, _ := peer.r.Headroom("peer", 3); got.ByPreempting != 0 {
+		t.Errorf("a standard model counted room in another's minimum: %+v", got)
+	}
+}
+
+func TestARestartBorrowsALowerClassesRoomAndGivesItBack(t *testing.T) {
+	// chat fills node 1; the only other node holds an opportunistic replica.
+	chat := gpuModel("chat", "", 1)
+	extra := gpuModel("extra", catalog.ClassOpportunistic, 1)
+	h := newFleetHarness(t, []catalog.Model{chat, extra}, gpuNode("1", 8), gpuNode("2", 8))
+	h.settle(t, 3)
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	var old string
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == "chat" {
+			old = rep.ID
+		}
+	}
+	if old == "" || replicasOf(h, "extra") != 1 {
+		t.Fatalf("setup: %+v", h.r.Replicas())
+	}
+
+	if err := h.r.Restart(old); err != nil {
+		t.Fatal(err)
+	}
+	// Replacement first, beside the old one, in the room extra gives up;
+	// the old one keeps serving until then.
+	servedThroughout := true
+	for range 8 {
+		h.r.Reconcile(t.Context())
+		h.markReady()
+		ready := 0
+		for _, rep := range h.r.Replicas() {
+			if rep.Model == "chat" && rep.State == Ready {
+				ready++
+			}
+		}
+		if ready == 0 {
+			servedThroughout = false
+		}
+		h.advance(h.r.cfg.PreemptGrace + time.Second)
+	}
+	if !servedThroughout {
+		t.Error("chat had no ready replica during its restart")
+	}
+	var newChat bool
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == "chat" && rep.ID != old && rep.State == Ready {
+			newChat = true
+		}
+		if rep.ID == old {
+			t.Errorf("the old replica is still there: %+v", rep)
+		}
+	}
+	if !newChat {
+		t.Fatalf("no new chat replica: %+v", h.r.Replicas())
+	}
+	// And extra has its replica back, in the room the old chat left.
+	h.settle(t, 4)
+	if replicasOf(h, "extra") != 1 {
+		t.Errorf("the opportunistic model did not get its room back: %+v", h.r.Replicas())
+	}
+}
+
+func TestHeadroomNamesOnlyTheReplicasThatWouldMakeRoom(t *testing.T) {
+	// vital needs a whole node and chat is on the only one; a second
+	// critical can never fit. Asking for 3: one by stopping chat, then no
+	// candidates left — nothing else may be named.
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+	got, _ := h.r.Headroom("vital", 3)
+	if got.ByPreempting != 1 || len(got.Preempts) != 1 {
+		t.Errorf("%+v", got)
+	}
+	// Free room for one, and candidates that cannot make room for a second:
+	// none of them may be listed.
+	small := catalog.Model{Name: "small", Source: "/m", Class: catalog.ClassOpportunistic,
+		Scaling:  catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "tp2", Engine: "vllm", GPUClass: "h100-80g", TP: 2}}}
+	h2 := fillThenWant(t, gpuModel("big", "", 1), small, gpuNode("1", 8), gpuNode("2", 8))
+	got2, _ := h2.r.Headroom("big", 3)
+	if got2.More != 1 || got2.ByPreempting != 1 || len(got2.Preempts) != 1 {
+		t.Errorf("one free, one by stopping small: %+v", got2)
+	}
+	h3 := fillThenWant(t, gpuModel("big", "", 1), small, gpuNode("1", 8), gpuNode("2", 6))
+	got3, _ := h3.r.Headroom("big", 3)
+	if got3.ByPreempting != 0 || len(got3.Preempts) != 0 {
+		t.Errorf("stopping small frees a 6-GPU node, too small for big; nobody may be named: %+v", got3)
+	}
+}

internal/gguf/gguf.go

@@ -25,6 +25,10 @@ type Header struct {
 	Version      uint32
 	Architecture string
 	Name         string
+	// Type is general.type: "model", or "adapter" for a LoRA adapter, which
+	// is applied to a model rather than run on its own. Empty in files made
+	// before the key existed, which are models.
+	Type string
 
 	// The shape the KV cache is sized from, from the architecture's own keys
 	// (llama.block_count, qwen2.attention.head_count_kv, ...). Zero when the
@@ -235,6 +239,8 @@ func Read(r io.Reader) (Header, error) {
 			h.Architecture = v
 		case "general.name":
 			h.Name = v
+		case "general.type":
+			h.Type = v
 		}
 		if h.Architecture != "" {
 			switch strings.TrimPrefix(key, h.Architecture+".") {
@@ -290,6 +296,41 @@ func FromURL(ctx context.Context, client *http.Client, url string) (Header, erro
 	return Read(io.LimitReader(resp.Body, headerBytes))
 }
 
+// IsAdapter reports whether the file is a LoRA adapter rather than a model.
+func (h Header) IsAdapter() bool { return h.Type == "adapter" }
+
+// adapterProbe is how much of a file is read to tell an adapter from a model:
+// general.type comes before the tokenizer, which is what makes a model's
+// header megabytes long.
+const adapterProbe = 256 << 10
+
+// KindFromURL reads just enough of a GGUF file to say what it is — its type
+// and architecture — over HTTP. A header cut short by the probe is fine: the
+// keys asked for come first.
+func KindFromURL(ctx context.Context, client *http.Client, url string) (Header, error) {
+	if client == nil {
+		client = &http.Client{Timeout: 30 * time.Second}
+	}
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	if err != nil {
+		return Header{}, err
+	}
+	req.Header.Set("Range", fmt.Sprintf("bytes=0-%d", adapterProbe-1))
+	resp, err := client.Do(req)
+	if err != nil {
+		return Header{}, err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusPartialContent {
+		return Header{}, fmt.Errorf("%s answered %s", url, resp.Status)
+	}
+	h, err := Read(io.LimitReader(resp.Body, adapterProbe))
+	if h.Architecture != "" {
+		return h, nil // what was asked for was read, whatever came after
+	}
+	return h, err
+}
+
 // IsGGUF reports whether a source names a GGUF file, or a directory of them.
 func IsGGUF(source string) bool {
 	s := strings.ToLower(strings.TrimSuffix(source, "/"))

internal/gguf/kind_live_test.go

@@ -0,0 +1,31 @@
+package gguf
+
+import (
+	"os"
+	"testing"
+)
+
+// Run by hand against the real adapter this was written for:
+//
+//	LLMGW_LIVE_GGUF=1 go test ./internal/gguf -run LiveAdapter -v
+func TestLiveAdapterIsAnAdapter(t *testing.T) {
+	if os.Getenv("LLMGW_LIVE_GGUF") == "" {
+		t.Skip("set LLMGW_LIVE_GGUF to read a real adapter from Hugging Face")
+	}
+	h, err := KindFromURL(t.Context(), nil,
+		"https://huggingface.co/ggml-org/LoRA-Qwen2.5-Coder-7B-Instruct-F16-GGUF/resolve/main/LoRA-Qwen2.5-Coder-7B-Instruct-f16.gguf")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !h.IsAdapter() || h.Architecture != "qwen2" {
+		t.Errorf("%+v", h)
+	}
+	m, err := KindFromURL(t.Context(), nil,
+		"https://huggingface.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m.IsAdapter() || m.Architecture != "qwen2" {
+		t.Errorf("a model read as %+v", m)
+	}
+}

internal/gguf/kind_test.go

@@ -0,0 +1,58 @@
+package gguf
+
+import (
+	"bytes"
+	"encoding/binary"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+// adapterHeader is the start of a LoRA adapter as llama.cpp's converter
+// writes it, followed by padding that stands in for its tensors.
+func adapterHeader(kind string, pad int) []byte {
+	var b bytes.Buffer
+	w := func(v any) { _ = binary.Write(&b, binary.LittleEndian, v) }
+	str := func(s string) { w(uint64(len(s))); b.WriteString(s) }
+	b.WriteString("GGUF")
+	w(uint32(3))
+	w(uint64(0))
+	w(uint64(3))
+	str("general.architecture")
+	w(uint32(tString))
+	str("qwen2")
+	str("general.type")
+	w(uint32(tString))
+	str(kind)
+	str("adapter.type")
+	w(uint32(tString))
+	str("lora")
+	b.Write(make([]byte, pad))
+	return b.Bytes()
+}
+
+func TestAnAdapterIsToldFromAModelByItsFirstBytes(t *testing.T) {
+	var asked string
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		asked = r.Header.Get("Range")
+		w.WriteHeader(http.StatusPartialContent)
+		if r.URL.Path == "/adapter.gguf" {
+			_, _ = w.Write(adapterHeader("adapter", 1<<20))
+			return
+		}
+		_, _ = w.Write(header("qwen2"))
+	}))
+	defer srv.Close()
+
+	h, err := KindFromURL(t.Context(), srv.Client(), srv.URL+"/adapter.gguf")
+	if err != nil || !h.IsAdapter() || h.Architecture != "qwen2" {
+		t.Errorf("an adapter read as %+v, %v", h, err)
+	}
+	if asked != "bytes=0-262143" {
+		t.Errorf("asked for %q: the probe should read the start only", asked)
+	}
+	m, err := KindFromURL(t.Context(), srv.Client(), srv.URL+"/model.gguf")
+	if err != nil || m.IsAdapter() || m.Architecture != "qwen2" {
+		t.Errorf("a model read as %+v, %v", m, err)
+	}
+}

internal/hf/hf.go

@@ -156,6 +156,10 @@ type Query struct {
 	// applies it, so a page of small models is a full page.
 	MaxParams int64
 	Limit     int
+	// Filters are further registry tags every result must carry, such as
+	// "lora", or "base_model:adapter:Qwen/Qwen2.5-7B-Instruct" for the
+	// adapters made for one model.
+	Filters []string
 }
 
 // exploreFields is everything a listing shows. Asking for any field with
@@ -188,7 +192,10 @@ func (c *Client) Explore(ctx context.Context, q Query) ([]Model, error) {
 		v.Set("pipeline_tag", q.Task)
 	}
 	if q.GGUF {
-		v.Set("filter", "gguf")
+		v.Add("filter", "gguf")
+	}
+	for _, f := range q.Filters {
+		v.Add("filter", f)
 	}
 	if q.MaxParams > 0 {
 		v.Set("num_parameters", fmt.Sprintf("min:0,max:%d", q.MaxParams))

internal/release/release.go

@@ -9,6 +9,7 @@ package release
 
 import (
 	"bufio"
+	"bytes"
 	"context"
 	"crypto/sha256"
 	"encoding/hex"
@@ -51,7 +52,7 @@ func Install(ctx context.Context, client *http.Client, base, version, goos, goar
 	asset := AssetName(version, goos, goarch)
 	root := strings.TrimSuffix(base, "/") + "/" + version + "/"
 
-	want, err := checksum(ctx, client, root+"checksums.txt", asset)
+	want, err := checksum(ctx, client, root, asset)
 	if err != nil {
 		return "", err
 	}
@@ -85,13 +86,35 @@ func Install(ctx context.Context, client *http.Client, base, version, goos, goar
 	return backup, nil
 }
 
-func checksum(ctx context.Context, client *http.Client, url, asset string) (string, error) {
-	body, err := get(ctx, client, url)
+// checksum reads the release's checksum for asset — after checking the
+// checksums' signature, when this build trusts a signing key. A release with
+// no signature is then refused rather than taken on its checksums: a missing
+// signature is exactly what replacing a release would look like.
+func checksum(ctx context.Context, client *http.Client, root, asset string) (string, error) {
+	body, err := get(ctx, client, root+"checksums.txt")
 	if err != nil {
 		return "", err
 	}
 	defer body.Close()
-	sc := bufio.NewScanner(io.LimitReader(body, 1<<20))
+	sums, err := io.ReadAll(io.LimitReader(body, 1<<20))
+	if err != nil {
+		return "", err
+	}
+	if key := TrustedKey(); key != "" {
+		sigBody, err := get(ctx, client, root+SignatureName)
+		if err != nil {
+			return "", fmt.Errorf("this build only installs signed releases, and %s could not be read: %w", SignatureName, err)
+		}
+		sig, err := io.ReadAll(io.LimitReader(sigBody, 4096))
+		sigBody.Close()
+		if err != nil {
+			return "", err
+		}
+		if err := Verify(key, sums, string(sig)); err != nil {
+			return "", err
+		}
+	}
+	sc := bufio.NewScanner(bytes.NewReader(sums))
 	for sc.Scan() {
 		f := strings.Fields(sc.Text())
 		if len(f) == 2 && f[1] == asset && len(f[0]) == 64 {

internal/release/sign.go

@@ -0,0 +1,72 @@
+package release
+
+import (
+	"crypto/ed25519"
+	"crypto/rand"
+	_ "embed"
+	"encoding/base64"
+	"errors"
+	"fmt"
+	"os"
+	"strings"
+)
+
+// trustedPub is the public key releases are signed with, base64, built into
+// every binary from the repository. Empty — as it is until a key is made —
+// trusts a release on its checksums alone, as every release before this did.
+//
+// Built in rather than fetched, because the point of a signature is that the
+// server the release comes from cannot vouch for it: a machine that already
+// runs llmgw checks the next release against the key the release it runs was
+// built with, so replacing both the binaries and the checksums on the server
+// is no longer enough.
+//
+//go:embed trusted.pub
+var trustedPub string
+
+// TrustedKey is the key releases are verified with: LLMGW_RELEASE_KEY when set
+// (for rotating a key, or testing), the built-in one otherwise; empty is none.
+func TrustedKey() string {
+	if k := strings.TrimSpace(os.Getenv("LLMGW_RELEASE_KEY")); k != "" {
+		return k
+	}
+	return strings.TrimSpace(trustedPub)
+}
+
+// SignatureName is the file a release's checksums are signed in.
+const SignatureName = "checksums.txt.sig"
+
+// GenerateKey makes a signing key pair, both base64: the private key is the
+// 32-byte seed, which is what goes into the CI secret.
+func GenerateKey() (public, private string, err error) {
+	pub, priv, err := ed25519.GenerateKey(rand.Reader)
+	if err != nil {
+		return "", "", err
+	}
+	return base64.StdEncoding.EncodeToString(pub), base64.StdEncoding.EncodeToString(priv.Seed()), nil
+}
+
+// Sign signs data with a base64 seed, returning the signature in base64.
+func Sign(privateSeed string, data []byte) (string, error) {
+	seed, err := base64.StdEncoding.DecodeString(strings.TrimSpace(privateSeed))
+	if err != nil || len(seed) != ed25519.SeedSize {
+		return "", errors.New("the signing key is not a base64 ed25519 seed")
+	}
+	return base64.StdEncoding.EncodeToString(ed25519.Sign(ed25519.NewKeyFromSeed(seed), data)), nil
+}
+
+// Verify checks a base64 signature of data against a base64 public key.
+func Verify(public string, data []byte, signature string) error {
+	pub, err := base64.StdEncoding.DecodeString(strings.TrimSpace(public))
+	if err != nil || len(pub) != ed25519.PublicKeySize {
+		return errors.New("the trusted release key is not a base64 ed25519 public key")
+	}
+	sig, err := base64.StdEncoding.DecodeString(strings.TrimSpace(signature))
+	if err != nil {
+		return fmt.Errorf("the release signature is not base64: %w", err)
+	}
+	if !ed25519.Verify(pub, data, sig) {
+		return errors.New("the release's checksums are not signed by the trusted key")
+	}
+	return nil
+}

internal/release/sign_test.go

@@ -0,0 +1,99 @@
+package release
+
+import (
+	"crypto/sha256"
+	"encoding/hex"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// signedServer publishes a release whose checksums are signed with seed,
+// or not signed at all when seed is empty; tamper rewrites the checksums
+// after signing.
+func signedServer(t *testing.T, seed string, body []byte, tamper bool) (*httptest.Server, string) {
+	t.Helper()
+	asset := AssetName("v2.0.0", "linux", "amd64")
+	sum := sha256.Sum256(body)
+	sums := hex.EncodeToString(sum[:]) + "  " + asset + "\n"
+	sig := ""
+	if seed != "" {
+		var err error
+		if sig, err = Sign(seed, []byte(sums)); err != nil {
+			t.Fatal(err)
+		}
+	}
+	if tamper {
+		sums += strings.Repeat("0", 64) + "  extra-file\n"
+	}
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/v2.0.0/checksums.txt":
+			_, _ = w.Write([]byte(sums))
+		case "/v2.0.0/" + SignatureName:
+			if sig == "" {
+				http.NotFound(w, r)
+				return
+			}
+			_, _ = w.Write([]byte(sig + "\n"))
+		case "/v2.0.0/" + asset:
+			_, _ = w.Write(body)
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	t.Cleanup(srv.Close)
+	return srv, asset
+}
+
+func TestABuildThatTrustsAKeyInstallsOnlyReleasesSignedWithIt(t *testing.T) {
+	pub, priv, err := GenerateKey()
+	if err != nil {
+		t.Fatal(err)
+	}
+	_, otherPriv, _ := GenerateKey()
+	t.Setenv("LLMGW_RELEASE_KEY", pub)
+	install := func(srv *httptest.Server) error {
+		target := filepath.Join(t.TempDir(), "llmgw")
+		_ = os.WriteFile(target, []byte("old"), 0o755)
+		_, err := Install(t.Context(), srv.Client(), srv.URL, "v2.0.0", "linux", "amd64", target)
+		return err
+	}
+	body := []byte("new binary")
+
+	good, _ := signedServer(t, priv, body, false)
+	if err := install(good); err != nil {
+		t.Fatalf("a release signed with the trusted key was refused: %v", err)
+	}
+	for name, srv := range map[string]*httptest.Server{
+		"unsigned":            func() *httptest.Server { s, _ := signedServer(t, "", body, false); return s }(),
+		"signed by another":   func() *httptest.Server { s, _ := signedServer(t, otherPriv, body, false); return s }(),
+		"checksums rewritten": func() *httptest.Server { s, _ := signedServer(t, priv, body, true); return s }(),
+	} {
+		if err := install(srv); err == nil {
+			t.Errorf("%s: installed", name)
+		}
+	}
+
+	// A build that trusts no key installs on checksums, as before.
+	t.Setenv("LLMGW_RELEASE_KEY", "")
+	if TrustedKey() != "" {
+		t.Skip("this build embeds a trusted key; the unsigned path cannot be tested here")
+	}
+	unsigned, _ := signedServer(t, "", body, false)
+	if err := install(unsigned); err != nil {
+		t.Errorf("without a trusted key an unsigned release was refused: %v", err)
+	}
+}
+
+func TestAKeyThatIsNotOneIsSaid(t *testing.T) {
+	if _, err := Sign("not-a-key", []byte("x")); err == nil {
+		t.Error("signed with a malformed key")
+	}
+	if err := Verify("bm9wZQ==", []byte("x"), "c2ln"); err == nil || !strings.Contains(err.Error(), "public key") {
+		t.Errorf("a malformed trusted key: %v", err)
+	}
+}

internal/router/adapter_test.go

@@ -0,0 +1,98 @@
+package router
+
+import (
+	"encoding/json"
+	"net/http"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/auth"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/config"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+func adapterModel() catalog.Model {
+	m := catModel("coder", nil)
+	m.Profiles = []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 4}}
+	m.Adapters = []catalog.Adapter{
+		{Name: "coder-sql", Source: "http://mirror/sql.gguf"},
+		{Name: "coder-docs", Source: "http://mirror/docs.gguf", Scale: 0.5},
+	}
+	return m
+}
+
+func TestAnAdapterIsAskedForByNameAndAppliedOnItsModel(t *testing.T) {
+	e := newFakeEngine(t, "coder")
+	cfg := baseConfig(config.Endpoint{ID: "a", URL: e.srv.URL, Models: []string{"coder"}})
+	cfg.Models = []catalog.Model{adapterModel()}
+	rt := newTestRouter(t, cfg)
+
+	sent := func(model string) map[string]any {
+		t.Helper()
+		rec := post(t, rt.Handler(), map[string]any{"model": model, "messages": []any{}}, nil)
+		if rec.Code != http.StatusOK {
+			t.Fatalf("%s: status %d: %s", model, rec.Code, rec.Body.String())
+		}
+		var body map[string]any
+		_ = json.Unmarshal(e.last.Load().([]byte), &body)
+		return body
+	}
+	body := sent("coder-docs")
+	if body["model"] != "coder" {
+		t.Errorf("the engine was asked for %v, not the model the adapter runs on", body["model"])
+	}
+	lora, _ := json.Marshal(body["lora"])
+	if string(lora) != `[{"id":1,"scale":0.5}]` {
+		t.Errorf("lora %s, expected the second adapter at its scale", lora)
+	}
+	if body := sent("coder-sql"); func() string { b, _ := json.Marshal(body["lora"]); return string(b) }() != `[{"id":0,"scale":1}]` {
+		t.Errorf("the first adapter, at the default scale: %v", body["lora"])
+	}
+	if body := sent("coder"); body["lora"] != nil {
+		t.Errorf("the base model was sent an adapter: %v", body["lora"])
+	}
+}
+
+func TestAKeyAllowedAnAdapterReachesThatAdapterOnly(t *testing.T) {
+	cfg := baseConfig()
+	cfg.Models = []catalog.Model{adapterModel()}
+	rt := newTestRouter(t, cfg)
+	key := func(allowed ...string) auth.Principal {
+		return auth.Principal{Limits: store.Effective{AllowedModels: allowed}}
+	}
+	for _, c := range []struct {
+		allowed   []string
+		requested string
+		want      bool
+	}{
+		{[]string{"coder-sql"}, "coder-sql", true},
+		{[]string{"coder-sql"}, "coder", false},
+		{[]string{"coder-sql"}, "coder-docs", false},
+		{[]string{"coder"}, "coder-sql", true}, // the model allows its adapters
+		{[]string{"coder"}, "coder", true},
+	} {
+		if got := rt.permits(key(c.allowed...), c.requested); got != c.want {
+			t.Errorf("allowed %v, asked for %s: %v", c.allowed, c.requested, got)
+		}
+	}
+}
+
+func TestAdaptersShareTheNamespaceOfModels(t *testing.T) {
+	a := adapterModel()
+	b := catModel("coder-sql", nil)
+	if _, err := catalog.New([]catalog.Model{a, b}); err == nil {
+		t.Error("an adapter named like another model was accepted")
+	}
+	bad := adapterModel()
+	bad.Profiles = []catalog.Profile{{Name: "gpu", Engine: "vllm", GPUClass: "h100-80g", TP: 1}}
+	if _, err := catalog.New([]catalog.Model{bad}); err == nil {
+		t.Error("adapters on a vLLM profile were accepted")
+	}
+	c, err := catalog.New([]catalog.Model{adapterModel()})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m, ad, i, ok := c.ResolveAdapter("coder-docs"); !ok || m.Name != "coder" || i != 1 || ad.EffectiveScale() != 0.5 {
+		t.Errorf("resolve: %v %+v %d %v", m.Name, ad, i, ok)
+	}
+}

internal/router/admission/controller.go

@@ -34,6 +34,12 @@ type Config struct {
 	// QueueCapacityPerBand bounds each band's queue. Rejecting immediately
 	// beats accumulating silently.
 	QueueCapacityPerBand int `koanf:"queue_capacity_per_band"`
+	// QueueCapacityTotal bounds everything queued, across bands. When it is
+	// reached, a request is not refused while a lower band has requests
+	// queued: the newest of the lowest band is turned away instead
+	// (displacement), since the oldest have waited longest and are nearest
+	// their TTL. Zero takes half of every band's capacity together.
+	QueueCapacityTotal int `koanf:"queue_capacity_total"`
 	// Interval is how often the dispatcher re-checks saturation. Releases also
 	// wake it, so this is only the backstop for saturation falling on its own.
 	Interval time.Duration `koanf:"interval"`
@@ -94,6 +100,9 @@ func (c *Config) normalize() error {
 	if c.QueueCapacityPerBand <= 0 {
 		c.QueueCapacityPerBand = 2000
 	}
+	if c.QueueCapacityTotal <= 0 {
+		c.QueueCapacityTotal = max(c.QueueCapacityPerBand*len(c.Bands)/2, 1)
+	}
 	if c.Interval <= 0 {
 		c.Interval = 10 * time.Millisecond
 	}
@@ -123,6 +132,10 @@ var (
 	ErrQueueTimeout = errors.New("admission queue timeout")
 	// ErrClosed means the controller is shutting down.
 	ErrClosed = errors.New("admission controller closed")
+	// ErrDisplaced means a queued request was turned away to make room for
+	// one of a higher band. Worth retrying: the queue is full now, not for
+	// good.
+	ErrDisplaced = errors.New("displaced from the admission queue by higher-priority requests")
 )
 
 // Stats is a snapshot of one band's queue, for metrics.
@@ -259,6 +272,12 @@ func (c *Controller) Acquire(ctx context.Context, r Request) (*Lease, error) {
 		c.mu.Unlock()
 		return nil, fmt.Errorf("%w: band %q", ErrQueueFull, band.Name)
 	}
+	if c.queuedLocked() >= c.cfg.QueueCapacityTotal && !c.displaceBelowLocked(idx) {
+		c.rejected[band.Name]++
+		c.mu.Unlock()
+		return nil, fmt.Errorf("%w: %d queued in all, and nothing of a lower band to make room",
+			ErrQueueFull, c.cfg.QueueCapacityTotal)
+	}
 
 	w := &waiter{
 		tenant:   tenantKey(r),
@@ -279,6 +298,10 @@ func (c *Controller) Acquire(ctx context.Context, r Request) (*Lease, error) {
 			c.mu.Unlock()
 			return nil, ErrClosed
 		}
+		if w.err != nil {
+			c.mu.Unlock()
+			return nil, w.err
+		}
 		c.waitTotal[band.Name] += time.Since(w.enqueued)
 		c.mu.Unlock()
 		return &Lease{c: c, band: band.Name, model: r.Model}, nil
@@ -297,8 +320,9 @@ func (c *Controller) Acquire(ctx context.Context, r Request) (*Lease, error) {
 func (c *Controller) remove(idx int, w *waiter, reason string) {
 	c.mu.Lock()
 	defer c.mu.Unlock()
-	if w.admitted {
-		// It won the race; the caller will see ready closed next time round.
+	if w.admitted || w.err != nil {
+		// It won the race, or was displaced; the caller will see ready
+		// closed next time round.
 		return
 	}
 	c.queues[idx].remove(w)
@@ -410,3 +434,31 @@ func tenantKey(r Request) string {
 	}
 	return "anonymous"
 }
+
+// queuedLocked is how many requests wait in every band. Called with mu held.
+func (c *Controller) queuedLocked() int {
+	n := 0
+	for _, q := range c.queues {
+		n += q.len()
+	}
+	return n
+}
+
+// displaceBelowLocked turns away the newest request of the lowest band below
+// idx that has any queued, and says whether there was one. Called with mu held.
+func (c *Controller) displaceBelowLocked(idx int) bool {
+	// Bands are sorted highest priority first, so lower bands have larger
+	// indices, and the lowest is last.
+	for j := len(c.queues) - 1; j > idx; j-- {
+		w := c.queues[j].newest()
+		if w == nil {
+			continue
+		}
+		c.queues[j].remove(w)
+		w.err = fmt.Errorf("%w (band %q)", ErrDisplaced, c.cfg.Bands[j].Name)
+		close(w.ready)
+		c.rejected[c.cfg.Bands[j].Name+"/displaced"]++
+		return true
+	}
+	return false
+}

internal/router/admission/displace_test.go

@@ -0,0 +1,89 @@
+package admission
+
+import (
+	"context"
+	"errors"
+	"testing"
+	"time"
+)
+
+// queueUntil starts an Acquire in the background and waits until the band
+// holds depth requests, returning where its result will arrive.
+func queueUntil(t *testing.T, c *Controller, band string, bandIdx, depth int) <-chan error {
+	t.Helper()
+	out := make(chan error, 1)
+	go func() {
+		l, err := c.Acquire(context.Background(), Request{Model: "m", Band: band})
+		if err == nil {
+			l.Release()
+		}
+		out <- err
+	}()
+	deadline := time.Now().Add(2 * time.Second)
+	for c.Depths()[bandIdx].Depth < depth && time.Now().Before(deadline) {
+		time.Sleep(time.Millisecond)
+	}
+	if got := c.Depths()[bandIdx].Depth; got != depth {
+		t.Fatalf("band %s holds %d, expected %d", band, got, depth)
+	}
+	return out
+}
+
+func TestAHigherBandDisplacesTheNewestOfTheLowest(t *testing.T) {
+	s := &fixedSat{}
+	s.set(5) // past every ceiling: everything queues
+	cfg := testConfig()
+	cfg.QueueCapacityTotal = 3
+	c := newController(t, cfg, s.fn)
+	// Bands, highest first: platinum 0, gold 1, silver 2, bronze 3.
+	oldest := queueUntil(t, c, "bronze", 3, 1)
+	time.Sleep(2 * time.Millisecond)
+	silver := queueUntil(t, c, "silver", 2, 1)
+	time.Sleep(2 * time.Millisecond)
+	newest := queueUntil(t, c, "bronze", 3, 2)
+
+	// Full at three. Gold arrives: the newest bronze goes, not the oldest,
+	// and not silver, which outranks bronze.
+	gold := queueUntil(t, c, "gold", 1, 1)
+	select {
+	case err := <-newest:
+		if !errors.Is(err, ErrDisplaced) {
+			t.Fatalf("the newest bronze got %v", err)
+		}
+	case <-time.After(time.Second):
+		t.Fatal("nothing was displaced")
+	}
+	if d := c.Depths(); d[3].Depth != 1 || d[2].Depth != 1 || d[1].Depth != 1 {
+		t.Errorf("depths %+v: one bronze, the silver and the gold should remain", d)
+	}
+
+	// Bronze cannot displace anything: nothing is below it.
+	_, err := c.Acquire(t.Context(), Request{Model: "m", Band: "bronze"})
+	if !errors.Is(err, ErrQueueFull) {
+		t.Errorf("a bronze request at a full queue got %v", err)
+	}
+	// Another gold takes the remaining bronze, then silver.
+	queueUntil(t, c, "gold", 1, 2)
+	if err := <-oldest; !errors.Is(err, ErrDisplaced) {
+		t.Errorf("the last bronze got %v", err)
+	}
+	queueUntil(t, c, "gold", 1, 3)
+	if err := <-silver; !errors.Is(err, ErrDisplaced) {
+		t.Errorf("silver got %v", err)
+	}
+	// All gold now: a fourth has nobody lower to displace.
+	if _, err := c.Acquire(t.Context(), Request{Model: "m", Band: "gold"}); !errors.Is(err, ErrQueueFull) {
+		t.Errorf("a gold request with only gold queued got %v", err)
+	}
+
+	// And they are served once there is room.
+	s.set(0)
+	select {
+	case err := <-gold:
+		if err != nil {
+			t.Errorf("the first gold was not served: %v", err)
+		}
+	case <-time.After(2 * time.Second):
+		t.Error("the queue did not drain")
+	}
+}

internal/router/admission/queue.go

@@ -12,6 +12,9 @@ type waiter struct {
 	// admitted guards the race between the dispatcher admitting a waiter and
 	// the waiter giving up on its own.
 	admitted bool
+	// err is why the waiter was turned away while queued — displaced by a
+	// higher band — set before ready is closed.
+	err error
 }
 
 // bandQueue holds one band's waiters, grouped by tenant.
@@ -166,3 +169,17 @@ func (q *bandQueue) dropExpired(now time.Time, onDrop func()) {
 		q.byTenant[tenant] = kept
 	}
 }
+
+// newest is the most recently queued waiter, across every tenant: the one
+// displacement takes, since it has invested the least waiting.
+func (q *bandQueue) newest() *waiter {
+	var out *waiter
+	for _, ws := range q.byTenant {
+		for _, w := range ws {
+			if out == nil || w.enqueued.After(out.enqueued) {
+				out = w
+			}
+		}
+	}
+	return out
+}

internal/router/endpoint/endpoint.go

@@ -37,8 +37,16 @@ type Info struct {
 	Weight float64
 	// Priority expresses pool preference; higher pools are tried first.
 	Priority int
+	// UpstreamModel is what the endpoint calls the model, when that differs
+	// from the catalog's name: an external provider's. Requests are rewritten
+	// to it. Empty sends the name as the client did.
+	UpstreamModel string
 }
 
+// EngineExternal marks an endpoint that is a provider outside the cluster.
+// It publishes no metrics, so it is never scraped and never reads as stale.
+const EngineExternal = "external"
+
 // Serves reports whether the replica serves the given model.
 func (i Info) Serves(model string) bool {
 	return slices.Contains(i.Models, model)
@@ -242,7 +250,7 @@ func (r *Registry) Snapshots(model string) []Snapshot {
 			continue
 		}
 		var age time.Duration
-		stale := true
+		stale := l.info.Engine != EngineExternal
 		if !l.scrapedAt.IsZero() {
 			age = now.Sub(l.scrapedAt)
 			stale = age > r.cfg.StaleAfter

internal/router/external_test.go

@@ -0,0 +1,89 @@
+package router
+
+import (
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"sync/atomic"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/config"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
+)
+
+func TestALocalModelWithNoCapacityFallsBackToAnExternalProvider(t *testing.T) {
+	local := newFakeEngine(t, "local")
+	local.status.Store(http.StatusServiceUnavailable)
+
+	var calls atomic.Int64
+	var gotModel, gotAuth atomic.Value
+	provider := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/metrics" {
+			t.Error("the provider was scraped for metrics")
+		}
+		calls.Add(1)
+		gotAuth.Store(r.Header.Get("Authorization"))
+		var body map[string]any
+		b, _ := io.ReadAll(r.Body)
+		_ = json.Unmarshal(b, &body)
+		gotModel.Store(fmt.Sprint(body["model"]))
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprint(w, `{"model":"gpt-4o-mini","choices":[{"message":{"content":"from outside"}}],"usage":{"prompt_tokens":3,"completion_tokens":2}}`)
+	}))
+	t.Cleanup(provider.Close)
+
+	cfg := baseConfig(config.Endpoint{ID: "l", URL: local.srv.URL, Models: []string{"local"}})
+	cloud := catalog.Model{Name: "cloud", Source: "external",
+		External: &catalog.External{URL: provider.URL, Model: "gpt-4o-mini"}}
+	cfg.Models = []catalog.Model{catModel("local", nil, "cloud"), cloud}
+	rt := newTestRouter(t, cfg)
+	rt.SetEndpoints(t.Context(), []endpoint.Info{
+		{ID: "l", URL: local.srv.URL, Models: []string{"local"}},
+		{ID: "external:cloud", URL: provider.URL, Models: []string{"cloud"}, APIKey: "provider-key",
+			Engine: endpoint.EngineExternal, Pool: "external", Weight: 1, UpstreamModel: "gpt-4o-mini"},
+	})
+
+	rec := post(t, rt.Handler(), map[string]any{"model": "local", "messages": []any{}}, nil)
+	if rec.Code != http.StatusOK {
+		t.Fatalf("status %d: %s", rec.Code, rec.Body.String())
+	}
+	if calls.Load() != 1 {
+		t.Fatalf("the provider was called %d times", calls.Load())
+	}
+	if gotModel.Load() != "gpt-4o-mini" {
+		t.Errorf("the provider was asked for %v, not its own name for the model", gotModel.Load())
+	}
+	if gotAuth.Load() != "Bearer provider-key" {
+		t.Errorf("the provider got %v, not its key", gotAuth.Load())
+	}
+	if got := rec.Header().Get("x-llmgw-served-model"); got != "cloud" {
+		t.Errorf("the client should be told it was served by cloud, got %q", got)
+	}
+	// Never scraped, and never stale for it.
+	for _, s := range rt.reg.Snapshots("cloud") {
+		if s.Stale {
+			t.Error("an external endpoint reads as stale")
+		}
+	}
+}
+
+func TestAnExternalModelIsNotAModelWithProfiles(t *testing.T) {
+	good := catalog.Model{Name: "cloud", Source: "external",
+		External: &catalog.External{URL: "https://api.example.com", Model: "gpt-4o-mini"}}
+	if _, err := catalog.New([]catalog.Model{good}); err != nil {
+		t.Fatalf("a valid external model was refused: %v", err)
+	}
+	for name, m := range map[string]catalog.Model{
+		"profiles": {Name: "x", Source: "external", External: &catalog.External{URL: "https://a.b", Model: "m"},
+			Profiles: []catalog.Profile{{Name: "p", Engine: "llamacpp"}}},
+		"no url":   {Name: "x", Source: "external", External: &catalog.External{URL: "api.example.com", Model: "m"}},
+		"no model": {Name: "x", Source: "external", External: &catalog.External{URL: "https://a.b"}},
+	} {
+		if _, err := catalog.New([]catalog.Model{m}); err == nil {
+			t.Errorf("%s: accepted", name)
+		}
+	}
+}

internal/router/finish_test.go

@@ -88,6 +88,9 @@ func TestARefusalBeforeDispatchSaysWhetherWaitingHelps(t *testing.T) {
 		{"queued too long", func(w http.ResponseWriter) {
 			rt.rejectAdmission(w, &parsedRequest{Model: "m"}, admission.ErrQueueTimeout, time.Now())
 		}, 504, ""},
+		{"displaced for a higher band", func(w http.ResponseWriter) {
+			rt.rejectAdmission(w, &parsedRequest{Model: "m"}, admission.ErrDisplaced, time.Now())
+		}, 503, "5"},
 		{"shutting down", func(w http.ResponseWriter) {
 			rt.rejectAdmission(w, &parsedRequest{Model: "m"}, admission.ErrClosed, time.Now())
 		}, 503, ""},

internal/router/handlers.go

@@ -60,9 +60,15 @@ func (r *Router) rejectAdmission(w http.ResponseWriter, req *parsedRequest, err
 		return
 	case errors.Is(err, admission.ErrQueueTimeout):
 		status = http.StatusGatewayTimeout
+	case errors.Is(err, admission.ErrDisplaced):
+		// Turned away for higher-priority work, not for good: said so the
+		// client backs off and comes back rather than giving up.
+		w.Header().Set("Retry-After", "5")
 	}
 	reason := "unavailable"
 	switch {
+	case errors.Is(err, admission.ErrDisplaced):
+		reason = "displaced"
 	case errors.Is(err, admission.ErrQueueFull):
 		reason = "queue_full"
 	case errors.Is(err, admission.ErrQueueTimeout):
@@ -185,6 +191,11 @@ func (r *Router) Handler() http.Handler {
 			pages.RegisterAdminUsers(mux, r.store)
 			// Quotas are written on the same tree the accounts page builds, so
 			// they are mounted together.
+			// Capacity quotas are the control plane's, reached through the
+			// same admin API the CLI uses.
+			if c, ok := r.clusterView.(web.CapacityAdmin); ok {
+				pages.SetCapacity(c)
+			}
 			pages.RegisterAdminQuotas(mux, r.store, r.acct)
 			// And the report of what that tree actually used.
 			pages.RegisterAdminUsage(mux, r.store, r.acct)
@@ -415,6 +426,12 @@ func (r *Router) handleInference(path string) http.HandlerFunc {
 // around the allowlist; matching only the canonical name would silently ignore
 // an allowlist written in the names clients actually use.
 func (r *Router) permits(p auth.Principal, requested string) bool {
+	// An adapter named in the allowlist allows that adapter: not its model,
+	// and not the model's other adapters, which the checks below would let
+	// through if the adapter's own name were not asked about first.
+	if _, ok := r.adapter(requested); ok && p.Allows(requested) {
+		return true
+	}
 	canonical, m := r.resolve(requested)
 	if p.Allows(canonical) {
 		return true
@@ -973,3 +990,44 @@ func (r *Router) rateLimitHeaders(w http.ResponseWriter, p auth.Principal) {
 		w.Header().Set("x-ratelimit-reset-tokens", reset)
 	}
 }
+
+// withModel is body with its model set to name, for an endpoint that knows the
+// model by another name — an external provider. The body unchanged when it is
+// not a JSON object, which the endpoint then refuses as it would have anyway.
+func withModel(body []byte, name string) []byte {
+	var m map[string]json.RawMessage
+	if err := json.Unmarshal(body, &m); err != nil {
+		return body
+	}
+	encoded, err := json.Marshal(name)
+	if err != nil {
+		return body
+	}
+	m["model"] = encoded
+	out, err := json.Marshal(m)
+	if err != nil {
+		return body
+	}
+	return out
+}
+
+// withAdapter is body asking for model with a LoRA adapter applied, in
+// llama.cpp's form: "lora": [{"id": index, "scale": scale}]. Adapters not
+// listed are applied at zero, so the list is the whole of it.
+func withAdapter(body []byte, model string, index int, scale float64) []byte {
+	var m map[string]json.RawMessage
+	if err := json.Unmarshal(body, &m); err != nil {
+		return body
+	}
+	name, err1 := json.Marshal(model)
+	lora, err2 := json.Marshal([]map[string]any{{"id": index, "scale": scale}})
+	if err1 != nil || err2 != nil {
+		return body
+	}
+	m["model"], m["lora"] = name, lora
+	out, err := json.Marshal(m)
+	if err != nil {
+		return body
+	}
+	return out
+}

internal/router/router.go

@@ -61,6 +61,8 @@ type Router struct {
 	// router's half of not having to. A request takes the map once and works
 	// from it, so a swap mid-request is invisible rather than a race.
 	models atomic.Pointer[map[string]catalog.Model]
+	// adapters are the LoRA adapters by the name clients ask for them by.
+	adapters atomic.Pointer[map[string]adapterRef]
 
 	// extraMetrics are collectors other components publish on /metrics.
 	extraMetrics []prometheus.Collector
@@ -315,14 +317,7 @@ func New(cfg config.Server, log *slog.Logger) (*Router, error) {
 	// all. A control plane pushes over the top of this at startup and on every
 	// change after.
 	if len(cfg.Models) > 0 {
-		seed := map[string]catalog.Model{}
-		for _, m := range cfg.Models {
-			seed[m.Name] = m
-			for _, a := range m.Aliases {
-				seed[a] = m
-			}
-		}
-		r.models.Store(&seed)
+		r.storeModels(cfg.Models)
 	}
 	return r, nil
 }
@@ -347,6 +342,7 @@ func admissionConfig(rc config.Router) admission.Config {
 		},
 		QueueTTL:             rc.Admission.QueueTTL,
 		QueueCapacityPerBand: rc.Admission.QueueCapacityPerBand,
+		QueueCapacityTotal:   rc.Admission.QueueCapacityTotal,
 		Interval:             rc.Admission.Interval,
 	}
 }
@@ -504,15 +500,44 @@ func (r *Router) modelMap() map[string]catalog.Model {
 // The catalog's counterpart to SetEndpoints, and pushed for the same reason:
 // the control plane owns what should exist, the router needs to know, and
 // neither has to be in the other's process for that to work.
-func (r *Router) SetCatalog(c *catalog.Catalog) {
+func (r *Router) SetCatalog(c *catalog.Catalog) { r.storeModels(c.Models()) }
+
+// storeModels makes models what names resolve to: each model by its name, its
+// aliases and its adapters. One place for it, whether the models came from
+// configuration at start or from the control plane since.
+func (r *Router) storeModels(models []catalog.Model) {
 	m := map[string]catalog.Model{}
-	for _, entry := range c.Models() {
+	ads := map[string]adapterRef{}
+	for _, entry := range models {
 		m[entry.Name] = entry
 		for _, a := range entry.Aliases {
 			m[a] = entry
 		}
+		// An adapter resolves to the model it runs on, as an alias does, and
+		// is applied to the request on the way there.
+		for i, a := range entry.Adapters {
+			m[a.Name] = entry
+			ads[a.Name] = adapterRef{base: entry.Name, index: i, scale: a.EffectiveScale()}
+		}
 	}
 	r.models.Store(&m)
+	r.adapters.Store(&ads)
+}
+
+// adapterRef is where an adapter runs and how it is applied.
+type adapterRef struct {
+	base  string
+	index int
+	scale float64
+}
+
+// adapter finds one by the name a client asked for.
+func (r *Router) adapter(name string) (adapterRef, bool) {
+	if p := r.adapters.Load(); p != nil {
+		a, ok := (*p)[name]
+		return a, ok
+	}
+	return adapterRef{}, false
 }
 
 // LastUsed reports when each endpoint was last sent a request, satisfying
@@ -714,10 +739,20 @@ func (r *Router) dispatchOne(ctx context.Context, w http.ResponseWriter, req *pa
 		uctx, uspan := tracer().Start(ctx, "upstream", trace.WithSpanKind(trace.SpanKindClient),
 			trace.WithAttributes(attribute.String("llmgw.endpoint", dec.Endpoint.ID),
 				attribute.String("url.full", dec.Endpoint.URL+req.Path)))
+		body := req.UpstreamBody
+		if dec.Endpoint.UpstreamModel != "" {
+			body = withModel(body, dec.Endpoint.UpstreamModel)
+		}
+		// An adapter asked for by name, served by its model's replicas: the
+		// engine is asked for the model, with the adapter applied. Not on a
+		// fallback to another model, which does not have it.
+		if a, ok := r.adapter(req.Model); ok && a.base == model {
+			body = withAdapter(body, model, a.index, a.scale)
+		}
 		out, perr := r.px.Do(uctx, w, proxy.Request{
 			URL:                dec.Endpoint.URL + req.Path,
 			APIKey:             dec.Endpoint.APIKey,
-			Body:               req.UpstreamBody,
+			Body:               body,
 			Header:             withTraceparent(uctx, req.Header),
 			Stream:             req.Stream,
 			SuppressUsageChunk: req.SuppressUsageChunk,

internal/router/scheduler/filters.go

@@ -1,6 +1,9 @@
 package scheduler
 
 import (
+	"sort"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/router/admission"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/router/breaker"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
 )
@@ -51,34 +54,41 @@ func KVSaturated(ceiling float64) Filter {
 	}
 }
 
-// PoolPriority keeps only the endpoints of the highest-priority pool
-// (plan §7.3, the two-stage decision: pool first, then replica).
+// PoolPriority keeps the endpoints of one pool: the highest-priority one,
+// unless it is saturated and a lower one is not (plan §7.3, the two-stage
+// decision — pool first, then replica).
 //
-// Spillover arrives in phase 5, at which point this filter will fall through
-// to the next pool once the preferred one is saturated.
-func PoolPriority() Filter {
-	return poolPriorityFilter{}
+// Saturated is what admission means by it, the same formula over the same
+// thresholds, so the two layers agree on when a pool is full: the H100 pool is
+// used until it is, and only then the A100 one — never the A100 because it
+// happens to be emptier. When every pool is saturated the preferred one is
+// kept: moving to a pool that is just as full buys nothing and loses whatever
+// made the first preferred.
+func PoolPriority(targetWaiting, kvCeiling float64) Filter {
+	return poolPriorityFilter{t: admission.Thresholds{TargetWaitingPerReplica: targetWaiting, KVCeiling: kvCeiling}}
 }
 
-type poolPriorityFilter struct{}
+type poolPriorityFilter struct{ t admission.Thresholds }
 
 func (poolPriorityFilter) Name() string { return "pool-priority" }
 
-func (poolPriorityFilter) Filter(_ *Ctx, eps []endpoint.Snapshot) []endpoint.Snapshot {
+func (f poolPriorityFilter) Filter(_ *Ctx, eps []endpoint.Snapshot) []endpoint.Snapshot {
 	if len(eps) == 0 {
 		return eps
 	}
-	best := eps[0].Priority
-	for _, e := range eps[1:] {
-		if e.Priority > best {
-			best = e.Priority
-		}
-	}
-	out := make([]endpoint.Snapshot, 0, len(eps))
+	byPriority := map[int][]endpoint.Snapshot{}
+	var order []int
 	for _, e := range eps {
-		if e.Priority == best {
-			out = append(out, e)
+		if _, ok := byPriority[e.Priority]; !ok {
+			order = append(order, e.Priority)
+		}
+		byPriority[e.Priority] = append(byPriority[e.Priority], e)
+	}
+	sort.Sort(sort.Reverse(sort.IntSlice(order)))
+	for _, p := range order {
+		if admission.Saturation(byPriority[p], 0, f.t) < 1 {
+			return byPriority[p]
 		}
 	}
-	return out
+	return byPriority[order[0]]
 }

internal/router/scheduler/profiles.go

@@ -50,7 +50,7 @@ func Chat(c ProfileConfig) *Profile {
 	c.normalize()
 	return &Profile{
 		Name:    "chat",
-		Filters: []Filter{BreakerOpen(), PoolPriority()},
+		Filters: []Filter{BreakerOpen(), PoolPriority(c.TargetWaiting, c.KVCeiling)},
 		Scorers: c.withPrefix([]WeightedScorer{
 			{Session(c.SessionLoadFactor), 3.0},
 			{Queue(c.TargetWaiting), 1.0},
@@ -81,7 +81,7 @@ func Batch(c ProfileConfig) *Profile {
 	c.normalize()
 	return &Profile{
 		Name:    "batch",
-		Filters: []Filter{BreakerOpen(), PoolPriority()},
+		Filters: []Filter{BreakerOpen(), PoolPriority(c.TargetWaiting, c.KVCeiling)},
 		Scorers: []WeightedScorer{
 			{Queue(c.TargetWaiting), 3.0},
 			{KV(c.KVCeiling), 1.0},
@@ -99,7 +99,7 @@ func LongContext(c ProfileConfig) *Profile {
 	c.normalize()
 	return &Profile{
 		Name:    "long-context",
-		Filters: []Filter{BreakerOpen(), PoolPriority()},
+		Filters: []Filter{BreakerOpen(), PoolPriority(c.TargetWaiting, c.KVCeiling)},
 		// Prefix matching earns more here than anywhere else: a long prompt is
 		// mostly prefill, and prefill is what a cache hit skips.
 		Scorers: c.withPrefix([]WeightedScorer{

internal/router/scheduler/scheduler_test.go

@@ -177,7 +177,7 @@ func TestStaleEndpointIsPenalizedNotDropped(t *testing.T) {
 func TestPoolPriority(t *testing.T) {
 	p := &Profile{
 		Name:    "t",
-		Filters: []Filter{PoolPriority()},
+		Filters: []Filter{PoolPriority(4, 0.9)},
 		Scorers: []WeightedScorer{{Queue(4), 1}},
 		Picker:  MaxScore(),
 	}
@@ -189,7 +189,8 @@ func TestPoolPriority(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	// The lower-priority pool must not win despite being emptier (spillover is phase 5).
+	// The lower-priority pool must not win for being emptier: the preferred
+	// one is not saturated (3 waiting of a healthy 4).
 	if d.Endpoint.ID != "h100" {
 		t.Errorf("pool priority was ignored: %s", d.Endpoint.ID)
 	}
@@ -383,3 +384,36 @@ func TestBuiltinProfilesSchedule(t *testing.T) {
 		}
 	}
 }
+
+func TestASaturatedPoolSpillsToTheNext(t *testing.T) {
+	p := &Profile{Name: "t", Filters: []Filter{PoolPriority(4, 0.9)},
+		Scorers: []WeightedScorer{{Queue(4), 1}}, Picker: MaxScore()}
+	h100 := func(w int) endpoint.Snapshot {
+		return ep("h100", waiting(w), func(s *endpoint.Snapshot) { s.Priority = 100; s.Pool = "ib-a/h100" })
+	}
+	a100 := func(w int) endpoint.Snapshot {
+		return ep("a100", waiting(w), func(s *endpoint.Snapshot) { s.Priority = 50; s.Pool = "ib-b/a100" })
+	}
+	pick := func(eps ...endpoint.Snapshot) string {
+		d, err := p.Schedule(newCtx(Request{Model: "m"}), eps)
+		if err != nil {
+			t.Fatal(err)
+		}
+		return d.Endpoint.ID
+	}
+	if got := pick(h100(8), a100(0)); got != "a100" {
+		t.Errorf("the preferred pool at twice its healthy queue did not spill: %s", got)
+	}
+	if got := pick(h100(8), a100(6)); got != "h100" {
+		t.Errorf("both pools saturated: the preferred one should be kept, got %s", got)
+	}
+	if got := pick(h100(4), a100(0)); got != "a100" {
+		t.Errorf("at exactly its healthy limit a pool is full: %s", got)
+	}
+	// KV pressure saturates a pool as queue depth does.
+	full := h100(0)
+	full.Stats.KVCacheUsage = 0.95
+	if got := pick(full, a100(0)); got != "a100" {
+		t.Errorf("a pool out of KV cache did not spill: %s", got)
+	}
+}

internal/router/scrape/poller.go

@@ -96,6 +96,9 @@ func (p *Poller) Sync(ctx context.Context) {
 
 	want := make(map[string]endpoint.Info, len(infos))
 	for _, in := range infos {
+		if in.Engine == endpoint.EngineExternal {
+			continue // a provider elsewhere publishes no /metrics
+		}
 		want[in.ID] = in
 	}
 

internal/web/account.go

@@ -9,6 +9,7 @@ import (
 	"net/http"
 	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"gitea.mynodes.xyz/ilker/llmgw/internal/account"
@@ -61,6 +62,8 @@ type AccountPages struct {
 	hasMirror bool
 	// hasQuotas is set when /admin/quotas is registered.
 	hasQuotas bool
+	// capacity sets capacity quotas, from the quotas page; nil leaves them off.
+	capacity CapacityAdmin
 	// personalOff refuses personal keys; see SetPersonalOff.
 	personalOff bool
 	// personalDefaults is what personal use falls under by default.
@@ -74,6 +77,9 @@ type AccountPages struct {
 	priced Priced
 	// registry browses a model registry, or is nil where there is no way out.
 	registry Registry
+	// kinds remembers what each held GGUF file is, by its address: a held
+	// version never changes, so it is read once.
+	kinds sync.Map
 	// catalog puts a mirrored model into service; fleet says whether anything
 	// could run it. Both nil where the mirror page is read-only.
 	catalog CatalogAdmin

internal/web/admin_capacity.go

@@ -0,0 +1,138 @@
+package web
+
+import (
+	"context"
+	"fmt"
+	"net/http"
+	"strconv"
+	"strings"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// CapacityAdmin sets what parts of the tenancy tree are guaranteed of the
+// cluster and may hold. The admin API implements it; the page and the CLI go
+// through the same checks.
+type CapacityAdmin interface {
+	QuotaViews(ctx context.Context) ([]adminapi.QuotaView, error)
+	SetQuota(ctx context.Context, req adminapi.QuotaRequest, by string) (string, error)
+	ClearQuota(ctx context.Context, scope string) (string, error)
+	Scopes(ctx context.Context) ([]adminapi.ScopeName, error)
+}
+
+// SetCapacity gives the quotas page capacity quotas to show and set. Without
+// it the section is left off.
+func (p *AccountPages) SetCapacity(c CapacityAdmin) { p.capacity = c }
+
+// capacityRow is one scope's capacity quota as the page shows it.
+type capacityRow struct {
+	adminapi.QuotaView
+	GPUs, Cores, Memory string
+}
+
+// capacityForm is what the form starts with: a scope's quota to change, or
+// nothing for a new one.
+type capacityForm struct {
+	Scope               string
+	GPUs, Cores, Memory string
+	LGPUs, LCores, LMem string
+}
+
+func (p *AccountPages) capacitySection(ctx context.Context, d *adminQuotasData, editing string) {
+	if p.capacity == nil {
+		return
+	}
+	d.HasCapacity = true
+	views, err := p.capacity.QuotaViews(ctx)
+	if err != nil {
+		d.Error = err.Error()
+		return
+	}
+	for _, v := range views {
+		d.Capacity = append(d.Capacity, capacityRow{QuotaView: v,
+			GPUs:   capCell(v.Holds.GPUs, v.Guarantee.GPUs, v.Limit.GPUs),
+			Cores:  capCell(v.Holds.CPUCores, v.Guarantee.CPUCores, v.Limit.CPUCores),
+			Memory: capCell(v.Holds.MemoryGB, v.Guarantee.MemoryGB, v.Limit.MemoryGB)})
+		if v.Scope == editing {
+			d.CapForm = capacityForm{Scope: v.Scope,
+				GPUs: intStr(v.Guarantee.GPUs), Cores: intStr(v.Guarantee.CPUCores), Memory: intStr(v.Guarantee.MemoryGB),
+				LGPUs: intStr(v.Limit.GPUs), LCores: intStr(v.Limit.CPUCores), LMem: intStr(v.Limit.MemoryGB)}
+		}
+	}
+	if d.Scopes, err = p.capacity.Scopes(ctx); err != nil {
+		d.Error = err.Error()
+	}
+}
+
+// capCell is "6 of 8 (at most 16)": held, guaranteed, limit.
+func capCell(holds int, g, l *int) string {
+	if g == nil && l == nil {
+		return ""
+	}
+	out := strconv.Itoa(holds)
+	if g != nil {
+		out += " of " + strconv.Itoa(*g)
+	}
+	if l != nil {
+		out += fmt.Sprintf(" (at most %d)", *l)
+	}
+	return out
+}
+
+func intStr(v *int) string {
+	if v == nil {
+		return ""
+	}
+	return strconv.Itoa(*v)
+}
+
+// applyCapacity sets or clears one scope's capacity quota from the form.
+func (p *AccountPages) applyCapacity(r *http.Request, admin store.User) (string, error) {
+	scope := strings.TrimSpace(r.FormValue("scope"))
+	if scope == "" {
+		return "", fmt.Errorf("choose where the quota applies")
+	}
+	if r.FormValue("clear") != "" {
+		key, err := p.capacity.ClearQuota(r.Context(), scope)
+		if err != nil {
+			return "", err
+		}
+		return "capacity quota removed from " + key, nil
+	}
+	field := func(name, what string) (*int, error) {
+		v := strings.TrimSpace(r.FormValue(name))
+		if v == "" {
+			return nil, nil
+		}
+		n, err := strconv.Atoi(v)
+		if err != nil || n < 0 {
+			return nil, fmt.Errorf("%s must be a whole number, or blank for none", what)
+		}
+		return &n, nil
+	}
+	var req adminapi.QuotaRequest
+	req.Scope = scope
+	var err error
+	for _, f := range []struct {
+		dst  **int
+		name string
+		what string
+	}{
+		{&req.Guarantee.GPUs, "gpus", "guaranteed GPUs"},
+		{&req.Guarantee.CPUCores, "cores", "guaranteed cores"},
+		{&req.Guarantee.MemoryGB, "memory", "guaranteed memory"},
+		{&req.Limit.GPUs, "limit_gpus", "the GPU limit"},
+		{&req.Limit.CPUCores, "limit_cores", "the core limit"},
+		{&req.Limit.MemoryGB, "limit_memory", "the memory limit"},
+	} {
+		if *f.dst, err = field(f.name, f.what); err != nil {
+			return "", err
+		}
+	}
+	key, err := p.capacity.SetQuota(r.Context(), req, admin.Email)
+	if err != nil {
+		return "", err
+	}
+	return "capacity quota set on " + key + "; it takes effect within half a minute", nil
+}

internal/web/admin_capacity_test.go

@@ -0,0 +1,88 @@
+package web
+
+import (
+	"context"
+	"errors"
+	"log/slog"
+	"net/http"
+	"net/url"
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+type fakeCapacity struct {
+	views   []adminapi.QuotaView
+	set     []adminapi.QuotaRequest
+	cleared []string
+}
+
+func (f *fakeCapacity) QuotaViews(context.Context) ([]adminapi.QuotaView, error) { return f.views, nil }
+func (f *fakeCapacity) SetQuota(_ context.Context, r adminapi.QuotaRequest, _ string) (string, error) {
+	if r.Scope == "team:nope" {
+		return "", errors.New(`there is no team "nope"`)
+	}
+	f.set = append(f.set, r)
+	return r.Scope, nil
+}
+func (f *fakeCapacity) ClearQuota(_ context.Context, s string) (string, error) {
+	f.cleared = append(f.cleared, s)
+	return s, nil
+}
+func (f *fakeCapacity) Scopes(context.Context) ([]adminapi.ScopeName, error) {
+	return []adminapi.ScopeName{{Key: "org:o1", Name: "Acme"}, {Key: "team:t1", Name: "Acme / ml"}}, nil
+}
+
+func TestCapacityQuotasAreShownAndSetFromTheQuotasPage(t *testing.T) {
+	eight, sixteen := 8, 16
+	fc := &fakeCapacity{views: []adminapi.QuotaView{{Scope: "team:t1", Name: "Acme / ml",
+		Guarantee: store.Resources{GPUs: &eight}, Limit: store.Resources{GPUs: &sixteen},
+		Holds: quota.Amount{GPUs: 6}, UpdatedBy: "ops@acme"}}}
+	a := fakeAccounts{user: store.User{ID: "u1", Email: "a@example.com", IsAdmin: true}}
+	p := NewAccountPages(a, nil, nil, "v0", slog.New(slog.DiscardHandler))
+	p.SetCapacity(fc)
+	mux := http.NewServeMux()
+	p.RegisterAdminQuotas(mux, &fakeQuotas{}, pricedYes{})
+
+	body := mirrorGet(t, mux, "/admin/quotas").Body.String()
+	for _, want := range []string{`id="capacity"`, "Acme / ml", "6 of 8 (at most 16)", "ops@acme",
+		`<option value="team:t1">Acme / ml</option>`} {
+		if !strings.Contains(body, want) {
+			t.Errorf("the page lacks %q", want)
+		}
+	}
+	// "change" fills the form with the scope's figures.
+	edit := mirrorGet(t, mux, "/admin/quotas?cap=team:t1").Body.String()
+	if !strings.Contains(edit, `<option value="team:t1" selected>`) || !strings.Contains(edit, `name="limit_gpus" value="16"`) {
+		t.Error("change did not open the form on the scope's quota")
+	}
+
+	res := mirrorPost(t, mux, "/admin/quotas/capacity", url.Values{"csrf": {"tok"}, "scope": {"team:t1"},
+		"gpus": {"8"}, "cores": {""}, "limit_memory": {"64"}}).Body.String()
+	if len(fc.set) != 1 || *fc.set[0].Guarantee.GPUs != 8 || fc.set[0].Guarantee.CPUCores != nil || *fc.set[0].Limit.MemoryGB != 64 {
+		t.Errorf("sent %+v", fc.set)
+	}
+	if !strings.Contains(res, "capacity quota set on team:t1") {
+		t.Errorf("no word of it: %s", res)
+	}
+	for _, bad := range []url.Values{{"scope": {"team:t1"}, "gpus": {"eight"}}, {"scope": {"team:nope"}, "gpus": {"1"}}, {"gpus": {"1"}}} {
+		bad.Set("csrf", "tok")
+		n := len(fc.set)
+		out := mirrorPost(t, mux, "/admin/quotas/capacity", bad).Body.String()
+		if len(fc.set) != n || !strings.Contains(out, `class="error"`) {
+			t.Errorf("%v was not refused with a reason", bad)
+		}
+	}
+	mirrorPost(t, mux, "/admin/quotas/capacity", url.Values{"csrf": {"tok"}, "scope": {"team:t1"}, "clear": {"1"}})
+	if len(fc.cleared) != 1 || fc.cleared[0] != "team:t1" {
+		t.Errorf("cleared %v", fc.cleared)
+	}
+	// Without a CSRF token nothing happens.
+	mirrorPost(t, mux, "/admin/quotas/capacity", url.Values{"scope": {"team:t1"}, "clear": {"1"}})
+	if len(fc.cleared) != 1 {
+		t.Error("a change was made without the CSRF token")
+	}
+}

internal/web/admin_mirror.go

@@ -11,6 +11,7 @@ import (
 	"sort"
 	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
@@ -46,6 +47,10 @@ type Registry interface {
 
 // repoSet is one thing worth fetching: a file, or every shard of one.
 type repoSet struct {
+	// Kind is what the file says it is, read from its first bytes: "adapter"
+	// for a LoRA adapter, "model", or empty when it was not read. Arch is the
+	// architecture it names, which an adapter must share with its model.
+	Kind, Arch string
 	// Verdict is whether the nodes' llama.cpp can load it, from the
 	// architecture the registry reports; a set that cannot be served is
 	// shown with the reason and no fetch button.
@@ -71,6 +76,13 @@ type modelUse struct {
 	Replicas int
 	// Min and Max are its scaling as the catalog holds it.
 	Min, Max int
+	// Class is what it set, empty for its default; EffClass what that comes
+	// to. Owner is org:ID or empty for shared. MemoryGB and CPUCores are what
+	// a replica of its first profile reserves.
+	Class, EffClass, Owner string
+	MemoryGB, CPUCores     int
+	// Adapters are the LoRA adapters it serves.
+	Adapters []catalog.Adapter
 	// Ready, Wanted and Nodes are how it runs now, for the row itself:
 	// replicas ready of those it should have, and the nodes they are on.
 	// Fleet is false where nothing could be asked, so the row says nothing
@@ -100,6 +112,9 @@ type mirrorRow struct {
 	// Detail is the card opened beneath the row: what runs it, on which
 	// nodes, and how many replicas it runs.
 	Detail *heldDetail
+	// Kind is what a single GGUF file is — "adapter" or "model" — and Arch
+	// its architecture; empty when not read.
+	Kind, Arch string
 	// Open is the row's ?open= value.
 	Open string
 
@@ -293,7 +308,8 @@ func (p *AccountPages) RegisterAdminMirror(mux *http.ServeMux, d MirrorDeps) {
 				return name + " now runs all the time", nil
 			}))
 
-		mux.HandleFunc("POST /admin/mirror/replicas", p.mirrorAction(m, p.setReplicas))
+		mux.HandleFunc("POST /admin/mirror/replicas", p.mirrorAction(m, p.setModel))
+		mux.HandleFunc("POST /admin/mirror/adapters", p.mirrorAction(m, p.setAdapters))
 
 		mux.HandleFunc("POST /admin/mirror/serve", p.mirrorAction(m,
 			func(ctx context.Context, admin store.User, r *http.Request) (string, error) {
@@ -463,8 +479,13 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
 			u := modelUse{
 				Name: m.Name, Mode: modeOf(m.Scaling), Replicas: max(m.Scaling.Max, 1),
 				Min: m.Scaling.Min, Max: m.Scaling.Max,
+				Class: m.Class, EffClass: m.EffectiveClass(), Owner: m.Owner,
+				Adapters: m.Adapters,
 				Declared: p.catalog.Declared(m.Name),
 			}
+			if len(m.Profiles) > 0 {
+				u.MemoryGB, u.CPUCores = m.Profiles[0].MinMemoryGB, m.Profiles[0].CPUCores
+			}
 			if v, ok := views[m.Name]; ok {
 				u.Fleet, u.Ready, u.Wanted = true, v.Ready, v.Desired
 				u.Parked, u.Unplaceable = v.Parked, v.Unplaceable
@@ -478,13 +499,16 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
 		}
 	}
 
+	p.heldKinds(r.Context(), d.Rows)
+
 	// The one row whose card is open. From the form too, so a change made in
 	// the card comes back with the card still open to show it.
 	open := r.FormValue("open")
 	for i := range d.Rows {
 		d.Rows[i].Open = openKey(d.Rows[i].Model, d.Rows[i].Version)
 		if open != "" && d.Rows[i].Open == open && d.Rows[i].State == mirror.Present {
-			d.Rows[i].Detail = p.heldDetailFor(d.Rows[i])
+			d.Rows[i].Detail = p.heldDetailFor(r.Context(), d.Rows[i])
+			d.Rows[i].Detail.addAdapterChoices(d.Rows[i], d.Rows)
 		}
 	}
 
@@ -746,6 +770,46 @@ func (p *AccountPages) openRepo(r *http.Request, d *adminMirrorData, repo string
 		}
 		d.RepoSets = append(d.RepoSets, rs)
 	}
+	p.probeKinds(r.Context(), d.RepoSets)
+}
+
+// kindOf reads what a GGUF file is; a variable so tests need no registry.
+var kindOf = func(ctx context.Context, url string) (gguf.Header, error) {
+	return gguf.KindFromURL(ctx, nil, url)
+}
+
+// probeKinds reads the start of each single-file GGUF set, to tell LoRA
+// adapters from models: a repository tagged "lora" holds a model with the
+// adapter merged in as often as an adapter, and only the file says which.
+// A few at once and a few in all, since each is a request to the registry.
+func (p *AccountPages) probeKinds(ctx context.Context, sets []repoSet) {
+	const most, together = 12, 6
+	sem := make(chan struct{}, together)
+	var wg sync.WaitGroup
+	n := 0
+	for i := range sets {
+		s := &sets[i]
+		if len(s.Sources) != 1 || !gguf.IsGGUF(s.Sources[0].Path) || n == most {
+			continue
+		}
+		n++
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			sem <- struct{}{}
+			defer func() { <-sem }()
+			cctx, cancel := context.WithTimeout(ctx, 10*time.Second)
+			defer cancel()
+			if h, err := kindOf(cctx, s.Sources[0].URL); err == nil {
+				s.Arch = h.Architecture
+				s.Kind = "model"
+				if h.IsAdapter() {
+					s.Kind = "adapter"
+				}
+			}
+		}()
+	}
+	wg.Wait()
 }
 
 // defaultModelName is the catalog name a repository suggests.

internal/web/admin_quotas.go

@@ -9,6 +9,7 @@ import (
 	"strings"
 	"time"
 
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
 )
 
@@ -129,12 +130,17 @@ type adminQuotasData struct {
 	// Editing is the "scope:id" whose form is open, so the template can keep
 	// the link on every other row.
 	Editing string
-	Since   string
-	Message string
-	Error   string
-	CSRF    string
-	Version string
-	Now     string
+	// The capacity section: quotas on GPUs, cores and memory.
+	HasCapacity bool
+	Capacity    []capacityRow
+	Scopes      []adminapi.ScopeName
+	CapForm     capacityForm
+	Since       string
+	Message     string
+	Error       string
+	CSRF        string
+	Version     string
+	Now         string
 }
 
 // RegisterAdminQuotas mounts quota administration under /admin/quotas.
@@ -176,6 +182,24 @@ func (p *AccountPages) RegisterAdminQuotas(mux *http.ServeMux, s QuotaAdmin, pri
 			}
 			p.renderQuotas(w, r, s, msg, "")
 		}))
+
+	mux.HandleFunc("POST /admin/quotas/capacity", p.adminOnly(
+		func(w http.ResponseWriter, r *http.Request, admin store.User) {
+			sess, _, _ := p.session(r)
+			if p.capacity == nil || !p.accounts.CheckCSRF(sess, r.FormValue("csrf")) {
+				http.Error(w, "bad request", http.StatusBadRequest)
+				return
+			}
+			msg, err := p.applyCapacity(r, admin)
+			if err != nil {
+				p.log.Warn("a capacity quota change was refused",
+					"admin", admin.Email, "scope", r.FormValue("scope"), "err", err)
+				p.renderQuotas(w, r, s, "", err.Error())
+				return
+			}
+			p.log.Info("capacity quota changed", "admin", admin.Email, "what", msg)
+			p.renderQuotas(w, r, s, msg, "")
+		}))
 }
 
 // applyQuota writes one scope's limits, and says what it did.
@@ -259,6 +283,7 @@ func (p *AccountPages) renderQuotas(w http.ResponseWriter, r *http.Request,
 
 	d.BudgetsBind = p.priced != nil && p.priced.AnythingPriced()
 	d.Editing = r.URL.Query().Get("edit")
+	p.capacitySection(r.Context(), &d, r.FormValue("cap"))
 
 	limits, err := s.AllLimits(r.Context())
 	if err != nil {

internal/web/admin_serve.go

@@ -22,6 +22,12 @@ type CatalogAdmin interface {
 	// Scale changes how a model added at runtime runs: always, on demand, or
 	// not at all. One declared in configuration is refused.
 	Scale(ctx context.Context, name string, sc catalog.Scaling, by string) (catalog.Model, error)
+	// SetClass, SetOwner and Resources change what a model's capacity is
+	// worth, whose quota it counts against, and what a replica reserves.
+	SetClass(ctx context.Context, name, class, by string) (catalog.Model, error)
+	SetOwner(ctx context.Context, name, owner, by string) (catalog.Model, error)
+	Resources(ctx context.Context, name, profile string, memoryGB, cpuCores int, by string) (catalog.Model, error)
+	SetAdapters(ctx context.Context, name string, adapters []catalog.Adapter, by string) (catalog.Model, error)
 }
 
 // serveForm is the offer to put one mirrored version into service.

internal/web/admin_serve_test.go

@@ -3,6 +3,7 @@ package web
 import (
 	"context"
 	"errors"
+	"fmt"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
@@ -15,12 +16,16 @@ import (
 )
 
 type fakeCatalog struct {
-	added     []catalog.Model
-	declared  string
-	sources   map[string][]catalog.Model
-	scaled    []string
-	lastScale catalog.Scaling
-	err       error
+	added       []catalog.Model
+	declared    string
+	sources     map[string][]catalog.Model
+	scaled      []string
+	lastScale   catalog.Scaling
+	err         error
+	classes     []string
+	owners      []string
+	resources   []string
+	adapterSets [][]catalog.Adapter
 }
 
 func (f *fakeCatalog) Add(_ context.Context, m catalog.Model, _ string) error {
@@ -35,6 +40,26 @@ func (f *fakeCatalog) Sources(context.Context) (map[string][]catalog.Model, erro
 	return f.sources, nil
 }
 
+func (f *fakeCatalog) SetAdapters(_ context.Context, name string, as []catalog.Adapter, _ string) (catalog.Model, error) {
+	f.adapterSets = append(f.adapterSets, as)
+	return catalog.Model{Name: name, Adapters: as}, f.err
+}
+
+func (f *fakeCatalog) SetClass(_ context.Context, name, class, _ string) (catalog.Model, error) {
+	f.classes = append(f.classes, name+"="+class)
+	return catalog.Model{Name: name, Class: class}, f.err
+}
+
+func (f *fakeCatalog) SetOwner(_ context.Context, name, owner, _ string) (catalog.Model, error) {
+	f.owners = append(f.owners, name+"="+owner)
+	return catalog.Model{Name: name, Owner: owner}, f.err
+}
+
+func (f *fakeCatalog) Resources(_ context.Context, name, _ string, mem, cores int, _ string) (catalog.Model, error) {
+	f.resources = append(f.resources, fmt.Sprintf("%s=%d/%d", name, mem, cores))
+	return catalog.Model{Name: name}, f.err
+}
+
 func (f *fakeCatalog) Scale(_ context.Context, name string, sc catalog.Scaling, _ string) (catalog.Model, error) {
 	f.scaled = append(f.scaled, name)
 	f.lastScale = sc

internal/web/mirror_detail.go

@@ -4,13 +4,19 @@ import (
 	"context"
 	"errors"
 	"fmt"
+	"html/template"
 	"net/http"
+	"net/url"
+	"slices"
 	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
 )
 
@@ -29,14 +35,23 @@ const maxReplicas = 64
 // and room to change it.
 type heldDetail struct {
 	Models []heldModel
+	// Scopes are the parts of the tenancy tree a model can belong to; empty
+	// when capacity quotas are not set up, and the owner choice is left off.
+	Scopes []adminapi.ScopeName
 }
 
 // heldModel is one catalog model running a held version.
 type heldModel struct {
 	modelUse
-	View     adminapi.ModelView
-	Replicas []heldReplica
-	Headroom *adminapi.HeadroomView
+	// Adapters are the model's own; AdapterChoices the held adapter files
+	// of its architecture not attached yet; FindAdapters the explorer's
+	// list of adapters made for it.
+	Adapters       []catalog.Adapter
+	AdapterChoices []adapterChoice
+	FindAdapters   template.URL
+	View           adminapi.ModelView
+	Replicas       []heldReplica
+	Headroom       *adminapi.HeadroomView
 	// MostNow is how many replicas could always run, Current plus the room
 	// found; zero when the room was not measured or had no end in sight.
 	MostNow int
@@ -64,11 +79,14 @@ func openKey(model, version string) string { return model + ":" + version }
 
 // heldDetailFor reads, for each model using a row, how it runs and how much
 // more room it has.
-func (p *AccountPages) heldDetailFor(row mirrorRow) *heldDetail {
+func (p *AccountPages) heldDetailFor(ctx context.Context, row mirrorRow) *heldDetail {
 	d := &heldDetail{}
+	if p.capacity != nil {
+		d.Scopes, _ = p.capacity.Scopes(ctx)
+	}
 	if p.fleet == nil {
 		for _, u := range row.UsedBy {
-			d.Models = append(d.Models, heldModel{modelUse: u})
+			d.Models = append(d.Models, heldModel{modelUse: u, Adapters: u.Adapters})
 		}
 		return d
 	}
@@ -82,7 +100,7 @@ func (p *AccountPages) heldDetailFor(row mirrorRow) *heldDetail {
 	}
 	hs, _ := p.fleet.(HeadroomSource)
 	for _, u := range row.UsedBy {
-		hm := heldModel{modelUse: u, View: views[u.Name]}
+		hm := heldModel{modelUse: u, View: views[u.Name], Adapters: u.Adapters}
 		for _, rep := range p.fleet.Replicas(u.Name) {
 			// Built field by field: a ReplicaView carries the engine's key,
 			// which is for the admin listener and never for a page.
@@ -98,7 +116,7 @@ func (p *AccountPages) heldDetailFor(row mirrorRow) *heldDetail {
 			if h, ok := hs.Headroom(u.Name, 16); ok {
 				hm.Headroom = &h
 				if !h.Limited {
-					hm.MostNow = h.Current + h.More
+					hm.MostNow = h.Current + h.More + h.ByPreempting
 				}
 			}
 		}
@@ -144,7 +162,7 @@ func (p *AccountPages) setReplicas(ctx context.Context, admin store.User, r *htt
 		if !ok {
 			return "", fmt.Errorf("no model %q", name)
 		}
-		fits := h.Current + h.More
+		fits := h.Current + h.More + h.ByPreempting
 		if lo > fits && !h.Limited {
 			why := "nothing more fits"
 			if len(h.Blocked) > 0 {
@@ -156,6 +174,11 @@ func (p *AccountPages) setReplicas(ctx context.Context, admin store.User, r *htt
 		if hi > fits && !h.Limited {
 			note = fmt.Sprintf("; there is room for %d now, so it grows past that only if room is made", fits)
 		}
+		// Room its class would make by stopping others: said, since it is
+		// somebody else's replica that goes.
+		if lo > h.Current+h.More && len(h.Preempts) > 0 {
+			note += "; to make room, replicas of " + strings.Join(h.Preempts, ", ") + " will be stopped"
+		}
 	}
 
 	sc := catalog.Scaling{Min: lo, Max: hi, IdleTimeout: current.Scaling.IdleTimeout}
@@ -171,7 +194,7 @@ func (p *AccountPages) setReplicas(ctx context.Context, admin store.User, r *htt
 	case lo == 0:
 		return fmt.Sprintf("%s loads when asked for, up to %d under load%s", name, hi, note), nil
 	case lo == hi:
-		return fmt.Sprintf("%s runs %s", name, plural(lo, "replica", "replicas")), nil
+		return fmt.Sprintf("%s runs %s%s", name, plural(lo, "replica", "replicas"), note), nil
 	}
 	return fmt.Sprintf("%s runs %d, and up to %d under load%s", name, lo, hi, note), nil
 }
@@ -205,3 +228,206 @@ func age(sec int64) string {
 	}
 	return fmt.Sprintf("%dd", int(d.Hours()/24))
 }
+
+// setModel applies the card's settings form: what a replica reserves, the
+// model's class and owner, then how many replicas it runs — in that order, so
+// the room checked for the replicas is the room its new class and reservation
+// would get. Only what changed is applied, and each change is said.
+func (p *AccountPages) setModel(ctx context.Context, admin store.User, r *http.Request) (string, error) {
+	name := r.FormValue("model")
+	current, err := p.currentModel(ctx, name)
+	if err != nil {
+		return "", err
+	}
+	if p.catalog.Declared(name) {
+		return "", fmt.Errorf("%s is set in server.yaml; change it there", name)
+	}
+	var done []string
+	if r.Form.Has("memory") && len(current.Profiles) > 0 {
+		mem, err1 := strconv.Atoi(strings.TrimSpace(r.FormValue("memory")))
+		cores, err2 := strconv.Atoi(strings.TrimSpace(r.FormValue("cores")))
+		if err1 != nil || err2 != nil || mem < 0 || cores < 0 {
+			return "", errors.New("memory and cores are whole numbers")
+		}
+		if cur := current.Profiles[0]; mem != cur.MinMemoryGB || cores != cur.CPUCores {
+			if _, err := p.catalog.Resources(ctx, name, "", mem, cores, admin.Email); err != nil {
+				return "", err
+			}
+			done = append(done, fmt.Sprintf("a replica reserves %d GB and %d cores (running ones keep theirs until restarted)", mem, cores))
+		}
+	}
+	if r.Form.Has("class") {
+		if c := r.FormValue("class"); c != current.Class {
+			m, err := p.catalog.SetClass(ctx, name, c, admin.Email)
+			if err != nil {
+				return "", err
+			}
+			done = append(done, "is "+m.EffectiveClass())
+		}
+	}
+	if r.Form.Has("owner") && p.capacity != nil {
+		if o := r.FormValue("owner"); o != current.Owner {
+			if _, err := p.catalog.SetOwner(ctx, name, o, admin.Email); err != nil {
+				return "", err
+			}
+			if o == "" {
+				done = append(done, "is shared")
+			} else {
+				done = append(done, "counts against "+o)
+			}
+		}
+	}
+	lo, _ := strconv.Atoi(strings.TrimSpace(r.FormValue("min")))
+	hi, _ := strconv.Atoi(strings.TrimSpace(r.FormValue("max")))
+	if r.Form.Has("min") && (lo != current.Scaling.Min || hi != current.Scaling.Max) {
+		msg, err := p.setReplicas(ctx, admin, r)
+		if err != nil {
+			if len(done) > 0 {
+				return "", fmt.Errorf("%s %s, but not the replicas: %w", name, strings.Join(done, ", "), err)
+			}
+			return "", err
+		}
+		done = append(done, strings.TrimPrefix(msg, name+" "))
+	}
+	if len(done) == 0 {
+		return name + ": nothing changed", nil
+	}
+	return name + " " + strings.Join(done, "; "), nil
+}
+
+// heldKinds reads what each held single-file GGUF is — an adapter or a model —
+// once per file, remembered after.
+func (p *AccountPages) heldKinds(ctx context.Context, rows []mirrorRow) {
+	var wg sync.WaitGroup
+	sem := make(chan struct{}, 6)
+	for i := range rows {
+		row := &rows[i]
+		if row.State != mirror.Present || row.Files != 1 || !gguf.IsGGUF(row.File) {
+			continue
+		}
+		if h, ok := p.kinds.Load(row.URL); ok {
+			row.Kind, row.Arch = kindName(h.(gguf.Header)), h.(gguf.Header).Architecture
+			continue
+		}
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			sem <- struct{}{}
+			defer func() { <-sem }()
+			cctx, cancel := context.WithTimeout(ctx, 5*time.Second)
+			defer cancel()
+			if h, err := kindOf(cctx, row.URL); err == nil {
+				p.kinds.Store(row.URL, h)
+				row.Kind, row.Arch = kindName(h), h.Architecture
+			}
+		}()
+	}
+	wg.Wait()
+}
+
+func kindName(h gguf.Header) string {
+	if h.IsAdapter() {
+		return "adapter"
+	}
+	return "model"
+}
+
+// adapterChoice is a held adapter file that could be attached to a model.
+type adapterChoice struct {
+	URL, Label, Suggest string
+}
+
+// addAdapterChoices offers, on each model of a held model's card, the held
+// adapter files of the same architecture, and a way to find more.
+func (d *heldDetail) addAdapterChoices(row mirrorRow, rows []mirrorRow) {
+	for i := range d.Models {
+		hm := &d.Models[i]
+		hm.FindAdapters = findAdaptersHref(row.Source)
+		if row.Kind == "adapter" {
+			continue
+		}
+		for _, r := range rows {
+			if r.Kind != "adapter" || (row.Arch != "" && r.Arch != row.Arch) {
+				continue
+			}
+			attached := false
+			for _, a := range hm.Adapters {
+				attached = attached || a.Source == r.URL
+			}
+			if !attached {
+				hm.AdapterChoices = append(hm.AdapterChoices, adapterChoice{URL: r.URL,
+					Label: r.Model + " " + r.Version + " · " + r.File, Suggest: hm.Name + "-" + r.Model})
+			}
+		}
+	}
+}
+
+// findAdaptersHref is the explorer's list of adapters for a held model: those
+// made for the model its repository was made from, when it came from the
+// registry, and every adapter otherwise.
+func findAdaptersHref(source string) template.URL {
+	const base = "/admin/mirror?tab=explore&view=adapters"
+	u, err := url.Parse(source)
+	if err != nil || u.Host != "huggingface.co" {
+		return base
+	}
+	parts := strings.Split(strings.Trim(u.Path, "/"), "/")
+	if len(parts) < 2 {
+		return base
+	}
+	return template.URL(base + "&from=" + url.QueryEscape(parts[0]+"/"+parts[1]))
+}
+
+// setAdapters attaches or detaches one adapter on a model, from its card.
+// An adapter is checked before it is attached: a file that is not an adapter,
+// or one made for another architecture, is a replica that fails to start.
+func (p *AccountPages) setAdapters(ctx context.Context, admin store.User, r *http.Request) (string, error) {
+	name := r.FormValue("model")
+	current, err := p.currentModel(ctx, name)
+	if err != nil {
+		return "", err
+	}
+	adapters := slices.Clone(current.Adapters)
+	switch r.FormValue("op") {
+	case "remove":
+		gone := r.FormValue("name")
+		adapters = slices.DeleteFunc(adapters, func(a catalog.Adapter) bool { return a.Name == gone })
+		if len(adapters) == len(current.Adapters) {
+			return "", fmt.Errorf("%s has no adapter %q", name, gone)
+		}
+		if _, err := p.catalog.SetAdapters(ctx, name, adapters, admin.Email); err != nil {
+			return "", err
+		}
+		return fmt.Sprintf("%s: adapter %s removed; replicas drop it when restarted (llmgw ctl restart --model %s)",
+			name, gone, name), nil
+	case "add":
+		an := strings.TrimSpace(r.FormValue("name"))
+		src := strings.TrimSpace(r.FormValue("source"))
+		scale := 1.0
+		if v := strings.TrimSpace(r.FormValue("scale")); v != "" {
+			if scale, err = strconv.ParseFloat(v, 64); err != nil || scale <= 0 {
+				return "", errors.New("the scale is a positive number")
+			}
+		}
+		if an == "" || src == "" {
+			return "", errors.New("an adapter needs a name and a file")
+		}
+		h, err := kindOf(ctx, src)
+		if err != nil {
+			return "", fmt.Errorf("the adapter file could not be read: %w", err)
+		}
+		if !h.IsAdapter() {
+			return "", errors.New("that file is a model, not a LoRA adapter")
+		}
+		if base, err := kindOf(ctx, current.Source); err == nil && base.Architecture != h.Architecture {
+			return "", fmt.Errorf("that adapter is for %s models and %s is %s", h.Architecture, name, base.Architecture)
+		}
+		adapters = append(adapters, catalog.Adapter{Name: an, Source: src, Scale: scale})
+		if _, err := p.catalog.SetAdapters(ctx, name, adapters, admin.Email); err != nil {
+			return "", err
+		}
+		return fmt.Sprintf("%s: adapter %s added; ask for it by that name once the replicas are restarted "+
+			"(llmgw ctl restart --model %s)", name, an, name), nil
+	}
+	return "", errors.New("add or remove an adapter")
+}

internal/web/mirror_detail_test.go

@@ -1,21 +1,28 @@
 package web
 
 import (
+	"context"
+	"log/slog"
+	"net/http"
 	"net/url"
 	"strings"
 	"testing"
 
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
 )
 
 // roomyFleet runs one replica of "coder" on n2 and has room for `more`.
 type roomyFleet struct {
 	fakeFleet
-	more    int
-	blocked []string
-	asked   int
+	more     int
+	blocked  []string
+	asked    int
+	preempt  int
+	preempts []string
 }
 
 func (f *roomyFleet) Nodes() []adminapi.NodeView {
@@ -41,6 +48,7 @@ func (f *roomyFleet) Headroom(model string, limit int) (adminapi.HeadroomView, b
 	} else {
 		h.Limited = true
 	}
+	h.ByPreempting, h.Preempts = f.preempt, f.preempts
 	return h, true
 }
 
@@ -181,3 +189,147 @@ func TestNodeNamesLoseOnlyWhatEveryNodeShares(t *testing.T) {
 		}
 	}
 }
+
+func settingsPage(t *testing.T, cat *fakeCatalog, withQuotas bool) *http.ServeMux {
+	t.Helper()
+	a := fakeAccounts{user: store.User{ID: "u1", Email: "a@example.com", IsAdmin: true}}
+	p := NewAccountPages(a, nil, nil, "v0", slog.New(slog.DiscardHandler))
+	if withQuotas {
+		p.SetCapacity(&fakeCapacity{})
+	}
+	mux := http.NewServeMux()
+	p.RegisterAdminMirror(mux, MirrorDeps{Mirror: &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 1},
+	}}, Catalog: cat, Fleet: &roomyFleet{more: 3}})
+	return mux
+}
+
+func TestACardSetsClassOwnerAndReservationAndOnlyWhatChanged(t *testing.T) {
+	cat := coderCatalog()
+	cat.sources["http://mirror:8081/coder/v1/c.gguf"][0].Profiles = []catalog.Profile{{Name: "cpu", MinMemoryGB: 12, CPUCores: 4}}
+	mux := settingsPage(t, cat, true)
+
+	card := mirrorGet(t, mux, "/admin/mirror?open=coder:v1").Body.String()
+	for _, want := range []string{`name="class"`, `<option value="" selected>default (standard)</option>`,
+		`name="owner"`, `<option value="team:t1">Acme / ml</option>`, `name="memory" value="12"`, `name="cores" value="4"`} {
+		if !strings.Contains(card, want) {
+			t.Errorf("the card lacks %q", want)
+		}
+	}
+	post := func(v url.Values) string {
+		v.Set("csrf", "tok")
+		v.Set("model", "coder")
+		v.Set("open", "coder:v1")
+		return mirrorPost(t, mux, "/admin/mirror/replicas", v).Body.String()
+	}
+	same := url.Values{"min": {"1"}, "max": {"1"}, "class": {""}, "owner": {""}, "memory": {"12"}, "cores": {"4"}}
+	if body := post(same); !strings.Contains(body, "coder: nothing changed") ||
+		len(cat.classes)+len(cat.owners)+len(cat.resources)+len(cat.scaled) != 0 {
+		t.Errorf("an unchanged form changed something: %v %v %v %v", cat.classes, cat.owners, cat.resources, cat.scaled)
+	}
+	changed := url.Values{"min": {"1"}, "max": {"1"}, "class": {"critical"}, "owner": {"team:t1"}, "memory": {"14"}, "cores": {"4"}}
+	body := post(changed)
+	if strings.Join(cat.resources, ",") != "coder=14/4" || strings.Join(cat.classes, ",") != "coder=critical" ||
+		strings.Join(cat.owners, ",") != "coder=team:t1" || len(cat.scaled) != 0 {
+		t.Errorf("applied %v %v %v %v", cat.resources, cat.classes, cat.owners, cat.scaled)
+	}
+	if !strings.Contains(body, "a replica reserves 14 GB and 4 cores") || !strings.Contains(body, "counts against team:t1") {
+		t.Errorf("the change was not said: %s", body)
+	}
+
+	// Without capacity quotas there is no owner to choose, and none is sent.
+	plain := mirrorGet(t, settingsPage(t, coderCatalog(), false), "/admin/mirror?open=coder:v1").Body.String()
+	if strings.Contains(plain, `name="owner"`) {
+		t.Error("an owner choice with no quotas to count against")
+	}
+}
+
+func TestRoomAHigherClassWouldMakeIsOfferedAndSaid(t *testing.T) {
+	fleet := &roomyFleet{more: 0, preempt: 1, preempts: []string{"chat"}, blocked: []string{"n1: full"}}
+	cat := coderCatalog()
+	mux := mirrorPageDeps(t, MirrorDeps{Mirror: &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 1},
+	}}, Catalog: cat, Fleet: fleet}, true)
+
+	card := mirrorGet(t, mux, "/admin/mirror?open=coder:v1").Body.String()
+	if !strings.Contains(card, "No free room for another replica now.") ||
+		!strings.Contains(card, "1 by stopping replicas its class outranks: chat.") ||
+		!strings.Contains(card, `name="min" value="1" min="0" max="2"`) {
+		t.Errorf("the card should offer the room preemption makes, and say whose")
+	}
+	body := mirrorPost(t, mux, "/admin/mirror/replicas", url.Values{"csrf": {"tok"},
+		"model": {"coder"}, "min": {"2"}, "max": {"2"}, "open": {"coder:v1"}}).Body.String()
+	if len(cat.scaled) != 1 || cat.lastScale.Min != 2 {
+		t.Fatalf("two were refused though preemption makes room: %s", body)
+	}
+	if !strings.Contains(body, "replicas of chat will be stopped") {
+		t.Errorf("stopping somebody else's replica should be said: %s", body)
+	}
+	body = mirrorPost(t, mux, "/admin/mirror/replicas", url.Values{"csrf": {"tok"},
+		"model": {"coder"}, "min": {"3"}, "max": {"3"}, "open": {"coder:v1"}}).Body.String()
+	if len(cat.scaled) != 1 || !strings.Contains(body, "at most 2") {
+		t.Errorf("three is past even what preemption makes: %s", body)
+	}
+}
+
+func TestAnAdapterHeldInTheMirrorIsAttachedFromItsModelsCard(t *testing.T) {
+	prev := kindOf
+	t.Cleanup(func() { kindOf = prev })
+	kindOf = func(_ context.Context, u string) (gguf.Header, error) {
+		switch {
+		case strings.Contains(u, "/sql/"):
+			return gguf.Header{Architecture: "qwen2", Type: "adapter"}, nil
+		case strings.Contains(u, "/llama-lora/"):
+			return gguf.Header{Architecture: "llama", Type: "adapter"}, nil
+		}
+		return gguf.Header{Architecture: "qwen2"}, nil
+	}
+	cat := coderCatalog()
+	coder := &cat.sources["http://mirror:8081/coder/v1/c.gguf"][0]
+	coder.Source = "http://mirror:8081/coder/v1/c.gguf"
+	coder.Profiles = []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 12}}
+	m := &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 1,
+			Source: "https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF/resolve/abc/c.gguf"},
+		{Model: "sql", Version: "v1", File: "sql.gguf", State: mirror.Present, BytesDone: 1},
+		{Model: "llama-lora", Version: "v1", File: "l.gguf", State: mirror.Present, BytesDone: 1},
+	}}
+	mux := mirrorPageDeps(t, MirrorDeps{Mirror: m, Catalog: cat, Fleet: &roomyFleet{more: 1}}, true)
+
+	list := mirrorGet(t, mux, "/admin/mirror").Body.String()
+	if strings.Count(list, ">LoRA adapter</span>") != 2 {
+		t.Error("the two adapter files are not marked as adapters in the list")
+	}
+	card := mirrorGet(t, mux, "/admin/mirror?open=coder:v1").Body.String()
+	if !strings.Contains(card, `<option value="http://mirror:8081/sql/v1/sql.gguf" data-suggest="coder-sql">`) {
+		t.Error("the qwen2 adapter is not offered for the qwen2 model")
+	}
+	if strings.Contains(card, "llama-lora") && strings.Contains(card, `value="http://mirror:8081/llama-lora/v1/l.gguf"`) {
+		t.Error("an adapter for another architecture is offered")
+	}
+	if !strings.Contains(card, "view=adapters&amp;from=bartowski%2FQwen2.5-Coder-7B-Instruct-GGUF") {
+		t.Error("finding adapters should start from the model's own repository")
+	}
+
+	post := func(v url.Values) string {
+		v.Set("csrf", "tok")
+		v.Set("model", "coder")
+		v.Set("open", "coder:v1")
+		return mirrorPost(t, mux, "/admin/mirror/adapters", v).Body.String()
+	}
+	body := post(url.Values{"op": {"add"}, "name": {"coder-sql"}, "source": {"http://mirror:8081/sql/v1/sql.gguf"}, "scale": {"0.8"}})
+	if len(cat.adapterSets) != 1 || len(cat.adapterSets[0]) != 1 || cat.adapterSets[0][0].Scale != 0.8 {
+		t.Fatalf("attached %+v: %s", cat.adapterSets, body)
+	}
+	if !strings.Contains(body, "adapter coder-sql added") {
+		t.Errorf("the attach was not said: %s", body)
+	}
+	for name, src := range map[string]string{"another architecture": "http://mirror:8081/llama-lora/v1/l.gguf",
+		"a model": "http://mirror:8081/coder/v1/c.gguf"} {
+		n := len(cat.adapterSets)
+		out := post(url.Values{"op": {"add"}, "name": {"x"}, "source": {src}})
+		if len(cat.adapterSets) != n || !strings.Contains(out, `class="error"`) {
+			t.Errorf("%s was attached", name)
+		}
+	}
+}

internal/web/mirror_explore.go

@@ -18,6 +18,9 @@ const (
 	viewTrending  = "trending"
 	viewNew       = "new"
 	viewDownloads = "downloads"
+	// viewAdapters lists LoRA adapters, all of them or those made for one
+	// model.
+	viewAdapters = "adapters"
 )
 
 // newWithin is how recent a repository has to be to count as new.
@@ -31,6 +34,7 @@ const exploreShown = 25
 
 var exploreViews = []struct{ key, label string }{
 	{viewTrending, "Trending"}, {viewNew, "New this month"}, {viewDownloads, "Most downloaded"},
+	{viewAdapters, "LoRA adapters"},
 }
 
 var exploreTasks = []struct{ key, label, tag string }{
@@ -48,6 +52,9 @@ var exploreSizes = []struct {
 // open: a listing, its filters, and links that keep them.
 type explorer struct {
 	View, Task, Size string
+	// Of is the model whose adapters are listed, in the adapters view; empty
+	// lists adapters for any model.
+	Of string
 	// Here keeps to what the nodes' llama.cpp can load. On by default: a
 	// trending list is mostly things this cluster cannot run, and finding
 	// that out one repository at a time is what the explorer is for avoiding.
@@ -123,11 +130,12 @@ func trimZero(s string) string { return strings.TrimSuffix(s, ".0") }
 // — the page opened from the menu — it is trending chat models this cluster
 // can run.
 func exploreState(q url.Values) explorer {
-	e := explorer{View: q.Get("view"), Task: q.Get("task"), Size: q.Get("size")}
+	e := explorer{View: q.Get("view"), Task: q.Get("task"), Size: q.Get("size"),
+		Of: strings.Trim(strings.TrimSpace(q.Get("of")), "/")}
 	// "x" marks a submitted form, so an unticked box reads as off rather than
 	// as not yet chosen.
 	e.Here = q.Get("x") != "1" || q.Get("here") == "1"
-	if !oneOf(e.View, viewTrending, viewNew, viewDownloads) {
+	if !oneOf(e.View, viewTrending, viewNew, viewDownloads, viewAdapters) {
 		e.View = viewTrending
 		// A search with no view asks for what a name usually means, which is
 		// the one everybody downloads.
@@ -165,6 +173,10 @@ func (e explorer) href(query, key, value string) template.URL {
 	if key != "" {
 		v.Set(key, value)
 	}
+	// Whose adapters, kept while it is adapters being looked at.
+	if e.Of != "" && v.Get("view") == viewAdapters {
+		v.Set("of", e.Of)
+	}
 	return template.URL("/admin/mirror?" + v.Encode())
 }
 
@@ -181,6 +193,16 @@ func (e explorer) hfQuery(search string) hf.Query {
 		// Trending, then kept to the recent: sorted by creation date alone,
 		// the newest repositories are mostly empty uploads nobody has used.
 		q.Limit = 100
+	case viewAdapters:
+		// Adapters rarely carry a task or a parameter count, so neither is
+		// asked for; what makes one useful is the model it was made for.
+		q.Sort = hf.SortDownloads
+		if e.Of != "" {
+			q.Filters = []string{"base_model:adapter:" + e.Of}
+		} else {
+			q.Filters = []string{"lora"}
+		}
+		return q
 	}
 	for _, t := range exploreTasks {
 		if t.key == e.Task {
@@ -198,6 +220,15 @@ func (e explorer) hfQuery(search string) hf.Query {
 // exploreList fills in the listing.
 func (p *AccountPages) exploreList(r *http.Request, d *adminMirrorData) {
 	e := exploreState(r.URL.Query())
+	// From a held model's card: its repository, whose adapters are those of
+	// the model it was made from — a GGUF repository is a quantisation, and
+	// adapters are made for the original.
+	if from := strings.TrimSpace(r.URL.Query().Get("from")); e.View == viewAdapters && e.Of == "" && from != "" {
+		e.Of = from
+		if repo, err := p.registry.Repo(r.Context(), from); err == nil && len(repo.BaseModel) > 0 {
+			e.Of = repo.BaseModel[0]
+		}
+	}
 	for _, v := range exploreViews {
 		e.Views = append(e.Views, exploreLink{v.label, e.href(d.Query, "view", v.key), v.key == e.View})
 	}
@@ -225,6 +256,14 @@ func (p *AccountPages) exploreList(r *http.Request, d *adminMirrorData) {
 		if a := m.Architecture(); a != "" && p.llamaVersion != "" {
 			row.Verdict = gguf.LlamaCpp(p.llamaVersion, a)
 		}
+		// Adapters are small and say little about their architecture in a
+		// listing; whether one fits is read from its file when it is opened.
+		if e.View == viewAdapters {
+			if len(e.Rows) < exploreShown {
+				e.Rows = append(e.Rows, row)
+			}
+			continue
+		}
 		// An adapter or a fragment, not a model: tens of megabytes of LoRA
 		// weights listed among the models they need a model to run on.
 		if e.Here && m.Params() > 0 && m.Params() < minParams {

internal/web/mirror_explore_test.go

@@ -1,15 +1,18 @@
 package web
 
 import (
+	"context"
 	"errors"
 	"html"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
+	"os"
 	"strings"
 	"testing"
 	"time"
 
+	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/hf"
 )
 
@@ -195,3 +198,52 @@ func TestAFetchFromTheListAnswersWithItsOutcomeAlone(t *testing.T) {
 		t.Errorf("a refusal: %d %s", w.Code, w.Body.String())
 	}
 }
+
+// noProbe answers every file as an unreadable one, for tests that are not
+// about what the files are.
+func TestMain(m *testing.M) {
+	kindOf = func(context.Context, string) (gguf.Header, error) {
+		return gguf.Header{}, errors.New("not probed in tests")
+	}
+	os.Exit(m.Run())
+}
+
+func TestAdaptersAreListedForAModelAndToldFromModelsByTheirFiles(t *testing.T) {
+	prev := kindOf
+	t.Cleanup(func() { kindOf = prev })
+	kindOf = func(_ context.Context, url string) (gguf.Header, error) {
+		if strings.Contains(url, "adapter") {
+			return gguf.Header{Architecture: "qwen2", Type: "adapter"}, nil
+		}
+		return gguf.Header{Architecture: "qwen2"}, nil
+	}
+	reg := &fakeRegistry{
+		results: []hf.Model{{ID: "ggml-org/LoRA-Qwen2.5-Coder-7B-Instruct-F16-GGUF", Downloads: 900}},
+		repo: hf.Repo{ID: "ggml-org/LoRA-Qwen2.5-Coder-7B-Instruct-F16-GGUF", Revision: "776530df0000",
+			BaseModel: []string{"Qwen/Qwen2.5-Coder-7B-Instruct"},
+			Files:     []hf.File{{Path: "lora-adapter-f16.gguf", Bytes: 726 << 20}, {Path: "merged-Q4_K_M.gguf", Bytes: 4 << 30}}},
+	}
+	mux := mirrorPageWith(t, &fakeMirror{}, reg, true)
+
+	// From a held model's card: its repository is looked up, and the
+	// adapters listed are those of the model it was made from.
+	body := mirrorGet(t, mux, "/admin/mirror?tab=explore&view=adapters&from=bartowski/Qwen2.5-Coder-7B-Instruct-GGUF").Body.String()
+	want := "base_model:adapter:Qwen/Qwen2.5-Coder-7B-Instruct"
+	if len(reg.query.Filters) != 1 || reg.query.Filters[0] != want {
+		t.Errorf("asked the registry for %v, expected %s", reg.query.Filters, want)
+	}
+	if !strings.Contains(body, "LoRA adapters made for <code>Qwen/Qwen2.5-Coder-7B-Instruct</code>") {
+		t.Error("the page does not say whose adapters these are")
+	}
+	if strings.Contains(body, `aria-label="Task"`) {
+		t.Error("task and size filters shown for adapters, which rarely carry either")
+	}
+
+	repo := mirrorGet(t, mux, "/admin/mirror?repo=ggml-org/LoRA-Qwen2.5-Coder-7B-Instruct-F16-GGUF").Body.String()
+	if !strings.Contains(repo, "LoRA adapter · qwen2") {
+		t.Error("the adapter file is not marked as one")
+	}
+	if !strings.Contains(repo, `<span class="opt">model · qwen2</span>`) {
+		t.Error("the merged file is not marked as a model")
+	}
+}

internal/web/static/base.css

@@ -855,3 +855,10 @@ table.explore .from { margin-left: .5rem; font-size: .78rem; color: var(--muted)
 .clip-inline { display: inline-block; max-width: 24rem; overflow: hidden; text-overflow: ellipsis;
                vertical-align: bottom; font-size: .8rem; color: var(--muted); margin-left: .4rem; }
 table.rows td.warn { color: var(--warn); cursor: help; }
+.held-card .replicas-form select, .capacity-form select { display: block; margin-top: .25rem; }
+.held-card .replicas-form .field-row { flex-wrap: wrap; gap: .75rem 1rem; }
+.quota-tree td.q-act form.inline { margin-left: .75rem; }
+.held-card .adapters { margin: .75rem 0; }
+.held-card .adapters h4 { font-size: .9rem; margin: 0 0 .35rem; display: flex; gap: .75rem; align-items: baseline; }
+.held-card .adapter-form .field-row { align-items: flex-end; flex-wrap: wrap; gap: .5rem 1rem; }
+.held-card .adapter-form select { display: block; margin-top: .25rem; max-width: 28rem; }

internal/web/templates/admin_mirror.html

@@ -47,7 +47,8 @@ reachable from this machine.</p>
 {{range .Rows}}{{$row0 := .}}<tr{{if $grouped}} class="model-version"{{end}}{{if .Detail}} class="open"{{end}}>
   <td>{{if eq (printf "%s" .State) "present"}}<a class="open-held" aria-expanded="{{if .Detail}}true{{else}}false{{end}}"
        href="/admin/mirror?tab=held{{if not .Detail}}&amp;open={{.Open}}{{end}}"><strong>{{.Model}}</strong></a>
-    {{else}}<strong>{{.Model}}</strong>{{end}} <span class="opt">{{.Version}}</span></td>
+    {{else}}<strong>{{.Model}}</strong>{{end}} <span class="opt">{{.Version}}</span>
+    {{if eq .Kind "adapter"}}<span class="tag new" title="a LoRA adapter for {{.Arch}} models: attach it on a model's card">LoRA adapter</span>{{end}}</td>
   <td class="file" title="{{.File}}{{if .SHA256}} · sha256 {{.SHA256}}{{end}}">
     {{if gt .Files 1}}{{.Files}} files{{else}}<code>{{.File}}</code>{{end}}</td>
   <td class="num">
@@ -215,16 +216,25 @@ document.addEventListener('click', function (e) {
 {{if and (eq .Tab "explore") .HasRegistry}}
 {{if not .Repo}}{{with .Explore}}
 <nav class="pills" aria-label="Which models">{{range .Views}}<a href="{{.Href}}"{{if .Active}} class="on" aria-current="page"{{end}}>{{.Label}}</a>{{end}}</nav>
+{{if eq .View "adapters"}}
+<p class="note">{{if .Of}}LoRA adapters made for <code>{{.Of}}</code> ·
+  <a href="/admin/mirror?tab=explore&amp;view=adapters">all adapters</a>{{else}}LoRA adapters for any model.
+  From a held model's card, <em>find adapters</em> lists those made for it.{{end}}
+  An adapter runs on the model it was made for, and only as a GGUF adapter file;
+  open one to see which of its files are.</p>
+{{else}}
 <div class="explore-filters">
   <nav class="pills small" aria-label="Task">{{range .Tasks}}<a href="{{.Href}}"{{if .Active}} class="on" aria-current="true"{{end}}>{{.Label}}</a>{{end}}</nav>
   <nav class="pills small" aria-label="Size">{{range .Sizes}}<a href="{{.Href}}"{{if .Active}} class="on" aria-current="true"{{end}}>{{.Label}}</a>{{end}}</nav>
 </div>
 {{end}}
+{{end}}
 <form method="get" action="/admin/mirror" class="explore-search">
   <input type="hidden" name="tab" value="explore">
   {{with .Explore}}<input type="hidden" name="view" value="{{.View}}">
   <input type="hidden" name="task" value="{{.Task}}">
   <input type="hidden" name="size" value="{{.Size}}">
+  {{if .Of}}<input type="hidden" name="of" value="{{.Of}}">{{end}}
   <input type="hidden" name="x" value="1">{{end}}
   <input type="search" name="q" value="{{.Query}}" placeholder="Search Hugging Face — qwen coder, gemma, bge…" aria-label="Search Hugging Face">
   <label class="check"><input type="checkbox" name="here" value="1" onchange="this.form.submit()"{{if .Explore}}{{if .Explore.Here}} checked{{end}}{{end}}> only what these nodes can run</label>
@@ -405,7 +415,9 @@ version and the old weights stay where they are.</p>
 <tr><th>What</th><th class="num">Size</th><th>Published checksum</th><th></th></tr>
 {{range .RepoSets}}<tr>
   <td><code>{{.Name}}</code>
-    {{if .Shards}}<span class="opt">{{.Shards}} shards, fetched together</span>{{end}}</td>
+    {{if .Shards}}<span class="opt">{{.Shards}} shards, fetched together</span>{{end}}
+    {{if eq .Kind "adapter"}}<span class="tag new" title="a LoRA adapter: it runs on a {{.Arch}} model it was made for, not on its own">LoRA adapter · {{.Arch}}</span>
+    {{else if eq .Kind "model"}}<span class="opt">model{{with .Arch}} · {{.}}{{end}}</span>{{end}}</td>
   <td class="num">{{bytes .Bytes}}</td>
   <td>{{with .SHA256}}<code title="{{.}}">{{slice . 0 12}}</code>{{else}}—{{end}}</td>
   <td>
@@ -443,7 +455,7 @@ computes once it has it.</p>
 {{end}}
 
 {{/* The card beneath a held version: what runs it, where, and how many. */}}
-{{define "mirror_held_detail"}}{{$row := .Row}}{{$csrf := .CSRF}}
+{{define "mirror_held_detail"}}{{$row := .Row}}{{$csrf := .CSRF}}{{$scopes := .D.Scopes}}
 <div class="held-card">
 <p class="repo-meta">
   <span>{{if gt $row.Files 1}}{{$row.Files}} files{{else}}<code>{{$row.File}}</code>{{end}}</span>
@@ -482,12 +494,50 @@ computes once it has it.</p>
 <p class="headroom">
 {{if .Limited}}Room for at least {{.More}} more replicas.
 {{else if .More}}Room for {{.More}} more now{{if .Nodes}}, on {{range $i, $n := .Nodes}}{{if $i}}, {{end}}{{join $n}}{{end}}{{end}}.
-{{else}}No room for another replica now.{{end}}
+{{else}}No free room for another replica now.{{end}}
+{{if .ByPreempting}}<br><span class="note">{{if .More}}And {{end}}{{.ByPreempting}} {{if .More}}more {{end}}by stopping replicas its class outranks: {{join .Preempts}}.</span>{{end}}
 </p>
 {{if and (not .Limited) .Blocked}}<details class="blocked"><summary>Why not {{if .More}}more{{else}}another{{end}}</summary>
 <ul>{{range .Blocked}}<li>{{.}}</li>{{end}}</ul></details>{{end}}
 {{end}}
 
+{{if not .Declared}}{{$mname := .Name}}
+<div class="adapters">
+  <h4>LoRA adapters <a class="note" href="{{.FindAdapters}}">find adapters on Hugging Face →</a></h4>
+  {{if .Adapters}}<table class="compact rows">
+  <tr><th>Asked for as</th><th>File</th><th class="num">Scale</th><th></th></tr>
+  {{range .Adapters}}{{$a := .}}<tr>
+    <td><code>{{.Name}}</code></td>
+    <td class="clip" title="{{.Source}}">{{.Source}}</td>
+    <td class="num">{{.EffectiveScale}}</td>
+    <td class="actions"><form method="post" action="/admin/mirror/adapters" class="inline">
+      <input type="hidden" name="csrf" value="{{$csrf}}">
+      <input type="hidden" name="model" value="{{$mname}}">
+      <input type="hidden" name="open" value="{{$row.Open}}">
+      <input type="hidden" name="op" value="remove">
+      <input type="hidden" name="name" value="{{$a.Name}}">
+      <button type="submit" class="link danger">remove</button></form></td>
+  </tr>{{end}}</table>
+  {{else}}<p class="note">None. A request for an adapter's name is served by this model's replicas with it applied.</p>{{end}}
+  {{if .AdapterChoices}}{{$m := .}}
+  <form method="post" action="/admin/mirror/adapters" class="adapter-form">
+    <input type="hidden" name="csrf" value="{{$csrf}}">
+    <input type="hidden" name="model" value="{{.Name}}">
+    <input type="hidden" name="open" value="{{$row.Open}}">
+    <input type="hidden" name="op" value="add">
+    <div class="field-row">
+      <label>Adapter file <select name="source" required
+          onchange="this.form.elements.name.value = this.selectedOptions[0].dataset.suggest">
+        {{range .AdapterChoices}}<option value="{{.URL}}" data-suggest="{{.Suggest}}">{{.Label}}</option>{{end}}
+      </select></label>
+      <label>Asked for as <input type="text" name="name" value="{{(index .AdapterChoices 0).Suggest}}" required></label>
+      <label>Scale <input type="text" name="scale" value="1" size="4"></label>
+      <button type="submit">Attach</button>
+    </div>
+  </form>{{else}}<p class="note">No held adapter file for this model's architecture. Fetch one from Hugging Face first.</p>{{end}}
+</div>
+{{end}}
+
 {{if .Declared}}
 <p class="note">Set in server.yaml, which is read again on every start: change its replicas there.</p>
 {{else}}
@@ -500,11 +550,30 @@ computes once it has it.</p>
       <input type="number" name="min" value="{{.Min}}" min="0"{{if .MostNow}} max="{{.MostNow}}"{{end}} required></label>
     <label>Up to, under load
       <input type="number" name="max" value="{{.Max}}" min="0" max="64" required></label>
-    <button type="submit">Update</button>
+    <label>Class
+      <select name="class">
+        <option value=""{{if eq .Class ""}} selected{{end}}>default ({{if eq .Mode "ondemand"}}opportunistic{{else}}standard{{end}})</option>
+        <option value="critical"{{if eq .Class "critical"}} selected{{end}}>critical</option>
+        <option value="standard"{{if eq .Class "standard"}} selected{{end}}>standard</option>
+        <option value="opportunistic"{{if eq .Class "opportunistic"}} selected{{end}}>opportunistic</option>
+      </select></label>
+    {{if $scopes}}{{$owner := .Owner}}<label>Owner
+      <select name="owner">
+        <option value=""{{if eq $owner ""}} selected{{end}}>shared</option>
+        {{range $scopes}}<option value="{{.Key}}"{{if eq .Key $owner}} selected{{end}}>{{.Name}}</option>{{end}}
+      </select></label>{{end}}
   </div>
-  <p class="note">{{if .MostNow}}At most {{.MostNow}} can always run with what is free now. {{end}}0 always running with a
+  <div class="field-row">
+    <label>Memory GB <span class="opt">per replica</span>
+      <input type="number" name="memory" value="{{.MemoryGB}}" min="0"></label>
+    <label>Cores <span class="opt">per replica, its threads</span>
+      <input type="number" name="cores" value="{{.CPUCores}}" min="0"></label>
+    <button type="submit">Save</button>
+  </div>
+  <p class="note">{{if .MostNow}}At most {{.MostNow}} can always run now{{if .Headroom}}{{if .Headroom.ByPreempting}}, counting what its class may stop to make room{{end}}{{end}}. {{end}}0 always running with a
   higher limit loads it when asked for; 0 and 0 stops it. A replica is added under load when requests wait, and
-  taken away after ten quiet minutes.</p>
+  taken away after ten quiet minutes. A higher class may stop replicas of lower ones to make room; memory and
+  cores apply to replicas started after the change.</p>
 </form>
 {{end}}
 </section>

internal/web/templates/admin_quotas.html

@@ -147,6 +147,60 @@ keys, projects and the chat page. {{.Personal}}</p>
 </table>
 {{else}}<p class="empty">Nobody has been approved yet.</p>{{end}}
 
+{{if .HasCapacity}}
+<h2 id="capacity">Capacity</h2>
+<p class="note">What a part of the tree is guaranteed of the cluster, and the most
+it may hold: GPUs, CPU cores and memory, counted from the replicas of the models
+it owns and of those beneath it. What it holds within its guarantee is never taken
+for someone else's model; beyond it, it borrows free room and gives it back when
+asked. A model's owner is set on its card on the mirror page.</p>
+{{if .Capacity}}
+<table class="rows quota-tree">
+<tr><th>Where</th><th class="num">GPUs</th><th class="num">Cores</th><th class="num">Memory GB</th><th>Set by</th><th></th></tr>
+{{range .Capacity}}<tr>
+  <td class="q-where"><span class="q-name">{{or .Name .Scope}}</span></td>
+  <td class="num">{{or .GPUs "—"}}</td>
+  <td class="num">{{or .Cores "—"}}</td>
+  <td class="num">{{or .Memory "—"}}</td>
+  <td class="note">{{.UpdatedBy}}</td>
+  <td class="q-act"><a href="/admin/quotas?cap={{.Scope}}#capacity">change</a>
+    <form method="post" action="/admin/quotas/capacity" class="inline">
+      <input type="hidden" name="csrf" value="{{$.CSRF}}">
+      <input type="hidden" name="scope" value="{{.Scope}}">
+      <button type="submit" name="clear" value="1" class="link danger">remove</button>
+    </form></td>
+</tr>{{end}}
+</table>
+<p class="q-legend">Each figure is held of guaranteed (at most the limit); — is not governed.</p>
+{{else}}<p class="empty">No capacity quotas: every model shares the cluster by its class alone.</p>{{end}}
+
+{{if .Scopes}}
+<form method="post" action="/admin/quotas/capacity" class="q-form capacity-form">
+  <input type="hidden" name="csrf" value="{{.CSRF}}">
+  <p class="q-form-title"><strong>{{if .CapForm.Scope}}Change{{else}}Set{{end}} a capacity quota.</strong>
+    A blank field is not governed; 0 is none at all.</p>
+  <label>Where
+    <select name="scope" required>
+      <option value="">choose…</option>
+      {{range .Scopes}}<option value="{{.Key}}"{{if eq .Key $.CapForm.Scope}} selected{{end}}>{{.Name}}</option>{{end}}
+    </select></label>
+  <div class="field-row">
+    <label>Guaranteed GPUs <input type="text" name="gpus" value="{{.CapForm.GPUs}}" placeholder="—"></label>
+    <label>Cores <input type="text" name="cores" value="{{.CapForm.Cores}}" placeholder="—"></label>
+    <label>Memory GB <input type="text" name="memory" value="{{.CapForm.Memory}}" placeholder="—"></label>
+  </div>
+  <div class="field-row">
+    <label>At most GPUs <input type="text" name="limit_gpus" value="{{.CapForm.LGPUs}}" placeholder="—"></label>
+    <label>Cores <input type="text" name="limit_cores" value="{{.CapForm.LCores}}" placeholder="—"></label>
+    <label>Memory GB <input type="text" name="limit_memory" value="{{.CapForm.LMem}}" placeholder="—"></label>
+  </div>
+  <div class="q-form-actions"><button type="submit">Save</button>
+    {{if .CapForm.Scope}}<a href="/admin/quotas#capacity">Cancel</a>{{end}}</div>
+</form>
+{{else}}<p class="empty">Nothing to give a capacity quota to yet. Make an organization on the
+<a href="/admin/users">people page</a> first.</p>{{end}}
+{{end}}
+
 <details class="explain">
   <summary>How limits combine</summary>
   <p>Along a branch the most restrictive value wins, so a team cannot raise what

internal/web/templates/dashboard.html

@@ -89,7 +89,7 @@
   <td{{if .HeldFor}} class="warn" title="gave room to {{.HeldFor}}"{{end}}>{{.Class}}{{if .Preempted}} <span class="note" title="replicas lost to a higher class">−{{.Preempted}}</span>{{end}}</td>
   <td{{with .Warnings}} class="warn" title="{{join .}}"{{end}}>{{if or .MemoryGB .CPUCores}}{{.MemoryGB}}G{{if .CPUCores}} · {{.CPUCores}}c{{end}}{{else}}—{{end}}
     {{- if .MeasuredMemoryGB}} <span class="note">{{printf "%.1f" .MeasuredMemoryGB}}G used</span>{{end}}{{if .Warnings}} ⚠{{end}}</td>
-  <td>{{join .Profiles}}</td>
+  <td>{{if .External}}<span class="tag" title="served outside the cluster; its prompts leave it">external</span> {{.External}}{{else}}{{join .Profiles}}{{end}}</td>
   {{if $.AnyFallbacks}}<td>{{join .Fallbacks}}</td>{{end}}
   {{/* Cut to the column, the whole sentence on hover: a row of its own for
        each description made the table twice as long as the catalog. */}}

llmgw-plan.md

@@ -1459,7 +1459,8 @@ The launch path is now built too:
       hits. It does nothing when the working set fits in one replica's cache,
       which is worth knowing rather than assuming.
 - [x] Fallback models
-- [ ] Pool spillover
+- [x] Pool spillover: a saturated pool passes requests to the next one that
+      is not, by admission's own measure of saturation
 - [x] `ctl bench`: a load ramp that reports capacity as the last concurrency
       level meeting the SLO, plus the queue depth and KV usage observed there,
       so `weight`, `target_waiting` and `kv_ceiling` stop being guesses
@@ -1477,7 +1478,8 @@ The launch path is now built too:
 ### Phase 6 — v2 ideas
 - [ ] Precise KV index (§7.5 tier 2) — vLLM KV events plus tokenizer integration
 - [ ] Dynamo-style block cost function, tuning the `softmax` picker
-- [ ] Admission displacement (dropping requests from lower bands)
+- [x] Admission displacement: a total queue bound, and the newest request of
+      the lowest band turned away (503, Retry-After) for a higher one
 - [x] Autoscaling on load: a replica added while requests wait, up to a
       model's max, and taken back after ten quiet minutes
 - [x] Priority-based preemption: capacity classes (critical, standard,
@@ -1495,8 +1497,11 @@ The launch path is now built too:
 - [x] Weight downloading and cache management: the mirror, agent fetches
       checked against their digest, and reclaiming what nothing uses
 - [ ] Prefill/decode disaggregation (§12)
-- [ ] LoRA adapter routing (affinity to a replica that already has the adapter)
-- [ ] External provider support (as a fallback)
+- [x] LoRA adapters on llama.cpp: every replica loads a model's adapters and
+      applies one per request, so any replica serves any adapter
+- [ ] LoRA affinity for engines that load adapters on demand (vLLM)
+- [x] External provider support: a catalog model served by an OpenAI-compatible
+      API elsewhere, usable as a fallback, under the same allowlists and budgets
 - [x] Read-only dashboard on the admin listener (nodes, replicas, models,
       placement failures, usage) and a public landing page
 - [x] `/chat`: an in-browser playground for signed-in people, dispatched through