v0.10.6

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Load LoRA adapters at scale zero
2026-10-01 08:52:01 +02:00 · 2026-10-01 07:39:12 +02:00 · 2026-10-01 07:30:13 +02:00 · 2026-10-01 07:20:35 +02:00 · 2026-10-01 07:19:08 +02:00 · 2026-09-30 23:40:47 +02:00
114 changed files with 8062 additions and 378 deletions
@@ -93,6 +93,24 @@ jobs:
          ./dist/llmgw-${version}-linux-amd64 --version
          cat dist/checksums.txt

+      # Signed when the repository has a signing key, so a machine that trusts
+      # it installs nothing the server alone could have put there. Without the
+      # secret nothing is signed and releases are checked on their checksums,
+      # as before; a build that trusts a key refuses those, so the secret has
+      # to be in place before trusted.pub is filled in.
+      - name: Sign the checksums
+        env:
+          RELEASE_SIGNING_KEY: ${{ secrets.RELEASE_SIGNING_KEY }}
+        run: |
+          set -euo pipefail
+          version="${GITHUB_REF_NAME:-$(git describe --tags --always)}"
+          if [ -z "${RELEASE_SIGNING_KEY:-}" ]; then
+            echo "no RELEASE_SIGNING_KEY secret; releases are not signed"
+            exit 0
+          fi
+          ./dist/llmgw-${version}-linux-amd64 release sign dist/checksums.txt
+          echo "signed: dist/checksums.txt.sig"
+
      - name: Attach to the release
        env:
          TOKEN: ${{ secrets.GITEA_TOKEN }}
@@ -20,10 +20,265 @@ is only released once its **exit criterion** is met, not when its code lands.
 Until v1.0.0 the minor version is the breaking one: configuration formats and Go
 APIs change between minors without a deprecation period.

-## Unreleased
+## v0.10.6 — 2026-10-01

-Changes since v0.10.4, published as release candidates (v0.10.5-rc.N) until
-there are enough for v0.10.5.
+Published first as release candidates v0.10.6-rc.1 to rc.8; rc.5 failed its
+format check and has no binaries.
+
+Tried on the PoC cluster before release: LoRA adapters asked for by name
+through the gateway, two on one model, each answering as trained and the model
+itself unchanged; a saturated pool spilling to the next (8 of 12 concurrent
+requests on the preferred replica, 4 on the other); and a bronze request
+displaced from a full queue by a gold one, with 503 and Retry-After 5. Not tried
+live: a model served from an external provider, which needs a provider key.
+
+### Capacity from the admin pages
+
+What v0.10.5 made settable from the command line is now on the pages too:
+
+- **Quotas** has a Capacity section: every scope's guaranteed and most GPUs,
+  cores and memory beside what it holds, with a form to set, change or remove
+  one. It goes through the same checks as `llmgw ctl quota`.
+- **A model's card** on the mirror page sets its class, its owner (when there
+  are quotas to count against), and the memory and cores a replica reserves,
+  beside its replica counts. Only what changed is applied, and each change is
+  said.
+- **Room counts what a class may take.** The card, `llmgw ctl model target` and
+  the headroom API count the replicas a model could get by stopping those its
+  class outranks, separately from free room, and name whose they would be. A
+  critical model's card no longer refuses a replica that preemption would make
+  room for.
+
+### LoRA adapters
+
+A llama.cpp model can carry `adapters`, each a name and a GGUF LoRA file. The
+nodes fetch them with the model's weights and llama.cpp loads them all at start,
+applied to nothing (`--lora-scaled FILE:0`); a request that names an
+adapter goes to the model's replicas with `"lora": [{"id": i, "scale": s}]`,
+so one replica serves the model and all its adapters. Adapter names share the
+namespace of models, are listed in `/v1/models`, and follow key allowlists on
+their own — a key allowed an adapter does not reach the model under it. Adapter
+files are kept from weight cleanup like the model's own. Not yet for vLLM or
+SGLang profiles, whose adapters are loaded differently.
+
+### Finding and attaching adapters from the mirror page
+
+- **Explore** has a *LoRA adapters* view: adapters for any model, or — from a
+  held model's card, *find adapters on Hugging Face* — those made for the model
+  its repository was made from, since a GGUF repository is a quantisation and
+  adapters are made for the original.
+- **Opening a repository** reads the first bytes of each GGUF file and marks it
+  a *LoRA adapter* or a *model*. Repositories tagged "lora" hold a model with
+  the adapter merged in about as often as an adapter, and only the file says.
+- **Held adapter files** are marked in the list, and a model's card lists its
+  adapters, offers the held adapter files of its architecture to attach under a
+  name and scale, and removes them. A file that is not an adapter, or is for
+  another architecture, is refused before it reaches the catalog.
+- **`llmgw ctl model adapters MODEL [--add NAME=URL[@SCALE]] [--remove NAME]`**
+  lists and changes them from the command line, through
+  `POST /api/v1/models/{name}/adapters`, with the same checks.
+
+### Models served outside the cluster
+
+A catalog model can be an OpenAI-compatible API elsewhere: `external: {url,
+model, api_key_file}` and no profiles. Nothing is placed for it; the router
+sends its requests to the provider, rewritten to the provider's name for the
+model and with its key, read from the file on every publish so a rotated key is
+picked up. Named in another model's `fallbacks`, it is used only when the
+cluster has no capacity left. It is never scraped for metrics, and never reads
+as stale. Prices, budgets, limits, usage and key allowlists apply as to any
+model — and since its prompts leave the cluster, the allowlist is how to decide
+who may send them (docs/install.md). `ctl models` and the cluster page mark it
+external.
+
+### Pool spillover
+
+A model served from pools of different priority — H100 before A100, say — was
+routed to the preferred pool however full it was. Now a pool whose replicas are
+saturated, by the same measure admission uses (queue past its healthy depth, or
+KV cache past its ceiling), passes requests to the next pool that is not. When
+every pool is saturated the preferred one is kept: a pool that is just as full
+gains nothing.
+
+### Displacement: a higher band is not refused while a lower one waits
+
+Each band's queue was bounded, and nothing bounded them together, so a full
+gateway turned an interactive request away while batch requests sat queued
+behind it. `admission.queue_capacity_total` now bounds everything queued —
+by default half of every band's capacity together. Reached, a request is not
+refused while a lower band has any queued: the newest request of the lowest
+band is turned away instead, with a 503 and `Retry-After: 5`, so it backs off
+and returns. The newest, because the oldest have waited longest and are
+nearest their TTL. Counted as `displaced` in the admission rejection metric.
+
+### Releases can be signed
+
+`llmgw release keygen` makes an ed25519 key pair. With the private key in the
+CI secret `RELEASE_SIGNING_KEY`, releases carry `checksums.txt.sig`; with the
+public key in `internal/release/trusted.pub`, every binary built after refuses
+a release not signed with it, wherever it installs one. Off until then, and
+docs/install.md says in which order to turn it on.
+
+### Restarts borrow a lower class's room
+
+A restart places the replacement beside the replica it replaces, and where
+there is no free room it now makes some the way any placement does: by
+stopping a replica its class outranks, which gets its room back once the old
+replica has gone. The in-place restart — the model unserved while its
+replacement loads — is left for when nothing of a lower class is running. On a
+cluster where every model is standard, that is every full one: mark the models
+that can wait `opportunistic` (`llmgw ctl model class`, or the model's card).
+
+## v0.10.5 — 2026-09-30
+
+Published first as release candidates v0.10.5-rc.1 to rc.14; rc.5 failed its
+format check and has no binaries.
+
+After upgrading:
+
+- **Declare what CPU models need.** A CPU model added from now on must say its
+  memory; one stored before this still loads and is flagged in `ctl models`
+  until `llmgw ctl model resources MODEL --memory GB --cores N` is run. Cores
+  are then reserved per replica and become the engine's thread count, for
+  replicas started after the change: `llmgw ctl restart --model MODEL`.
+- **Upgrade the mirror on its own host**, now with one command:
+  `sudo llmgw upgrade v0.10.5 --service llmgw-mirror` (by hand once, for a
+  mirror older than this). `llmgw ctl upgrade` says so at the end.
+
+### CPU cores are reserved, and memory has to be declared
+
+The first step of resource allocation for larger clusters: what a replica
+holds of a node is reserved, not assumed.
+
+- **Cores.** A profile's `cpu_cores` is reserved out of its node's CPUs, as
+  memory is, and is what its engine computes on (llama.cpp's `--threads`). A
+  node holds as many CPU replicas as it has cores for, and two on one node no
+  longer spin against each other. Agents report their CPU count; a node that
+  does not, and a profile that declares no cores, constrain nothing.
+- **Memory is required of new CPU models.** A model with no GPU is all host
+  memory, and one declaring none looked to placement as if it fitted
+  everywhere, any number of times. Adding one without `min_memory_gb` is now
+  refused. Models stored before this still load, and say what is missing.
+- **Declared against measured.** Each model shows what one replica reserves
+  and the most one of its replicas uses — in `ctl models` (RESERVES) and on the
+  cluster page — with a warning when nothing is declared or when it uses more
+  than a tenth over what it reserves.
+- **Setting them.** `llmgw ctl model resources MODEL --memory GB --cores N`
+  (and `POST /api/v1/models/{name}/resources`) for a model added at runtime.
+  `ctl model serve` and the mirror's serve form fill in cores — half the
+  smallest node the model fits on — beside the memory they already estimate.
+  Running replicas keep their old figures until restarted.
+
+The commands are `llmgw ctl model resources` and `llmgw ctl model class`; in
+rc.7 the first was briefly `llmgw ctl resources`.
+
+### Capacity classes, and preemption
+
+A model has a class — `critical`, `standard` or `opportunistic` — saying what
+its capacity is worth against another's when there is not enough for both.
+Unset, an on-demand model is opportunistic and every other model standard,
+which is how the catalog behaved before. `llmgw ctl model class MODEL CLASS`
+sets it.
+
+When a model cannot be placed, replicas are stopped to make room for it, by
+these rules:
+
+- A lower class gives way to a higher one.
+- Within a class, only replicas above a model's minimum — room it grew into
+  under load — and only for a model still below its own minimum.
+- A critical model's minimum is never taken, nor a replica that has only just
+  loaded, nor one spanning several nodes.
+- Idle on-demand replicas go first, then what lower classes grew into, then
+  the same within the class, then lower classes' minimums; the longest unused
+  first within each, and of equally idle ones the smallest.
+- The set is tried on a copy of the inventory first, at most three replicas,
+  and nothing is stopped unless it makes the room. Of a set that does, each
+  replica whose stopping turns out not to be needed is left running, so an
+  idle replica tried first on the wrong node is not stopped with the one that
+  made the room.
+
+A replica stopped this way is drained and given 30 seconds to finish what it
+is serving. Its model is held back for up to five minutes, or until the model
+it gave way to has its replicas, so it does not take the room straight back.
+`ctl models` shows each model's class and "gave room to X" while held; the
+cluster page shows the class and how many replicas were lost; the metric is
+`llmgw_model_preempted_total{model,class}`, and every preemption is logged.
+
+### Capacity quotas for the tenancy tree
+
+An organization, team or project can be guaranteed GPUs, CPU cores and
+memory, and limited to at most some amount of each. What it holds is what the
+replicas of the models it owns hold — and those of the teams and projects
+beneath it. `llmgw ctl model owner MODEL team:Acme/ml` says whose a model is;
+one with no owner is shared and counts against nobody.
+
+- **A guarantee is kept.** A replica within its owner's guarantee is never
+  taken for another scope's model, whatever the classes: a guarantee a higher
+  class elsewhere could take would not be one.
+- **Beyond it is borrowing.** Free room is used; when a scope within its own
+  guarantee needs room, what others hold beyond theirs is taken back first,
+  whatever their class — except a critical model's minimum.
+- **A limit is a ceiling.** A model whose owner would go past a limit anywhere
+  up the tree gets no further replica, and says which limit; headroom and the
+  mirror card stop there too.
+
+`llmgw ctl quota set team:Acme/ml --gpus 8 --limit-gpus 16` (also `--cores`,
+`--memory` and their limits), `llmgw ctl quota list` with what each scope holds,
+`llmgw ctl quota clear`. Scopes are named by path or ID; a name that could mean
+two is refused. Changes take effect within half a minute. Metrics:
+`llmgw_capacity_holds`, `llmgw_capacity_guarantee` and `llmgw_capacity_limit`
+by scope and resource; alert: CapacityAtLimit.
+
+### Sized by what a replica was measured to serve
+
+What `llmgw ctl bench MODEL --save` measured — the most requests at once one
+replica served while meeting its SLO, and its throughput there — now drives
+decisions rather than only being printed:
+
+- **Scaling up** when requests per replica pass what was measured, not only
+  when the engines' queues fill: an engine takes requests into its slots long
+  after latency has gone.
+- **Scaling down** only when the replicas left would stay under 70% of what
+  they were measured to hold, so a replica taken back does not put the rest
+  straight over.
+- **Choosing a profile.** Among profiles of equal priority, the one measured
+  to serve the most per GPU — or per core — is tried first. Priority set by an
+  operator still wins.
+- **Sizing.** `llmgw ctl model target MODEL --tokens-per-sec N` says how many
+  replicas that takes, how many run, and whether the rest fit, with the reason
+  when they do not; `--apply` sets it if they do. Headroom reports what one
+  more replica would serve.
+
+A model never measured behaves exactly as before. Measurements are read from
+the store every half minute.
+
+### Upgrading a machine the control plane does not reach
+
+`sudo llmgw upgrade VERSION --service UNIT` upgrades the machine it runs on —
+the mirror's host above all, whose service runs unprivileged and cannot replace
+its own binary as an agent does. The release's checksums are checked, the old
+binary kept, the unit restarted and watched until it stays up.
+
+### Nodes report their own heat and power
+
+The first step of facility-aware placement (docs/facility-aware-placement.md),
+observe-only: nothing is decided on these yet. Every agent reads, at most every
+ten seconds, and sends with its heartbeat:
+
+- **GPUs** from nvidia-smi: temperature, HBM temperature where the driver has
+  it, power draw and the enforced limit, and what is holding the clocks down —
+  power cap, thermal slowdown, power brake. A GPU's temperature was read once,
+  at registration; it is now current.
+- **CPUs** from hwmon, which is what lm_sensors reads, with nothing to install:
+  the package temperature, and the board's inlet air sensor where it has one.
+- **CPU power** from RAPL's energy counters.
+
+A sensor a machine does not have is not reported, never shown as zero. New
+metrics: `llmgw_gpu_power_watts`, `llmgw_gpu_power_limit_watts`,
+`llmgw_gpu_memory_temperature_celsius`, `llmgw_gpu_throttled{reason}`,
+`llmgw_node_cpu_temperature_celsius`, `llmgw_node_cpu_power_watts`,
+`llmgw_node_inlet_temperature_celsius`. Each node's page shows them, with a GPU
+table of temperature, power against its limit, and what is throttling it.

 ### The mirror page in tabs

@@ -55,6 +310,19 @@ in amber when short, in red when parked or when it does not fit. It is one line
 per model beside its mode, and nodes drop the part of the name every node
 shares (node05 for llmgw-node05, the full name on hover).

+### One line per record, in every table
+
+The same rule across the admin pages. Headers do not wrap; a record is one
+line; free text is cut to its column with the whole of it on hover rather than
+given a row of its own; a row across the whole table is only a note or an open
+form. On the cluster page that means a node is one line (its address and logs
+beside its name, not under it), a model's description is a column rather than a
+second row, node names drop the prefix they all share, and the GPUs, classes,
+fabrics and fallbacks columns appear only when some row has something in them —
+on a CPU-only cluster they were three columns of dashes pushing the table past
+the page. On People, a person in many teams shows two and a count, the rest on
+hover.
+
 ## v0.10.4 — 2026-09-30

 Published first as release candidates v0.10.4-rc.1 to rc.9.
@@ -120,6 +120,10 @@ message Register {
  // meant "an agent too old to say" and constrained nothing — so a freshly
  // built node with no engine installed yet was a candidate for every model.
  bool engines_probed = 14;
+  // cpu_count is the CPUs the agent may run on. Placement reserves cores
+  // for CPU replicas out of it, the way it reserves memory out of
+  // memory_total_mb; zero, from an older agent, reserves nothing.
+  int32 cpu_count = 18;
  // imex is the service that lets a multi-node NVLink domain share memory.
  IMEX imex = 15;

@@ -281,6 +285,25 @@ message NodeLoad {
  int64 memory_used_bytes = 6;
  int64 memory_available_bytes = 7;
  repeated ShardLoad shards = 8;
+  // What the machine's own sensors say; zero is not measured. cpu_temp_c is
+  // the hottest CPU sensor, cpu_power_w the CPU packages' draw from RAPL,
+  // inlet_temp_c the board's inlet or ambient sensor where it has one.
+  double cpu_temp_c = 9;
+  double cpu_power_w = 10;
+  double inlet_temp_c = 11;
+  repeated GPUSensor gpus = 12;
+}
+
+// GPUSensor is one card's heat, power, and what is holding it back.
+message GPUSensor {
+  int32 index = 1;
+  double temp_c = 2;
+  double memory_temp_c = 3;
+  double power_w = 4;
+  double power_limit_w = 5;
+  // throttle names what is holding the clocks down now: sw_power_cap,
+  // hw_slowdown, sw_thermal, hw_thermal, power_brake.
+  repeated string throttle = 6;
 }

 // ShardLoad is one engine process group's use.
@@ -420,6 +443,8 @@ message Heartbeat {
  // domain, and the node has to stop being offered work that spans it.
  repeated GPUFabricState gpu_fabric = 16;
  IMEX imex = 17;
+  // cpu_count as in Register, for a VM given more CPUs while it runs.
+  int32 cpu_count = 18;
 }

 // Provision is the software a node should have: which engines, at which
@@ -530,6 +555,9 @@ message StartShard {
  // control plane from the replica id. The router sends it; nothing else
  // that can reach the node's ports has it.
  string api_key = 16;
+  // cpu_threads is how many CPU threads a shard with no GPU computes on:
+  // the cores placement reserved for it. Zero leaves it to the agent.
+  int32 cpu_threads = 17;
 }

 message EngineProfile {
@@ -544,6 +572,9 @@ message EngineProfile {
  map<string, string> env = 9;
  string model_path = 10;
  string served_model_name = 11;
+  // adapters are LoRA adapters loaded beside the model, in order: the index
+  // is the id a request applies one by.
+  repeated LoraAdapter adapters = 12;
 }

 // RemoveWeights deletes cached model files a node no longer needs.
@@ -676,3 +707,10 @@ message UpgradeReply {
  string version = 2;
  string error = 3;
 }
+
+// LoraAdapter is one adapter: the name clients ask for it by, and its source,
+// which the agent fetches as it does the model's weights.
+message LoraAdapter {
+  string name = 1;
+  string source = 2;
+}
@@ -84,6 +84,28 @@ groups:
            <model>` says what each node lacks. For an on-demand model, requests
            for it are told this rather than asked to retry.

+      - alert: ModelPreempted
+        expr: increase(llmgw_model_preempted_total[15m]) > 0
+        labels: { severity: info }
+        annotations:
+          summary: "{{ $labels.model }} ({{ $labels.class }}) gave up replicas to a higher class"
+          description: >-
+            Capacity ran short and a model of a higher class took this one's
+            room. Expected now and then; if it keeps firing for the same model,
+            the cluster is too small for what is asked of it at these classes.
+            `llmgw ctl models` shows who it gave way to.
+
+      - alert: CapacityAtLimit
+        expr: llmgw_capacity_holds >= on(scope, resource) llmgw_capacity_limit
+        for: 15m
+        labels: { severity: info }
+        annotations:
+          summary: "{{ $labels.scope }} holds all the {{ $labels.resource }} it may"
+          description: >-
+            Its models get no further replicas, under load or not, until
+            something of theirs stops or the limit is raised: `llmgw ctl quota
+            list`, `llmgw ctl quota set`.
+
      - alert: EngineInstallFailed
        expr: llmgw_node_engine_install{state="failed"} == 1
        for: 15m
@@ -0,0 +1,303 @@
+# Facility-aware placement: rack power, temperature and cooling
+
+**Status: step 0, reading the nodes' own sensors, is built; steps 1–5 are
+deferred** until the rack, PDU and power information of a real site is at
+hand — without it they would be designed against guesses.
+This is the design for step 5 of resource allocation, after reserved CPU cores
+and memory (1), capacity classes and preemption (2), tenant capacity quotas (3)
+and sizing by measured throughput (4), all of which shipped in v0.10.5. Of
+what follows, only the node sensors (rollout step 0) exist in the code: they
+are read, published and shown, and change no decision.
+
+## Why
+
+In a large installation the scarce thing is often not a GPU but the room it
+sits in. Three failures are invisible to placement today:
+
+- **A rack goes over its power budget.** Every node in it reports healthy;
+  placement sees free cards and fills them; the breaker, or the facility's
+  power cap, decides the rest. A rack with A and B feeds is worse: it runs
+  fine until one feed fails and the other cannot carry the load.
+- **Hot hardware is slow hardware.** GPUs throttle long before they fail. A
+  replica placed in a rack whose inlet air is too warm serves fewer tokens per
+  second than its measured capacity says, and nothing connects the two.
+- **Correlated failure.** Two replicas of a model on two nodes in the same rack
+  or on the same PDU are one failure away from zero. Placement spreads replicas
+  across nodes, not across racks.
+
+The goal is for placement to treat facility limits the way it already treats
+memory and cores: reserve against a budget, refuse what does not fit, say why,
+and never let missing data stop the cluster.
+
+## What it builds on
+
+| Already in place | Used for |
+|---|---|
+| Node labels, reported by agents, matched by `node_selector` | Where a node is: its rack, row, PDU |
+| `inventory.Allocation` carries GPUs, memory and cores per shard | Adding watts, summed per rack and PDU |
+| Placement rejections (`explain-placement`, headroom's "why not") | Naming the rack that is full or hot |
+| Headroom: the scheduler dry-run on a copy of the inventory | Room that respects power budgets, for free |
+| Preemption with classes, minimums and a dry run (step 2) | Shedding load from a hot rack in the right order |
+| Measured throughput per profile (step 4) | Tokens per watt as a placement preference |
+| Node sensors, read by every agent (step 0, built) | Heat and power with no facility integration: see below |
+| `internal/promq`, a Prometheus client | Reading facility metrics that DCIM exporters publish |
+
+## Concepts
+
+**Location.** Each node has a place in a tree: site, room, row, rack. Power has
+its own tree beside it: a rack draws from one or more PDUs, and each PDU from a
+feed. Both are read from node labels with well-known keys:
+
+```yaml
+# agent.yaml
+labels:
+  llmgw.io/site: sto1
+  llmgw.io/room: hall-b
+  llmgw.io/row: "12"
+  llmgw.io/rack: r12-07
+  llmgw.io/pdu: r12-07-a,r12-07-b   # both feeds, for a dual-fed rack
+```
+
+A node without these labels is its own rack of one: placement behaves as it
+does today.
+
+**Domain.** A rack, a row, a PDU, a room: anything with a budget or a
+condition. Domains are declared in a facility file (below) or discovered from
+node labels; a label naming a domain the file does not declare creates it with
+no budget and no condition.
+
+**Budget.** A domain may have a power cap in watts. Placement reserves each
+replica's estimated draw against every domain its node belongs to and refuses a
+placement that would take any of them past its cap less a safety margin. For a
+dual-fed rack the cap that matters is what one feed can carry alone, since that
+is what the rack must survive.
+
+**Condition.** A domain has a thermal and cooling condition: `ok`, `warm`, `hot`
+or `unknown`, derived from readings against thresholds with hysteresis.
+Conditions change what placement prefers and allows; they are not amounts to
+fill.
+
+**Reading.** A measured value with a time: inlet temperature, power draw, a
+cooling unit's alarm state. Readings older than their staleness limit are
+treated as absent.
+
+## Estimating a replica's draw
+
+What placement reserves has to be known before the replica runs, so it is an
+estimate first and a measurement later:
+
+1. **Declared.** A profile may say `power_watts`. The operator's figure wins.
+2. **Derived.** Otherwise GPU profiles take each card's power limit from NVML
+   (the agent already reads the cards; the power limit is one more field) plus
+   a per-node share for CPU, memory and fans; CPU profiles take their reserved
+   cores times a per-core figure for the node's CPU model.
+3. **Measured.** Once replicas run, the agent reports their cards' actual draw
+   (NVML) and, where the BMC allows, the node's (Redfish or IPMI DCMI). A
+   domain's headroom is computed from the larger of what is reserved and what
+   is measured, so an estimate that was too low is corrected by the room it
+   stops offering, not by an outage.
+
+## Decisions
+
+### Power
+
+- **Hard limit.** A placement that would take any domain past `cap × (1 −
+  margin)` is rejected, with the domain and the figures: *rack r12-07: 14.2 of
+  16.0 kW reserved, replica needs ~2.8, margin 10%*.
+- **Feeds.** For a domain with two feeds the cap is the smaller feed's rating
+  times its derating (typically 80%), not the sum.
+- **Already over.** A domain found over its cap, because the estimate was low
+  or the cap was lowered, gets no new placements, and an alert. It is not
+  emptied automatically; shedding power is the facility's decision, and
+  stopping inference to do it is ours only if asked (see shedding below).
+
+### Temperature and cooling
+
+Thresholds are per domain with defaults, and each condition needs to hold for a
+while before it changes, in both directions, so a single reading neither
+starts nor ends anything:
+
+| Condition | Enters when | Placement |
+|---|---|---|
+| `ok` | below the warm threshold for 5 min | normal |
+| `warm` | inlet above the warm threshold for 2 min, GPU temperatures near throttling, or any GPU thermally throttled | allowed, scored down: other domains first |
+| `hot` | inlet above the hot threshold for 2 min, a cooling unit in alarm, or a sustained share of the domain's GPUs thermally throttled | no new placements, as a cordon would, but automatic and reversible |
+| `unknown` | no fresh reading | as `ok`, and reported as unknown |
+
+**Shedding** (optional, off by default): a domain `hot` for longer than its
+shed delay has load moved off it using step 2's machinery. The lowest class goes
+first, and replicas above their model's minimum go before any at it. A critical
+minimum is never touched, and moves are rate limited to one replica per domain
+per interval. The replica is placed elsewhere first where there is room, so
+shedding moves work rather than dropping it. GPU power capping (`nvidia-smi
+-pl`) is a gentler alternative worth measuring against shedding: less
+throughput, no restart.
+
+### Spreading
+
+Replicas of one model are spread across racks, and across PDUs, the way they
+are spread across nodes today. This is a preference, dropped when it is the only
+thing standing in the way, as node spreading already is.
+
+### Scoring
+
+Placement today filters, then takes the first candidate in a fixed order
+(fewest free GPUs for GPU profiles, fewest replicas for CPU ones, nodes holding
+the weights first). Facility signals are better as a score than as more
+filters. Candidates that pass the hard limits are ordered by:
+
+1. condition: `ok` before `warm`;
+2. power headroom left in the rack after this replica, more first, so load
+   levels across racks;
+3. the existing order.
+
+Once measurements include power, step 4's profile preference can use tokens per
+watt instead of tokens per GPU where both are known.
+
+## Where the data comes from
+
+A `FacilitySource` interface, implemented several ways, chosen in
+`server.yaml`:
+
+```go
+type FacilitySource interface {
+    // Domains is what the source knows: budgets, thresholds, members.
+    Domains(ctx) ([]Domain, error)
+    // Readings is the latest value of each signal, per domain.
+    Readings(ctx) ([]Reading, error)
+}
+```
+
+- **Static file.** Domains, caps and thresholds, written by hand or generated
+  from a DCIM export. Enough for power budgets with no live data at all.
+- **Prometheus.** DCIM, PDU (SNMP exporter) and BMC (Redfish exporter)
+  readings usually end up in Prometheus. The source runs configured instant
+  queries and maps their labels onto domains:
+
+  ```yaml
+  facility:
+    source: prometheus
+    prometheus: http://monitoring:9090
+    signals:
+      inlet_temp_c:
+        query: max by (rack) (dcim_rack_inlet_temperature_celsius)
+        domain_label: rack
+        stale_after: 2m
+      power_watts:
+        query: sum by (pdu) (pdu_outlet_power_watts)
+        domain_label: pdu
+      cooling_alarm:
+        query: max by (row) (crac_alarm)
+        domain_label: row
+  ```
+- **Agents (built, step 0).** What each node reads about itself, every ten
+  seconds at most, sent with its heartbeat:
+  - **GPUs, from nvidia-smi:** temperature, memory (HBM) temperature where the
+    driver reports it, power draw, the enforced power limit, and what is
+    holding the clocks down now — `sw_power_cap`, `hw_slowdown`, `sw_thermal`,
+    `hw_thermal`, `power_brake`.
+  - **CPUs, from hwmon** (what lm_sensors reads, without it installed): the
+    package temperature, or the hottest core; and the board's inlet or ambient
+    sensor where it has one (`SYSTIN`, `Inlet`, `Ambient`).
+  - **CPU power, from RAPL:** the packages' energy counters over time,
+    subdomains not counted twice, counter wrap handled.
+
+  A machine without a sensor reports nothing for it, never a zero; a VM has
+  none of them. Published as `llmgw_gpu_temperature_celsius`,
+  `llmgw_gpu_memory_temperature_celsius`, `llmgw_gpu_power_watts`,
+  `llmgw_gpu_power_limit_watts`, `llmgw_gpu_throttled{reason}`,
+  `llmgw_node_cpu_temperature_celsius`, `llmgw_node_cpu_power_watts` and
+  `llmgw_node_inlet_temperature_celsius`, and on each node's page.
+
+  What they can and cannot say:
+  - **Throttling is the strongest signal there is.** A temperature is a proxy;
+    a card holding its clocks down is the fact that invalidates its measured
+    capacity, and it needs no threshold to be chosen.
+  - **Measured power** is GPU draw plus CPU packages; memory, disks, fans and
+    power-supply losses are not in it, so a node's draw is that plus an
+    estimated remainder. A rack's is its llmgw nodes' plus a declared figure
+    for everything else in it: switches and storage report nothing here.
+  - **CPU and GPU temperatures follow the load, not the room.** A hot rack is
+    inferred from many nodes running hot at comparable load, or from
+    throttling, not from one node's CPU. Only a board inlet sensor reads the
+    air, and most boards have none.
+  - **No budget comes from a sensor.** A reading is a draw; the cap it is
+    measured against still comes from the facility.
+- **Simulated.** Readings from a script, for tests and for rehearsing a
+  thermal event on a cluster without facility data.
+
+## Safety
+
+- **Missing data never blocks.** A source that is down, a reading that is stale,
+  a node without location labels: placement behaves as it does today, and says
+  the data is missing (a metric, a line on the cluster page). Facility
+  awareness can make placement more careful, never impossible.
+- **Hysteresis and hold times** on every condition, so a noisy sensor does not
+  flap a rack between allowed and forbidden.
+- **Observe-only first.** A mode where every decision is computed, logged and
+  shown ("would have refused: rack r12-07 hot") but nothing is enforced, run
+  for long enough to trust the thresholds before turning enforcement on.
+- **Manual override.** `llmgw ctl facility hold RACK` and `release RACK`, for the
+  facility team to mark a domain unavailable, or to clear a condition they know
+  to be a sensor fault.
+- **Shedding is opt-in**, per domain or cluster-wide, and rate limited.
+
+## What people see
+
+- `explain-placement` and headroom name the domain: *rack r12-07 is hot (inlet
+  31°C for 6 min)*, *pdu r12-07-a: 7.1 of 7.4 kW*.
+- `llmgw ctl facility` lists domains with budget, reserved, measured and
+  condition. The cluster page gets a rack summary beside the nodes table.
+- Metrics: `llmgw_domain_power_budget_watts`, `_reserved_watts`,
+  `_measured_watts`, `llmgw_domain_condition{condition}`,
+  `llmgw_domain_reading_age_seconds`; alerts for a domain near its cap, a
+  domain hot, a domain over its cap, and facility data gone stale.
+
+## Rollout
+
+Each step ships on its own and is useful without the next:
+
+0. **Node sensors, observe-only (built).** GPU heat, power and throttling,
+   CPU heat and power, inlet air where the board has a sensor; published and
+   shown, no decision changed.
+1. **Location and spreading.** Well-known labels, domains from labels, replicas
+   spread across racks and PDUs, locations on the node views. No facility data
+   needed.
+2. **Power budgets from a static file.** Caps and margins, estimated draw per
+   replica (declared or derived), hard limit in placement and headroom.
+3. **Readings, observe-only.** The Prometheus source beside the agents',
+   conditions with hysteresis (throttling first among them), measured power
+   folded into headroom; decisions shown, not enforced.
+4. **Thermal enforcement.** `warm` scored down, `hot` refused, manual hold and
+   release.
+5. **Shedding and tokens per watt.** Opt-in shedding through step 2's
+   preemption; measured power in profile preference.
+
+## Testing
+
+- The simulated source drives unit tests of every condition transition, the
+  hysteresis, staleness, and the placement outcomes, with mutation checks as
+  for steps 1–4.
+- Property tests: whatever the readings, placement never exceeds a declared
+  cap with fresh data, and never refuses everything because data is missing.
+- On the PoC cluster, with no facility data: step 1 with labelled VMs, step 2
+  with a static file of made-up caps, steps 3–4 with the simulated source
+  replaying a thermal event.
+- Before enforcement anywhere real: a period in observe-only mode against the
+  site's actual data.
+
+## Open questions
+
+- **What the sites have.** Which DCIM, which PDUs, which BMCs, and whether any
+  of it already reaches a Prometheus. The source design assumes Prometheus is
+  the common ground; a site without it needs another source.
+- **Who owns the numbers.** Caps, deratings and thresholds belong to the
+  facility team. Whether they edit llmgw's file, or llmgw reads theirs.
+- **Phase balance.** Three-phase PDUs have per-phase limits; whether placement
+  should see phases or only the PDU total.
+- **Cooling granularity.** Cooling is often per row or per aisle, not per rack;
+  the domain tree allows either, but the defaults should match what sites
+  actually measure.
+- **Power capping instead of shedding.** Whether to lower GPU power limits in a
+  warm rack rather than move replicas, and how that interacts with measured
+  capacity from step 4, which would no longer hold at the lower limit.
@@ -671,6 +671,74 @@ llmgw ctl explain-placement qwen2.5-coder-3b-instruct
 which names every candidate and why each was rejected, rather than leaving you
 to infer it.

+### LoRA adapters
+
+A llama.cpp model can carry LoRA adapters, each asked for by its own name as
+though it were a model:
+
+```yaml
+- name: qwen2.5-coder-7b-instruct
+  source: http://mirror:8081/qwen2.5-coder-7b-instruct/5d567d26/model.gguf
+  adapters:
+    - name: coder-sql
+      source: http://mirror:8081/coder-sql/v1/adapter.gguf
+      scale: 1.0          # optional; 1 when left out
+  profiles: [...]
+```
+
+Every replica of the model fetches every adapter and loads it at start, applied
+to nothing. A request for `coder-sql` goes to the model's replicas with the
+adapter applied to it alone; a request for the model gets the model. Adapter
+names share the namespace of models and aliases, appear in `/v1/models`, and
+follow key allowlists on their own: a key allowed `coder-sql` reaches that
+adapter and not the model under it, while a key allowed the model reaches its
+adapters too. Adding or removing an adapter is a change to the model; running
+replicas load the new set when restarted (`llmgw ctl restart --model M`).
+Besides the mirror page's model card, adapters are attached from the command
+line, which reads the file first and refuses one that is not an adapter for
+this model's architecture:
+
+    llmgw ctl model adapters coder --add coder-sql=http://mirror:8081/sql-lora/v1/sql.gguf@0.8
+    llmgw ctl model adapters coder --remove coder-sql
+
+The adapter file must be a GGUF LoRA made for this model — llama.cpp's
+`convert_lora_to_gguf.py` makes one from a PEFT adapter — and not a model with
+the adapter already merged in, which is what most "LoRA GGUF" repositories hold.
+
+### Models served outside the cluster
+
+A model can be an OpenAI-compatible API elsewhere — a hosted provider, or
+another gateway — rather than replicas here. Nothing is placed for it; the
+router sends its requests on, with the provider's key and the provider's name
+for the model:
+
+```yaml
+- name: cloud-mini
+  source: external
+  external:
+    url: https://api.openai.com          # without /v1
+    model: gpt-4o-mini                   # what the provider calls it
+    api_key_file: /etc/llmgw/secrets/openai.key
+```
+
+Named in another model's `fallbacks`, it is used only when the cluster has no
+capacity left for that model:
+
+```yaml
+- name: qwen2.5-coder-7b-instruct
+  fallbacks: [cloud-mini]
+  ...
+```
+
+The key is read from its file on the control plane, never stored in the
+catalog, and a key rotated in the file is used within seconds. A model whose key
+file cannot be read is not offered. Everything else applies as to any model:
+prices and budgets, rate limits, usage records, and key allowlists.
+
+**Its prompts leave the cluster.** Decide who may send them: a key or project
+whose allowlist leaves the external model out cannot reach it, as a fallback or
+by name. The cluster page and `ctl models` mark it `external`.
+
 ### How many replicas, and how much each one carries

 Replica count is changed while the cluster runs, without editing the
@@ -940,8 +1008,18 @@ restart either — it rebuilds the fleet from what the agents report.
 fields, and both sides treat an absent one as unknown rather than as a default.
 Agents older than v0.10.4 cannot upgrade themselves: move those by hand once,
 installing the release binary and restarting `llmgw-agent`, and every upgrade
-after that is the command above. The mirror, which is not an agent, is
-upgraded the same way by hand.
+after that is the command above.
+
+**The mirror is upgraded on its own host**, since it is not an agent and its
+service cannot replace its own binary:
+
+```
+sudo llmgw upgrade v0.10.5 --service llmgw-mirror
+```
+
+The same checks — the release's checksums, the old binary kept beside the new —
+then a restart of the unit, waiting for it to stay up. A mirror older than
+v0.10.5 does not have the command: install the release binary by hand once.

 Then confirm what you have, rather than assuming:

@@ -956,6 +1034,33 @@ replica alone on purpose, because churning one costs more than the difference.
 `llmgw ctl restart <replica>` replaces it with one on the new profile, starting
 the new one before stopping the old.

+### Signed releases
+
+Every install above checks a release against its published checksums, which
+proves the download is what the server holds — not that the server holds what
+was built. Signing closes that: CI signs the checksums, and a binary that
+trusts a key installs only releases signed with it. Off until a key exists;
+turning it on, once:
+
+```
+llmgw release keygen
+```
+
+1. Put the **private** key in the repository's Actions secret
+   `RELEASE_SIGNING_KEY`. From the next tag, releases carry
+   `checksums.txt.sig`.
+2. Put the **public** key in `internal/release/trusted.pub` and release. Every
+   binary built from then on carries it and refuses releases that are not
+   signed with it — `ctl upgrade`, agents upgrading themselves, and
+   `llmgw upgrade` alike.
+
+In that order: a build that trusts a key refuses unsigned releases, so the
+secret has to be signing before the key is trusted. The first trusting release
+is itself installed by a binary that trusts nothing; every one after is checked
+by the one before. `LLMGW_RELEASE_KEY` overrides the built-in key, for rotating
+it. The onboarding script fetches with `curl` and checks checksums only; a node
+is checked from its first upgrade on.
+
 ## Backing up

 `llmgw server backup` writes one archive holding everything a control plane
@@ -15,12 +15,16 @@ rather than repeated.

 | Key | Type | In |
 |---|---|---|
+| `name` | `string` | [Adapter](#adapter) |
+| `source` | `string` | [Adapter](#adapter) |
+| `scale` | `float64` | [Adapter](#adapter) |
 | `enabled` | `bool` | [Admission](#admission) |
 | `bands` | `[]Band` | [Admission](#admission) |
 | `default_band` | `string` | [Admission](#admission) |
 | `band_header` | `string` | [Admission](#admission) |
 | `queue_ttl` | `time.Duration` | [Admission](#admission) |
 | `queue_capacity_per_band` | `int` | [Admission](#admission) |
+| `queue_capacity_total` | `int` | [Admission](#admission) |
 | `interval` | `time.Duration` | [Admission](#admission) |
 | `enabled` | `bool` | [Auth](#auth) |
 | `store` | `string` | [Auth](#auth) |
@@ -77,6 +81,9 @@ rather than repeated.
 | `pool` | `string` | [Endpoint](#endpoint) |
 | `weight` | `float64` | [Endpoint](#endpoint) |
 | `priority` | `int` | [Endpoint](#endpoint) |
+| `url` | `string` | [External](#external) |
+| `model` | `string` | [External](#external) |
+| `api_key_file` | `string` | [External](#external) |
 | `contact` | `string` | [Landing](#landing) |
 | `notice` | `string` | [Landing](#landing) |
 | `bin` | `string` | [LlamaCppConfig](#llamacppconfig) |
@@ -92,6 +99,10 @@ rather than repeated.
 | `source` | `string` | [Model](#model) |
 | `profiles` | `[]Profile` | [Model](#model) |
 | `scaling` | `Scaling` | [Model](#model) |
+| `class` | `string` | [Model](#model) |
+| `owner` | `string` | [Model](#model) |
+| `external` | `*External` | [Model](#model) |
+| `adapters` | `[]Adapter` | [Model](#model) |
 | `fallbacks` | `[]string` | [Model](#model) |
 | `routing_profile` | `string` | [Model](#model) |
 | `slo` | `SLO` | [Model](#model) |
@@ -129,6 +140,7 @@ rather than repeated.
 | `args` | `[]string` | [Profile](#profile) |
 | `env` | `map[string]string` | [Profile](#profile) |
 | `min_memory_gb` | `int` | [Profile](#profile) |
+| `cpu_cores` | `int` | [Profile](#profile) |
 | `min_disk_gb` | `int` | [Profile](#profile) |
 | `node_selector` | `map[string]string` | [Profile](#profile) |
 | `weight` | `float64` | [Profile](#profile) |
@@ -192,6 +204,27 @@ rather than repeated.
 | `launch_mode` | `string` | [VLLMConfig](#vllmconfig) |
 | `ray_bin` | `string` | [VLLMConfig](#vllmconfig) |

+## Adapter
+
+### `name`
+
+`string`
+
+Name is what clients ask for, like a model name.
+
+### `source`
+
+`string`
+
+Source is the adapter file (GGUF for llama.cpp), fetched by each node
+as the model's weights are.
+
+### `scale`
+
+`float64`
+
+Scale is how strongly it is applied; zero takes 1.
+
 ## Admission

 ### `enabled`
@@ -225,6 +258,15 @@ should not be able to promote themselves.

 `int`

+### `queue_capacity_total`
+
+`int`
+
+QueueCapacityTotal bounds everything queued across bands. Reached, a
+request is not refused while a lower band has any queued: the newest of
+the lowest band is turned away instead, with a 503 and Retry-After.
+Zero takes half of all the bands' capacities together.
+
 ### `interval`

 `time.Duration`
@@ -603,6 +645,28 @@ called. Empty means vLLM's names.

 `int`

+## External
+
+### `url`
+
+`string`
+
+URL is the provider's base address, without /v1: https://api.openai.com.
+
+### `model`
+
+`string`
+
+Model is what the provider calls it; requests are rewritten to it.
+
+### `api_key_file`
+
+`string`
+
+APIKeyFile holds the provider's key, on the control plane. A file
+rather than the key itself, so the key is in neither the catalog, the
+store nor the audit log; empty for a provider that asks for none.
+
 ## Landing

 ### `contact`
@@ -735,6 +799,45 @@ Source is the weights path or a HuggingFace id.

 Takes a [Scaling](#scaling) block.

+### `class`
+
+`string`
+
+Class is how much this model's capacity is worth against another's,
+when there is not enough for both: critical, standard or opportunistic.
+Empty is standard, or opportunistic for a model loaded on demand. See
+EffectiveClass.
+
+### `owner`
+
+`string`
+
+Owner is the part of the tenancy tree whose capacity quota this model
+counts against, written org:ID, team:ID or project:ID. Empty is shared:
+held by the cluster, counted against nobody.
+
+### `external`
+
+`*External`
+
+External is a model served by someone else — an OpenAI-compatible API
+outside the cluster — rather than by replicas placed here. Such a model
+has no profiles and nothing is placed for it; the router sends its
+requests to the provider. Most useful named in another model's
+fallbacks: used only when the cluster has no capacity left. Its
+prompts leave the cluster, so which keys may name it is worth an
+allowlist.
+
+Takes a [External](#external) block.
+
+### `adapters`
+
+`[]Adapter`
+
+Adapters are LoRA adapters served on this model's replicas, each asked
+for by its own name as though it were a model. Every replica loads
+every adapter, applying one only to a request that names it.
+
 ### `fallbacks`

 `[]string`
@@ -966,6 +1069,20 @@ this. A node that does not report its memory is not excluded either —
 an older agent must not have its nodes quietly emptied by an upgrade of
 the control plane.

+### `cpu_cores`
+
+`int`
+
+CPUCores is the cores a replica of a profile with no GPU is given:
+reserved out of its node's CPUs by placement, as memory is, and what
+its engine computes on (llama.cpp's --threads).
+
+Reserved because an engine takes every core it is told about, and two
+on one node that each think they have all of them spin against each
+other: on an 8-vCPU node, two 7B replicas fell from 10 tokens a second
+to under one each. Zero reserves nothing and leaves the thread count to
+the node, which is every profile from before this.
+
 ### `min_disk_gb`

 `int`
@@ -16,6 +16,7 @@ import (
 	"net/http"
 	"os"
 	"path/filepath"
+	"runtime"
 	"slices"
 	"sync"
 	"time"
@@ -100,6 +101,9 @@ type Agent struct {
 	smsErr map[string]string
 	// load measures the machine between heartbeats.
 	load *discovery.LoadSampler
+	// sensors reads heat and power: CPU and board from hwmon and RAPL, GPUs
+	// from nvidia-smi.
+	sensors *discovery.SensorSampler

 	// fetchWatchers are the shards waiting on each source, so progress on one
 	// download reaches every shard that asked for it. A fetch is single-flight
@@ -145,6 +149,7 @@ func New(cfg Config, httpClient connectHTTPClient, log *slog.Logger) (*Agent, er
 		logRing:      ring,
 		provKick:     make(chan struct{}, 1),
 		load:         discovery.NewLoadSampler(),
+		sensors:      discovery.NewSensorSampler(),
 	}
 	a.prov = &provision.Installer{Root: cfg.EnginesDir, Log: log, OnStatus: a.provisionStatus}
 	return a, nil
@@ -549,6 +554,7 @@ func (a *Agent) heartbeat() *llmgwv1.AgentMessage {
 	// and a node that grew from 7 GB to 32 GB has to be seen to have grown
 	// without anybody restarting anything.
 	hb.MemoryTotalMb = int64(discovery.MemoryTotalMB())
+	hb.CpuCount = int32(runtime.NumCPU())

 	a.reprobeEngines()
 	a.mu.Lock()
@@ -636,6 +642,16 @@ func (a *Agent) sampleLoad() *llmgwv1.NodeLoad {
 			ShardId: sh.ID, CpuCores: sh.CPUCores, MemoryRssBytes: sh.RSSBytes,
 		})
 	}
+	if a.sensors != nil {
+		se := a.sensors.Sample(context.Background())
+		out.CpuTempC, out.CpuPowerW, out.InletTempC = se.CPUTempC, se.CPUPowerW, se.InletTempC
+		for _, g := range se.GPUs {
+			out.Gpus = append(out.Gpus, &llmgwv1.GPUSensor{
+				Index: int32(g.Index), TempC: g.TempC, MemoryTempC: g.MemoryTempC,
+				PowerW: g.PowerW, PowerLimitW: g.PowerLimitW, Throttle: g.Throttle,
+			})
+		}
+	}
 	return out
 }

@@ -875,6 +891,17 @@ func (a *Agent) startShard(ctx context.Context, m *llmgwv1.StartShard, out chan<
 		stop()
 		spec.Profile.ModelPath = path
 	}
+	// Adapters are fetched like the model, and before the engine starts: it
+	// loads them at start, and one missing is a replica that answers for the
+	// base model under the adapter's name.
+	for i, ad := range spec.Profile.Adapters {
+		path, err := a.weights().Ensure(ctx, ad.Path)
+		if err != nil {
+			report(supervisor.Failed, fmt.Sprintf("adapter %s: %v", ad.Name, err), nil)
+			return
+		}
+		spec.Profile.Adapters[i].Path = path
+	}

 	cmds, err := eng.Commands(spec)
 	if err != nil {
@@ -0,0 +1,242 @@
+package discovery
+
+import (
+	"context"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+)
+
+// Sensors is what a node's own hardware says about heat and power: the CPUs,
+// the board where it has an inlet sensor, and each GPU. A zero is not
+// measured, never a reading of zero — a VM has none of these, and must not
+// look like a machine running at 0 °C on no power.
+type Sensors struct {
+	// CPUTempC is the hottest CPU sensor: the package, or the hottest core
+	// where no package reading exists.
+	CPUTempC float64
+	// CPUPowerW is the CPU packages' draw, from RAPL: the change in their
+	// energy counters over the time between two samples.
+	CPUPowerW float64
+	// InletTempC is the air coming in, where the board has a sensor it calls
+	// that. Most do not, and a CPU's temperature is not a substitute: it
+	// follows the load, not the room.
+	InletTempC float64
+	GPUs       []GPUSensor
+}
+
+// GPUSensor is one card's heat, power and whether it is slowing itself down.
+type GPUSensor struct {
+	Index       int
+	TempC       float64
+	MemoryTempC float64
+	PowerW      float64
+	PowerLimitW float64
+	// Throttle names what is holding the card's clocks down now:
+	// sw_power_cap, hw_slowdown, sw_thermal, hw_thermal, power_brake. Empty
+	// is running free. Idle and application clock settings are left out;
+	// they are not the card being held back.
+	Throttle []string
+}
+
+// sensorEvery bounds how often nvidia-smi is run. Heartbeats come more often
+// than temperatures change, and each run costs a few hundred milliseconds.
+const sensorEvery = 10 * time.Second
+
+// SensorSampler reads Sensors. The first sample has no CPU power, which needs
+// two readings of the energy counters.
+type SensorSampler struct {
+	sys string // /sys, or a test's copy of it
+	smi func(ctx context.Context, fields string) ([]byte, error)
+	now func() time.Time
+
+	mu         sync.Mutex
+	last       Sensors
+	lastAt     time.Time
+	prevEnergy map[string]uint64
+	prevAt     time.Time
+}
+
+// NewSensorSampler reads this machine.
+func NewSensorSampler() *SensorSampler {
+	return &SensorSampler{sys: "/sys", smi: runSMI, now: time.Now}
+}
+
+func runSMI(ctx context.Context, fields string) ([]byte, error) {
+	ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
+	defer cancel()
+	//nolint:gosec // fixed binary, fields from this file
+	return exec.CommandContext(ctx, "nvidia-smi", "--query-gpu="+fields,
+		"--format=csv,noheader,nounits").Output()
+}
+
+// Sample reads the sensors, or returns the last reading when that is recent.
+func (s *SensorSampler) Sample(ctx context.Context) Sensors {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	now := s.now()
+	if !s.lastAt.IsZero() && now.Sub(s.lastAt) < sensorEvery {
+		return s.last
+	}
+	out := Sensors{}
+	out.CPUTempC, out.InletTempC = s.hwmon()
+	out.CPUPowerW = s.rapl(now)
+	out.GPUs = s.gpus(ctx)
+	s.last, s.lastAt = out, now
+	return out
+}
+
+// hwmon reads the hottest CPU sensor and an inlet sensor, from every hwmon
+// device the kernel exposes — what lm_sensors reads, without it installed.
+func (s *SensorSampler) hwmon() (cpu, inlet float64) {
+	devs, _ := filepath.Glob(filepath.Join(s.sys, "class/hwmon/hwmon*"))
+	var pkg, core float64
+	for _, d := range devs {
+		name := readTrim(filepath.Join(d, "name"))
+		inputs, _ := filepath.Glob(filepath.Join(d, "temp*_input"))
+		for _, in := range inputs {
+			milli, err := strconv.ParseFloat(readTrim(in), 64)
+			if err != nil || milli <= 0 {
+				continue
+			}
+			c := milli / 1000
+			label := strings.ToLower(readTrim(strings.TrimSuffix(in, "_input") + "_label"))
+			switch {
+			case isCPUSensor(name) && (strings.HasPrefix(label, "package") || label == "tctl" || label == "tdie"):
+				pkg = max(pkg, c)
+			case isCPUSensor(name):
+				core = max(core, c)
+			case label == "cpu":
+				// A board's own reading of the CPU, as Dell's SMM gives.
+				core = max(core, c)
+			case strings.Contains(label, "inlet") || strings.Contains(label, "ambient") ||
+				strings.Contains(label, "intake") || label == "systin":
+				inlet = max(inlet, c)
+			}
+		}
+	}
+	if pkg > 0 {
+		return pkg, inlet
+	}
+	return core, inlet
+}
+
+func isCPUSensor(name string) bool {
+	switch name {
+	case "coretemp", "k10temp", "zenpower", "cpu_thermal":
+		return true
+	}
+	return false
+}
+
+// rapl is the CPU packages' power from their energy counters. Package
+// domains only: the dram and core subdomains are parts of a package, and
+// adding them would count the same energy twice.
+func (s *SensorSampler) rapl(now time.Time) float64 {
+	doms, _ := filepath.Glob(filepath.Join(s.sys, "class/powercap/intel-rapl:*"))
+	energy := map[string]uint64{}
+	wrap := map[string]uint64{}
+	for _, d := range doms {
+		if strings.Count(filepath.Base(d), ":") != 1 {
+			continue // intel-rapl:0:0 is a subdomain of intel-rapl:0
+		}
+		e, err := strconv.ParseUint(readTrim(filepath.Join(d, "energy_uj")), 10, 64)
+		if err != nil {
+			continue // unreadable without root on newer kernels
+		}
+		energy[d] = e
+		wrap[d], _ = strconv.ParseUint(readTrim(filepath.Join(d, "max_energy_range_uj")), 10, 64)
+	}
+	prev, prevAt := s.prevEnergy, s.prevAt
+	s.prevEnergy, s.prevAt = energy, now
+	dt := now.Sub(prevAt).Seconds()
+	if prev == nil || dt <= 0 || len(energy) == 0 {
+		return 0
+	}
+	var joules float64
+	for d, e := range energy {
+		p, ok := prev[d]
+		if !ok {
+			return 0 // a domain appeared: no fair difference this time
+		}
+		if e < p { // the counter wrapped
+			e += wrap[d]
+		}
+		joules += float64(e-p) / 1e6
+	}
+	return joules / dt
+}
+
+// The throttle reasons nvidia-smi reports, as bits. Only those that mean the
+// card is being held back are named.
+var throttleBits = []struct {
+	bit  uint64
+	name string
+}{
+	{0x4, "sw_power_cap"},
+	{0x8, "hw_slowdown"},
+	{0x20, "sw_thermal"},
+	{0x40, "hw_thermal"},
+	{0x80, "power_brake"},
+}
+
+// gpus asks nvidia-smi for each card's heat and power. The memory temperature
+// is not known to every driver, and a field nvidia-smi does not know fails the
+// whole query, so it is asked for first and dropped on failure.
+func (s *SensorSampler) gpus(ctx context.Context) []GPUSensor {
+	withMem := "index,temperature.gpu,power.draw,enforced.power.limit,clocks_throttle_reasons.active,temperature.memory"
+	out, err := s.smi(ctx, withMem)
+	if err != nil {
+		if out, err = s.smi(ctx, strings.TrimSuffix(withMem, ",temperature.memory")); err != nil {
+			return nil
+		}
+	}
+	var gs []GPUSensor
+	for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") {
+		f := strings.Split(line, ",")
+		if len(f) < 5 {
+			continue
+		}
+		for i := range f {
+			f[i] = strings.TrimSpace(f[i])
+		}
+		idx, err := strconv.Atoi(f[0])
+		if err != nil {
+			continue
+		}
+		g := GPUSensor{Index: idx, TempC: num(f[1]), PowerW: num(f[2]), PowerLimitW: num(f[3])}
+		if bits, err := strconv.ParseUint(strings.TrimPrefix(f[4], "0x"), 16, 64); err == nil {
+			for _, b := range throttleBits {
+				if bits&b.bit != 0 {
+					g.Throttle = append(g.Throttle, b.name)
+				}
+			}
+		}
+		if len(f) > 5 {
+			g.MemoryTempC = num(f[5])
+		}
+		gs = append(gs, g)
+	}
+	return gs
+}
+
+// num reads a figure nvidia-smi may give as "[N/A]" or "N/A".
+func num(s string) float64 {
+	v, err := strconv.ParseFloat(s, 64)
+	if err != nil || v < 0 {
+		return 0
+	}
+	return v
+}
+
+func readTrim(path string) string {
+	b, err := os.ReadFile(path) //nolint:gosec // sysfs
+	if err != nil {
+		return ""
+	}
+	return strings.TrimSpace(string(b))
+}
@@ -0,0 +1,169 @@
+package discovery
+
+import (
+	"context"
+	"errors"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+)
+
+func put(t *testing.T, root, rel, body string) {
+	t.Helper()
+	p := filepath.Join(root, rel)
+	if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(p, []byte(body+"\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+}
+
+// dellHost is the layout of the Proxmox host these tests were written
+// against: Dell's SMM with a CPU and SODIMM sensors, coretemp with a package
+// and cores, and RAPL with a package and its dram subdomain.
+func dellHost(t *testing.T) string {
+	root := t.TempDir()
+	put(t, root, "class/hwmon/hwmon0/name", "dell_smm")
+	put(t, root, "class/hwmon/hwmon0/temp1_input", "55000")
+	put(t, root, "class/hwmon/hwmon0/temp1_label", "CPU")
+	put(t, root, "class/hwmon/hwmon0/temp2_input", "47000")
+	put(t, root, "class/hwmon/hwmon0/temp2_label", "SODIMM")
+	put(t, root, "class/hwmon/hwmon1/name", "coretemp")
+	put(t, root, "class/hwmon/hwmon1/temp1_input", "58000")
+	put(t, root, "class/hwmon/hwmon1/temp1_label", "Package id 0")
+	put(t, root, "class/hwmon/hwmon1/temp10_input", "61000")
+	put(t, root, "class/hwmon/hwmon1/temp10_label", "Core 8")
+	put(t, root, "class/powercap/intel-rapl:0/energy_uj", "1000000000")
+	put(t, root, "class/powercap/intel-rapl:0/max_energy_range_uj", "262143328850")
+	put(t, root, "class/powercap/intel-rapl:0:0/energy_uj", "5000000")
+	return root
+}
+
+func sampler(sys string, smi func(context.Context, string) ([]byte, error), clock *time.Time) *SensorSampler {
+	return &SensorSampler{sys: sys, smi: smi, now: func() time.Time { return *clock }}
+}
+
+func noSMI(context.Context, string) ([]byte, error) { return nil, errors.New("not found") }
+
+func TestAHostsCPUHeatAndPowerAreReadAsLmSensorsWould(t *testing.T) {
+	root := dellHost(t)
+	now := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)
+	s := sampler(root, noSMI, &now)
+
+	first := s.Sample(t.Context())
+	// The package reading, not the hotter core: the package is what the
+	// chip itself reports as its temperature.
+	if first.CPUTempC != 58 {
+		t.Errorf("CPU %.1f °C, expected the package's 58", first.CPUTempC)
+	}
+	if first.CPUPowerW != 0 {
+		t.Error("power from one reading of a counter is invented")
+	}
+	if first.InletTempC != 0 || first.GPUs != nil {
+		t.Errorf("sensors this host does not have were reported: %+v", first)
+	}
+
+	// 10 s later the package has used 650 J, and its dram subdomain some of
+	// that — counted once.
+	put(t, root, "class/powercap/intel-rapl:0/energy_uj", "1650000000")
+	put(t, root, "class/powercap/intel-rapl:0:0/energy_uj", "99000000")
+	now = now.Add(10 * time.Second)
+	if got := s.Sample(t.Context()).CPUPowerW; got < 64.9 || got > 65.1 {
+		t.Errorf("CPU power %.2f W, expected 65", got)
+	}
+}
+
+func TestACounterThatWrapsIsNotANegativeDraw(t *testing.T) {
+	root := dellHost(t)
+	now := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)
+	put(t, root, "class/powercap/intel-rapl:0/energy_uj", "262143000000")
+	s := sampler(root, noSMI, &now)
+	s.Sample(t.Context())
+	// 328,850 µJ to the wrap, then 499,671,150 past it: 500 J in 10 s.
+	put(t, root, "class/powercap/intel-rapl:0/energy_uj", "499671150")
+	now = now.Add(10 * time.Second)
+	if got := s.Sample(t.Context()).CPUPowerW; got < 49.9 || got > 50.1 {
+		t.Errorf("across a wrap: %.2f W, expected 50", got)
+	}
+}
+
+func TestAVMHasNoSensorsAndSaysNothing(t *testing.T) {
+	root := t.TempDir()
+	put(t, root, "class/thermal/cooling_device0/type", "Processor")
+	now := time.Now()
+	got := sampler(root, noSMI, &now).Sample(t.Context())
+	if got.CPUTempC != 0 || got.CPUPowerW != 0 || got.InletTempC != 0 || got.GPUs != nil {
+		t.Errorf("a machine with no sensors reported %+v", got)
+	}
+}
+
+func TestABoardsInletSensorIsTheRoomNotTheCPU(t *testing.T) {
+	root := t.TempDir()
+	put(t, root, "class/hwmon/hwmon2/name", "nct6798")
+	put(t, root, "class/hwmon/hwmon2/temp1_input", "24500")
+	put(t, root, "class/hwmon/hwmon2/temp1_label", "SYSTIN")
+	put(t, root, "class/hwmon/hwmon3/name", "k10temp")
+	put(t, root, "class/hwmon/hwmon3/temp1_input", "71250")
+	put(t, root, "class/hwmon/hwmon3/temp1_label", "Tctl")
+	now := time.Now()
+	got := sampler(root, noSMI, &now).Sample(t.Context())
+	if got.InletTempC != 24.5 || got.CPUTempC != 71.25 {
+		t.Errorf("inlet %.2f, CPU %.2f", got.InletTempC, got.CPUTempC)
+	}
+}
+
+func TestEachGPUSaysHowHotHowHungryAndWhatHoldsItBack(t *testing.T) {
+	var asked []string
+	smi := func(_ context.Context, fields string) ([]byte, error) {
+		asked = append(asked, fields)
+		return []byte(
+			"0, 71, 612.45, 700.00, 0x0000000000000000, 84\n" +
+				"1, 88, 698.10, 700.00, 0x0000000000000024, 95\n" + // sw_power_cap + sw_thermal
+				"2, 45, [N/A], 700.00, 0x0000000000000001, [N/A]\n"), nil // idle only
+	}
+	now := time.Now()
+	ss := sampler(t.TempDir(), smi, &now)
+	got := ss.Sample(t.Context())
+	if len(got.GPUs) != 3 {
+		t.Fatalf("%+v", got.GPUs)
+	}
+	g0, g1, g2 := got.GPUs[0], got.GPUs[1], got.GPUs[2]
+	if g0.TempC != 71 || g0.PowerW != 612.45 || g0.PowerLimitW != 700 || g0.MemoryTempC != 84 || g0.Throttle != nil {
+		t.Errorf("card 0 %+v", g0)
+	}
+	if strings.Join(g1.Throttle, ",") != "sw_power_cap,sw_thermal" {
+		t.Errorf("card 1 throttled by %v", g1.Throttle)
+	}
+	if g2.PowerW != 0 || g2.MemoryTempC != 0 || g2.Throttle != nil {
+		t.Errorf("card 2: N/A is not a reading, and idle is not being held back: %+v", g2)
+	}
+	// Within ten seconds the last reading is reused rather than running
+	// nvidia-smi on every heartbeat; after, it is asked again.
+	now = now.Add(5 * time.Second)
+	ss.Sample(t.Context())
+	if len(asked) != 1 {
+		t.Errorf("nvidia-smi was asked %d times within ten seconds", len(asked))
+	}
+	now = now.Add(6 * time.Second)
+	ss.Sample(t.Context())
+	if len(asked) != 2 {
+		t.Errorf("nvidia-smi was asked %d times after eleven seconds, expected 2", len(asked))
+	}
+}
+
+func TestADriverThatDoesNotKnowMemoryTemperatureStillReportsTheRest(t *testing.T) {
+	smi := func(_ context.Context, fields string) ([]byte, error) {
+		if strings.Contains(fields, "temperature.memory") {
+			return nil, errors.New(`Field "temperature.memory" is not a valid field to query.`)
+		}
+		return []byte("0, 66, 250.0, 300.0, 0x0000000000000040\n"), nil
+	}
+	now := time.Now()
+	got := sampler(t.TempDir(), smi, &now).Sample(t.Context())
+	if len(got.GPUs) != 1 || got.GPUs[0].TempC != 66 || strings.Join(got.GPUs[0].Throttle, ",") != "hw_thermal" {
+		t.Errorf("%+v", got.GPUs)
+	}
+}
@@ -40,6 +40,9 @@ type Profile struct {
 	ModelPath string
 	// ServedModelName is the name clients use, which need not match the path.
 	ServedModelName string
+	// Adapters are LoRA adapters to load beside the model, in order: an
+	// adapter's index is the id requests apply it by.
+	Adapters []Adapter
 }

 // ShardSpec is one shard of one replica on this node.
@@ -76,6 +79,10 @@ type ShardSpec struct {
 	// and accounting. Given to the engine in its environment rather than on
 	// its command line, which every user on the node can read.
 	APIKey string `json:"api_key,omitempty"`
+	// Threads is the CPU cores placement reserved for a shard with no GPU,
+	// and so what its engine computes on. Zero leaves it to the engine's own
+	// configuration.
+	Threads int `json:"threads,omitempty"`

 	// Pool and Fabric are control plane concepts the agent cannot derive. They
 	// are carried so the agent can report them back after a restart; Fabric
@@ -369,3 +376,10 @@ func quantizationOf(args []string) string {
 	}
 	return ""
 }
+
+// Adapter is a LoRA adapter an engine loads beside its model. Path is a
+// source until the agent has fetched it, a local file after.
+type Adapter struct {
+	Name string `json:"name"`
+	Path string `json:"path"`
+}
@@ -133,12 +133,26 @@ func (l *LlamaCpp) Commands(s ShardSpec) ([]Command, error) {
 		args = append(args, "--n-gpu-layers", "999")
 	}
 	if len(s.GPUIndices) == 0 && !hasFlag(p.Args, "--threads", "-t") {
-		n := l.cfg.Threads
+		// What placement reserved first: it is what the node's other
+		// replicas were placed around.
+		n := s.Threads
+		if n <= 0 {
+			n = l.cfg.Threads
+		}
 		if n <= 0 {
 			n = max(numCPU()/2, 1)
 		}
 		args = append(args, "--threads", strconv.Itoa(n))
 	}
+	// Loaded at start, applied per request: a request that names an adapter
+	// carries "lora": [{"id": i, "scale": s}], and one that names none is the
+	// base model. Each is loaded at scale zero, not with --lora and
+	// --lora-init-without-apply: that leaves an adapter's scale at 1, and
+	// llama-server applies it at that scale to every request with no "lora"
+	// of its own — every request for the model itself.
+	for _, a := range p.Adapters {
+		args = append(args, "--lora-scaled", a.Path+":0")
+	}
 	args = append(args, p.Args...)

 	env := baseEnv(s)
@@ -294,3 +294,32 @@ func TestACPUShardLeavesHalfTheMachineForAnother(t *testing.T) {
 		t.Errorf("--threads %q on one CPU", v)
 	}
 }
+
+func TestTheCoresPlacementReservedAreTheThreads(t *testing.T) {
+	s := llamaSpec(nil)
+	s.Threads = 3
+	cmds, err := (&LlamaCpp{cfg: LlamaCppConfig{Threads: 8}}).Commands(s)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if v, _ := flagValue(cmds[0].Args, "--threads"); v != "3" {
+		t.Errorf("--threads %q; what placement reserved should win over the node's setting", v)
+	}
+}
+
+func TestAdaptersAreLoadedAtStartAndAppliedToNoneByDefault(t *testing.T) {
+	s := llamaSpec(nil)
+	s.Profile.Adapters = []Adapter{{Name: "sql", Path: "/w/sql.gguf"}, {Name: "docs", Path: "/w/docs.gguf"}}
+	cmds, err := (&LlamaCpp{}).Commands(s)
+	if err != nil {
+		t.Fatal(err)
+	}
+	args := strings.Join(cmds[0].Args, " ")
+	if !strings.Contains(args, "--lora-scaled /w/sql.gguf:0 --lora-scaled /w/docs.gguf:0") {
+		t.Errorf("args %s: both adapters, in order, loaded at scale zero", args)
+	}
+	plain, _ := (&LlamaCpp{}).Commands(llamaSpec(nil))
+	if strings.Contains(strings.Join(plain[0].Args, " "), "--lora") {
+		t.Error("a model with no adapters was given lora flags")
+	}
+}
@@ -137,7 +137,7 @@ func newCtlCmd() *cobra.Command {
 		// apply to it; it lives here because this is where operators look.
 		newBenchCmd(client, &asJSON),
 		newStatusCmd(client, &asJSON),
-		newModelCmd(client),
+		newModelCmd(client, &asJSON),
 		newScaleCmd(client),
 		newRestartCmd(client),
 		newRetryCmd(client),
@@ -146,6 +146,7 @@ func newCtlCmd() *cobra.Command {
 		newToolCheckCmd(),
 		newWeightsCmd(client, &asJSON),
 		newCapacityCmd(client, &asJSON),
+		newQuotaCmd(client, &asJSON),
 		newUsageCmd(client, &asJSON),
 		newCertCmd(),
 	)
@@ -253,6 +254,10 @@ func modelStatus(m adminapi.ModelView) string {
 	switch {
 	case m.Parked:
 		return "parked"
+	case m.External != "":
+		return "external: " + m.External
+	case m.HeldFor != "":
+		return "gave room to " + m.HeldFor
 	case m.Unplaceable:
 		return "cannot place"
 	case m.Min == 0 && m.Total == 0:
@@ -323,17 +328,27 @@ func newModelsCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
 			}

 			w := tab()
-			fmt.Fprintln(w, "MODEL\tSERVING\tREPLICAS\tSTATUS\tALIASES\tPROFILES\tFALLBACKS")
+			fmt.Fprintln(w, "MODEL\tSERVING\tREPLICAS\tSTATUS\tCLASS\tRESERVES\tALIASES\tPROFILES\tFALLBACKS")
+			var warnings []string
 			for _, m := range models {
 				serving := "no"
 				if m.Ready > 0 {
 					serving = "yes"
 				}
-				fmt.Fprintf(w, "%s\t%s\t%d/%d\t%s\t%s\t%s\t%s\n",
-					m.Name, serving, m.Ready, m.Min, modelStatus(m),
+				fmt.Fprintf(w, "%s\t%s\t%d/%d\t%s\t%s\t%s\t%s\t%s\t%s\n",
+					m.Name, serving, m.Ready, m.Min, modelStatus(m), m.Class, reserves(m),
 					join(m.Aliases), join(m.Profiles), join(m.Fallbacks))
+				for _, x := range m.Warnings {
+					warnings = append(warnings, m.Name+": "+x)
+				}
 			}
-			return w.Flush()
+			if err := w.Flush(); err != nil {
+				return err
+			}
+			for _, x := range warnings {
+				fmt.Println("warning:", x)
+			}
+			return nil
 		},
 	}
 }
@@ -511,3 +526,22 @@ func dash(s string) string {
 	}
 	return s
 }
+
+// reserves is what one replica of a model holds of a node, with what it was
+// measured using beside it: "6G 2c (5.1G used)".
+func reserves(m adminapi.ModelView) string {
+	if m.MemoryGB == 0 && m.CPUCores == 0 {
+		return "-"
+	}
+	out := fmt.Sprintf("%dG", m.MemoryGB)
+	if m.CPUCores > 0 {
+		out += fmt.Sprintf(" %dc", m.CPUCores)
+	}
+	if m.MeasuredMemoryGB > 0 {
+		out += fmt.Sprintf(" (%.1fG used)", m.MeasuredMemoryGB)
+	}
+	if len(m.Warnings) > 0 {
+		out += " !"
+	}
+	return out
+}
@@ -172,6 +172,10 @@ func upgradeNodes(ctx context.Context, out io.Writer, client func() *ctlClient,
 		}
 	}
 	fmt.Fprintf(out, "%d node(s) upgraded to %s, %d already on it\n", done, v, skipped)
+	// The one machine this does not reach. Said every time, because it is
+	// exactly the kind of thing that stays three versions behind unnoticed.
+	fmt.Fprintf(out, "The model mirror's host is not upgraded by this; on it, run:\n"+
+		"  sudo llmgw upgrade %s --service llmgw-mirror\n", v)
 	return nil
 }

@@ -5,6 +5,9 @@ import (
 	"fmt"
 	"os"
 	"os/user"
+	"slices"
+	"strconv"
+	"strings"
 	"time"

 	"github.com/knadh/koanf/parsers/yaml"
@@ -24,12 +27,14 @@ import (
 // minutes of outage. For a fixed production set that is a reasonable trade; for
 // a platform whose promise is that people can try things, it is the difference
 // between trying one and not bothering.
-func newModelCmd(client func() *ctlClient) *cobra.Command {
+func newModelCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
 	c := &cobra.Command{
 		Use:   "model",
 		Short: "Add or remove models while the cluster is running",
 	}
 	c.AddCommand(newModelAddCmd(client), newModelRemoveCmd(client), newModelPlanCmd(client),
+		newResourcesCmd(client), newClassCmd(client), newOwnerCmd(client),
+		newTargetCmd(client, asJSON), newAdaptersCmd(client),
 		newModelServeCmd(client))
 	return c
 }
@@ -192,3 +197,139 @@ func newScaleCmd(client func() *ctlClient) *cobra.Command {
 		"how long an on-demand model stays up unasked-for; zero takes the cluster default")
 	return c
 }
+
+func newResourcesCmd(client func() *ctlClient) *cobra.Command {
+	var memory, cores int
+	var profile string
+	c := &cobra.Command{
+		Use:   "resources <model> --memory GB [--cores N]",
+		Short: "Set what a model reserves of a node: memory, and CPU cores",
+		Long: "--memory is the host RAM a replica needs: for a model with no GPU, its\n" +
+			"weights, KV cache and engine together. Placement reserves it on the node,\n" +
+			"so a node holds only as many replicas as it has memory for.\n\n" +
+			"--cores is the CPU cores a replica with no GPU computes on. They are\n" +
+			"reserved the same way, and become the engine's thread count, so two\n" +
+			"replicas on one node stop competing for the same cores. Zero reserves\n" +
+			"none and leaves the thread count to the node.\n\n" +
+			"Running replicas keep what they were started with; `llmgw ctl restart\n" +
+			"--model M` moves them to the new figures.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if !cmd.Flags().Changed("memory") {
+				return fmt.Errorf("--memory is required")
+			}
+			body, err := json.Marshal(adminapi.ResourcesRequest{Profile: profile, MemoryGB: memory, CPUCores: cores})
+			if err != nil {
+				return err
+			}
+			var m catalog.Model
+			if err := client().post(cmd.Context(), "/api/v1/models/"+args[0]+"/resources", body, &m); err != nil {
+				return err
+			}
+			for _, p := range m.Profiles {
+				fmt.Fprintf(cmd.OutOrStdout(), "%s/%s: %d GB memory, %d cores\n", m.Name, p.Name, p.MinMemoryGB, p.CPUCores)
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "Running replicas keep their old figures until restarted: llmgw ctl restart --model %s\n", m.Name)
+			return nil
+		},
+	}
+	c.Flags().IntVar(&memory, "memory", 0, "host RAM a replica needs, in GB")
+	c.Flags().IntVar(&cores, "cores", 0, "CPU cores a replica with no GPU computes on; 0 reserves none")
+	c.Flags().StringVar(&profile, "profile", "", "one profile; every profile when left out")
+	return c
+}
+
+func newClassCmd(client func() *ctlClient) *cobra.Command {
+	return &cobra.Command{
+		Use:   "class <model> critical|standard|opportunistic|default",
+		Short: "Set how much a model's capacity is worth against another's",
+		Long: "When a model cannot be placed, replicas of a lower class are stopped to\n" +
+			"make room for it:\n\n" +
+			"  critical       never gives up the replicas of its minimum\n" +
+			"  standard       gives way to critical\n" +
+			"  opportunistic  gives way to both; the default for an on-demand model\n\n" +
+			"Within a class, a model below its minimum may take what another grew into\n" +
+			"under load, but not its minimum. \"default\" goes back to what the model's\n" +
+			"scaling implies: opportunistic when on demand, standard otherwise.",
+		Args: cobra.ExactArgs(2),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			class := args[1]
+			if class == "default" {
+				class = ""
+			}
+			body, _ := json.Marshal(map[string]string{"class": class})
+			var res map[string]string
+			if err := client().post(cmd.Context(), "/api/v1/models/"+args[0]+"/class", body, &res); err != nil {
+				return err
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "%s is %s\n", res["model"], res["class"])
+			return nil
+		},
+	}
+}
+
+func newAdaptersCmd(client func() *ctlClient) *cobra.Command {
+	var add, remove []string
+	c := &cobra.Command{
+		Use:   "adapters <model> [--add NAME=URL[@SCALE]] [--remove NAME]",
+		Short: "List, add or remove a model's LoRA adapters",
+		Long: "An adapter is asked for by its NAME as though it were a model, and is served\n" +
+			"by this model's replicas with it applied. URL is the adapter file, usually in\n" +
+			"the mirror; it must be a GGUF LoRA adapter for this model's architecture,\n" +
+			"which is checked before it is attached. Replicas load a changed set when\n" +
+			"restarted: llmgw ctl restart --model MODEL.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			name := args[0]
+			var models []adminapi.ModelView
+			if err := client().get(cmd.Context(), "/api/v1/models", &models); err != nil {
+				return err
+			}
+			var current []catalog.Adapter
+			found := false
+			for _, m := range models {
+				if m.Name == name {
+					current, found = m.Adapters, true
+				}
+			}
+			if !found {
+				return fmt.Errorf("no model %q", name)
+			}
+			out := cmd.OutOrStdout()
+			if len(add) == 0 && len(remove) == 0 {
+				if len(current) == 0 {
+					fmt.Fprintf(out, "%s has no adapters\n", name)
+				}
+				for _, a := range current {
+					fmt.Fprintf(out, "%s\tscale %g\t%s\n", a.Name, a.EffectiveScale(), a.Source)
+				}
+				return nil
+			}
+			next := slices.DeleteFunc(slices.Clone(current), func(a catalog.Adapter) bool { return slices.Contains(remove, a.Name) })
+			for _, spec := range add {
+				n, rest, ok := strings.Cut(spec, "=")
+				if !ok || n == "" || rest == "" {
+					return fmt.Errorf("--add %q is not NAME=URL[@SCALE]", spec)
+				}
+				ad := catalog.Adapter{Name: n, Source: rest}
+				if i := strings.LastIndex(rest, "@"); i > 0 {
+					if s, err := strconv.ParseFloat(rest[i+1:], 64); err == nil {
+						ad.Source, ad.Scale = rest[:i], s
+					}
+				}
+				next = append(next, ad)
+			}
+			body, _ := json.Marshal(adminapi.AdaptersRequest{Adapters: next})
+			var res adminapi.AdaptersRequest
+			if err := client().post(cmd.Context(), "/api/v1/models/"+name+"/adapters", body, &res); err != nil {
+				return err
+			}
+			fmt.Fprintf(out, "%s serves %d adapter(s); restart its replicas to load them: llmgw ctl restart --model %s\n",
+				name, len(res.Adapters), name)
+			return nil
+		},
+	}
+	c.Flags().StringArrayVar(&add, "add", nil, "NAME=URL[@SCALE]; repeatable")
+	c.Flags().StringArrayVar(&remove, "remove", nil, "an adapter's name; repeatable")
+	return c
+}
@@ -75,6 +75,7 @@ func newModelServeCmd(client func() *ctlClient) *cobra.Command {
 	f.IntVar(&req.Replicas, "replicas", 1, "how many replicas to run")
 	f.BoolVar(&onDemand, "on-demand", false, "run none until first asked for")
 	f.IntVar(&req.CtxPool, "ctx-pool", 16384, "llama.cpp's --ctx-size, shared by its parallel slots")
+	f.IntVar(&req.CPUCores, "cores", 0, "CPU cores a replica computes on; 0 takes half the smallest node it fits")
 	f.IntVar(&req.ContextLength, "context-length", 4096, "what one request may use")
 	return c
 }
@@ -0,0 +1,234 @@
+package cli
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/url"
+	"strconv"
+	"strings"
+
+	"github.com/spf13/cobra"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+func newQuotaCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
+	c := &cobra.Command{
+		Use:   "quota",
+		Short: "What parts of the tenancy tree are guaranteed of the cluster, and may hold",
+		Long: "A capacity quota belongs to an organization, team or project, and counts the\n" +
+			"GPUs, CPU cores and memory held by replicas of the models it owns (see\n" +
+			"`llmgw ctl model owner`), and of those its teams and projects own.\n\n" +
+			"The guarantee is kept for it: when it holds less and a model of its needs\n" +
+			"room, replicas other scopes hold beyond their own guarantees are stopped to\n" +
+			"make it — whatever their class. What it holds within its guarantee is\n" +
+			"never taken by another scope's model. Beyond it, it borrows: free room is\n" +
+			"used, and given back when asked for.\n\n" +
+			"The limit is the most it may hold, borrowing included; a model that would\n" +
+			"take it past is not given another replica.\n\n" +
+			"Scopes are written org:Acme, team:Acme/ml or project:Acme/ml/training, or\n" +
+			"with IDs. Changes take effect within half a minute.",
+	}
+	c.AddCommand(newQuotaListCmd(client, asJSON), newQuotaSetCmd(client), newQuotaClearCmd(client))
+	return c
+}
+
+func newQuotaListCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
+	return &cobra.Command{
+		Use:   "list",
+		Short: "Every quota, with what its scope holds now",
+		RunE: func(cmd *cobra.Command, _ []string) error {
+			var qs []adminapi.QuotaView
+			if err := client().get(cmd.Context(), "/api/v1/quotas", &qs); err != nil {
+				return err
+			}
+			if *asJSON {
+				return json.NewEncoder(cmd.OutOrStdout()).Encode(qs)
+			}
+			if len(qs) == 0 {
+				fmt.Fprintln(cmd.OutOrStdout(), "No capacity quotas: everything shares the cluster by class alone.")
+				return nil
+			}
+			w := tab()
+			fmt.Fprintln(w, "SCOPE\tNAME\tGPUS\tCORES\tMEMORY GB")
+			for _, q := range qs {
+				fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", q.Scope, q.Name,
+					quotaCell(q.Holds.GPUs, q.Guarantee.GPUs, q.Limit.GPUs),
+					quotaCell(q.Holds.CPUCores, q.Guarantee.CPUCores, q.Limit.CPUCores),
+					quotaCell(q.Holds.MemoryGB, q.Guarantee.MemoryGB, q.Limit.MemoryGB))
+			}
+			if err := w.Flush(); err != nil {
+				return err
+			}
+			fmt.Fprintln(cmd.OutOrStdout(), "\nEach cell: holds / guaranteed (at most limit). - is not governed.")
+			return nil
+		},
+	}
+}
+
+// quotaCell is "6 / 8 (≤ 16)": what is held, what is guaranteed, the limit.
+func quotaCell(holds int, guarantee, limit *int) string {
+	if guarantee == nil && limit == nil {
+		return "-"
+	}
+	out := strconv.Itoa(holds) + " / "
+	if guarantee != nil {
+		out += strconv.Itoa(*guarantee)
+	} else {
+		out += "-"
+	}
+	if limit != nil {
+		out += fmt.Sprintf(" (≤ %d)", *limit)
+	}
+	return out
+}
+
+func newQuotaSetCmd(client func() *ctlClient) *cobra.Command {
+	var gpus, cores, mem, lgpus, lcores, lmem int
+	c := &cobra.Command{
+		Use:   "set SCOPE [--gpus N] [--cores N] [--memory GB] [--limit-gpus N] ...",
+		Short: "Set a scope's guarantee and limit, replacing what it had",
+		Args:  cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			f := cmd.Flags()
+			pick := func(name string, v int) *int {
+				if !f.Changed(name) {
+					return nil
+				}
+				return &v
+			}
+			req := adminapi.QuotaRequest{Scope: args[0],
+				Guarantee: store.Resources{GPUs: pick("gpus", gpus), CPUCores: pick("cores", cores), MemoryGB: pick("memory", mem)},
+				Limit:     store.Resources{GPUs: pick("limit-gpus", lgpus), CPUCores: pick("limit-cores", lcores), MemoryGB: pick("limit-memory", lmem)},
+			}
+			body, _ := json.Marshal(req)
+			var res map[string]string
+			if err := client().put(cmd.Context(), "/api/v1/quotas", body, &res); err != nil {
+				return err
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "%s: set; it takes effect within half a minute\n", res["scope"])
+			return nil
+		},
+	}
+	f := c.Flags()
+	f.IntVar(&gpus, "gpus", 0, "GPUs guaranteed")
+	f.IntVar(&cores, "cores", 0, "CPU cores guaranteed")
+	f.IntVar(&mem, "memory", 0, "memory guaranteed, in GB")
+	f.IntVar(&lgpus, "limit-gpus", 0, "the most GPUs it may hold")
+	f.IntVar(&lcores, "limit-cores", 0, "the most CPU cores it may hold")
+	f.IntVar(&lmem, "limit-memory", 0, "the most memory it may hold, in GB")
+	return c
+}
+
+func newQuotaClearCmd(client func() *ctlClient) *cobra.Command {
+	return &cobra.Command{
+		Use:   "clear SCOPE",
+		Short: "Remove a scope's quota",
+		Args:  cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			var res map[string]string
+			if err := client().delete(cmd.Context(), "/api/v1/quotas/"+url.PathEscape(args[0]), &res); err != nil {
+				return err
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "%s: quota removed\n", res["removed"])
+			return nil
+		},
+	}
+}
+
+func newOwnerCmd(client func() *ctlClient) *cobra.Command {
+	return &cobra.Command{
+		Use:   "owner <model> SCOPE|shared",
+		Short: "Set whose capacity quota a model counts against",
+		Long: "SCOPE is org:Acme, team:Acme/ml or project:Acme/ml/training. A shared\n" +
+			"model belongs to the cluster and counts against nobody's quota.",
+		Args: cobra.ExactArgs(2),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			owner := args[1]
+			if owner == "shared" {
+				owner = ""
+			}
+			body, _ := json.Marshal(map[string]string{"owner": owner})
+			var res map[string]string
+			if err := client().post(cmd.Context(), "/api/v1/models/"+args[0]+"/owner", body, &res); err != nil {
+				return err
+			}
+			if res["owner"] == "" {
+				fmt.Fprintf(cmd.OutOrStdout(), "%s is shared\n", res["model"])
+			} else {
+				fmt.Fprintf(cmd.OutOrStdout(), "%s counts against %s\n", res["model"], res["owner"])
+			}
+			return nil
+		},
+	}
+}
+
+func newTargetCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
+	var tps float64
+	var apply bool
+	c := &cobra.Command{
+		Use:   "target <model> --tokens-per-sec N [--apply]",
+		Short: "How many replicas a model needs for a throughput, and whether they fit",
+		Long: "Sized from what one replica was measured to serve while meeting its SLO\n" +
+			"(`llmgw ctl bench MODEL --save`), on the profile its replicas run on.\n" +
+			"--apply makes that the number it always runs, if the cluster has room.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if tps <= 0 {
+				return fmt.Errorf("--tokens-per-sec is required")
+			}
+			var v adminapi.TargetView
+			path := "/api/v1/models/" + url.PathEscape(args[0]) + "/target?tokens_per_sec=" +
+				strconv.FormatFloat(tps, 'f', -1, 64)
+			if err := client().get(cmd.Context(), path, &v); err != nil {
+				return err
+			}
+			if *asJSON {
+				return json.NewEncoder(cmd.OutOrStdout()).Encode(v)
+			}
+			out := cmd.OutOrStdout()
+			fmt.Fprintf(out, "%s: %.0f tokens/s needs %d replicas at %.1f each (measured on %s); %d running\n",
+				v.Model, v.TokPerSec, v.Needed, v.PerReplica, v.Profile, v.Current)
+			switch {
+			case v.More == 0:
+				fmt.Fprintln(out, "It already has them.")
+			case v.Fits && len(v.Preempts) > 0:
+				fmt.Fprintf(out, "%d more; there is room once replicas of %s are stopped, which its class may do.\n",
+					v.More, strings.Join(v.Preempts, ", "))
+			case v.Fits:
+				fmt.Fprintf(out, "%d more, and there is room for them.\n", v.More)
+			default:
+				fmt.Fprintf(out, "%d more, and there is room for %d:\n", v.More, v.Room)
+				for _, b := range v.Blocked {
+					fmt.Fprintln(out, "  "+b)
+				}
+			}
+			if !apply || v.More == 0 {
+				return nil
+			}
+			if !v.Fits {
+				return fmt.Errorf("not applied: the cluster has no room for them")
+			}
+			var models []adminapi.ModelView
+			if err := client().get(cmd.Context(), "/api/v1/models", &models); err != nil {
+				return err
+			}
+			top := v.Needed
+			for _, m := range models {
+				if m.Name == v.Model {
+					top = max(top, m.Max)
+				}
+			}
+			body, _ := json.Marshal(adminapi.ScaleRequest{Min: v.Needed, Max: top})
+			if err := client().post(cmd.Context(), "/api/v1/models/"+url.PathEscape(v.Model)+"/scale", body, nil); err != nil {
+				return err
+			}
+			fmt.Fprintf(out, "Applied: %s runs %d (up to %d under load).\n", v.Model, v.Needed, top)
+			return nil
+		},
+	}
+	c.Flags().Float64Var(&tps, "tokens-per-sec", 0, "the throughput to size for")
+	c.Flags().BoolVar(&apply, "apply", false, "set it as the number of replicas the model always runs")
+	return c
+}
@@ -0,0 +1,67 @@
+package cli
+
+import (
+	"errors"
+	"fmt"
+	"os"
+
+	"github.com/spf13/cobra"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/release"
+)
+
+func newReleaseCmd() *cobra.Command {
+	c := &cobra.Command{
+		Use:   "release",
+		Short: "Sign releases, and make the key they are signed with",
+	}
+	c.AddCommand(&cobra.Command{
+		Use:   "keygen",
+		Short: "Make a release signing key pair",
+		Long: "Prints a public key and a private key. The private key goes into the CI\n" +
+			"secret RELEASE_SIGNING_KEY and nowhere else; the public key into\n" +
+			"internal/release/trusted.pub, which every build from then on carries\n" +
+			"and checks releases against. Run once, on a machine you trust.",
+		Args: cobra.NoArgs,
+		RunE: func(cmd *cobra.Command, _ []string) error {
+			pub, priv, err := release.GenerateKey()
+			if err != nil {
+				return err
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "public:  %s\nprivate: %s\n", pub, priv)
+			fmt.Fprintln(cmd.ErrOrStderr(), "\nThe private key signs every release: put it in the CI secret "+
+				"RELEASE_SIGNING_KEY, then clear this terminal's scrollback.")
+			return nil
+		},
+	})
+	c.AddCommand(&cobra.Command{
+		Use:   "sign FILE",
+		Short: "Sign a release's checksums, writing FILE.sig",
+		Long: "Reads the private key from RELEASE_SIGNING_KEY, never from an argument,\n" +
+			"so it does not land in a shell history or a process list.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			key := os.Getenv("RELEASE_SIGNING_KEY")
+			if key == "" {
+				return errors.New("RELEASE_SIGNING_KEY is not set")
+			}
+			data, err := os.ReadFile(args[0])
+			if err != nil {
+				return err
+			}
+			sig, err := release.Sign(key, data)
+			if err != nil {
+				return err
+			}
+			if pub := release.TrustedKey(); pub != "" {
+				// Signed with a key this build would not accept is a release
+				// every upgraded machine refuses: stopped here instead.
+				if err := release.Verify(pub, data, sig); err != nil {
+					return fmt.Errorf("the signing key does not match this build's trusted key: %w", err)
+				}
+			}
+			return os.WriteFile(args[0]+".sig", []byte(sig+"\n"), 0o644) //nolint:gosec // public
+		},
+	})
+	return c
+}
@@ -47,6 +47,8 @@ func Root() *cobra.Command {
 		newSimEngineCmd(),
 		newDoctorCmd(),
 		newNodeCmd(),
+		newSelfUpgradeCmd(),
+		newReleaseCmd(),
 	)
 	return root
 }
@@ -0,0 +1,87 @@
+package cli
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"os/exec"
+	"runtime"
+	"strings"
+	"time"
+
+	"github.com/spf13/cobra"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/release"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/version"
+)
+
+// newSelfUpgradeCmd upgrades this machine's llmgw on its own, for the machines
+// `llmgw ctl upgrade` does not reach: a mirror host, above all, whose service
+// runs unprivileged with the filesystem read-only and so cannot replace its own
+// binary the way an agent does.
+func newSelfUpgradeCmd() *cobra.Command {
+	var releases, service string
+	c := &cobra.Command{
+		Use:   "upgrade VERSION [--service UNIT]",
+		Short: "Upgrade this machine's llmgw, and restart the service that runs it",
+		Long: "For a machine the control plane does not upgrade — the model mirror's host.\n" +
+			"The control plane and the nodes are upgraded with `llmgw ctl upgrade`.\n\n" +
+			"Fetches the release for this platform, checks it against the release's\n" +
+			"checksums, keeps the old binary beside the new one, and restarts --service,\n" +
+			"waiting for it to be running again. Run it with sudo.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			v := args[0]
+			out := cmd.OutOrStdout()
+			if !release.Tag.MatchString(v) {
+				return fmt.Errorf("%q is not a release version (v1.2.3 or v1.2.3-rc.4)", v)
+			}
+			if version.Version == v {
+				fmt.Fprintf(out, "this machine already runs %s\n", v)
+				return nil
+			}
+			if os.Geteuid() != 0 {
+				return errors.New("replacing this machine's binary needs root: run it with sudo")
+			}
+			self, err := os.Executable()
+			if err != nil {
+				return err
+			}
+			backup, err := release.Install(cmd.Context(), nil, releases, v, runtime.GOOS, runtime.GOARCH, self)
+			if err != nil {
+				return err
+			}
+			fmt.Fprintf(out, "%s -> %s: installed, checksum verified; the old binary is %s\n",
+				version.Version, v, backup)
+			if service == "" {
+				return nil
+			}
+			//nolint:gosec // the unit name is the operator's own flag
+			if b, err := exec.CommandContext(cmd.Context(), "systemctl", "restart", service).CombinedOutput(); err != nil {
+				return fmt.Errorf("restarting %s: %v: %s", service, err, strings.TrimSpace(string(b)))
+			}
+			// Running, and still running a few seconds on: a unit that starts
+			// and dies on a bad flag is "active" for a moment first.
+			deadline := time.Now().Add(30 * time.Second)
+			steady := 0
+			for time.Now().Before(deadline) {
+				//nolint:gosec // as above
+				b, _ := exec.CommandContext(cmd.Context(), "systemctl", "is-active", service).Output()
+				if strings.TrimSpace(string(b)) == "active" {
+					if steady++; steady >= 3 {
+						fmt.Fprintf(out, "%s restarted and running\n", service)
+						return nil
+					}
+				} else {
+					steady = 0
+				}
+				time.Sleep(2 * time.Second)
+			}
+			return fmt.Errorf("%s is not running after the upgrade; see journalctl -u %s, and %s holds the previous binary",
+				service, service, backup)
+		},
+	}
+	c.Flags().StringVar(&releases, "releases", release.DefaultBase, "where release binaries are fetched from")
+	c.Flags().StringVar(&service, "service", "", "the systemd unit to restart afterwards, e.g. llmgw-mirror")
+	return c
+}
@@ -226,6 +226,8 @@ func newServerCmd() *cobra.Command {
 					// And what each group of the tenancy tree spent, which is
 					// the other half of a quota being reviewable.
 					api.SetUsage(st)
+					// Capacity quotas, set and read over this API.
+					api.SetQuotas(st)
 				}
 				api.Register(adminMux)

@@ -483,6 +485,12 @@ func startControlPlane(ctx context.Context, cfg config.Server, cat *catalog.Live
 		if st := h.Store(); st != nil {
 			lead = control.NewLeadership(st, cfg.Control.LeaseTTL, log)
 			ctl.SetLeadership(lead)
+			// The tenancy tree's capacity quotas: what each part is
+			// guaranteed and may hold.
+			ctl.Reconciler().SetQuotas(control.QuotaPolicy(st, log))
+			// And what each model was measured to serve, for scaling and for
+			// choosing between profiles.
+			ctl.Reconciler().SetMeasurements(control.Measurements(st, log))
 			log.Info("competing to be the control plane that places work",
 				"owner", lead.Owner())
 		}
@@ -378,7 +378,12 @@ type Admission struct {

 	QueueTTL             time.Duration `koanf:"queue_ttl"`
 	QueueCapacityPerBand int           `koanf:"queue_capacity_per_band"`
-	Interval             time.Duration `koanf:"interval"`
+	// QueueCapacityTotal bounds everything queued across bands. Reached, a
+	// request is not refused while a lower band has any queued: the newest of
+	// the lowest band is turned away instead, with a 503 and Retry-After.
+	// Zero takes half of all the bands' capacities together.
+	QueueCapacityTotal int           `koanf:"queue_capacity_total"`
+	Interval           time.Duration `koanf:"interval"`
 }

 // Prefix configures approximate prefix routing.
@@ -4,6 +4,8 @@ import (
 	"net/http/httptest"
 	"strings"
 	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
 )

 func TestAClaimedActorIsOneShortName(t *testing.T) {
@@ -22,3 +24,22 @@ func TestAClaimedActorIsOneShortName(t *testing.T) {
 		}
 	}
 }
+
+func TestAReservationThatPlacementCannotTrustIsSaid(t *testing.T) {
+	cpu := func(mem int) catalog.Model {
+		return catalog.Model{Name: "m", Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: mem}}}
+	}
+	if w := reservationWarnings(cpu(0), 0); len(w) != 1 || !strings.Contains(w[0], "declares no memory") {
+		t.Errorf("no memory declared: %v", w)
+	}
+	if w := reservationWarnings(cpu(6), 8.2); len(w) != 1 || !strings.Contains(w[0], "uses 8.2 GB and reserves 6") {
+		t.Errorf("under-declared: %v", w)
+	}
+	if w := reservationWarnings(cpu(6), 6.4); len(w) != 0 {
+		t.Errorf("within a tenth is mapped weights, not a wrong figure: %v", w)
+	}
+	gpu := catalog.Model{Name: "g", Profiles: []catalog.Profile{{Name: "tp1", Engine: "vllm", GPUClass: "h100-80g"}}}
+	if w := reservationWarnings(gpu, 3); len(w) != 0 {
+		t.Errorf("a GPU model needs no host memory figure: %v", w)
+	}
+}
@@ -0,0 +1,71 @@
+package adminapi
+
+import (
+	"bytes"
+	"encoding/binary"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+)
+
+// ggufStart is the start of a GGUF file with these string keys.
+func ggufStart(kv ...string) []byte {
+	var b bytes.Buffer
+	w := func(v any) { _ = binary.Write(&b, binary.LittleEndian, v) }
+	str := func(s string) { w(uint64(len(s))); b.WriteString(s) }
+	b.WriteString("GGUF")
+	w(uint32(3))
+	w(uint64(0))
+	w(uint64(len(kv) / 2))
+	for i := 0; i < len(kv); i += 2 {
+		str(kv[i])
+		w(uint32(8)) // string
+		str(kv[i+1])
+	}
+	b.Write(make([]byte, 4096))
+	return b.Bytes()
+}
+
+func TestOnlyAnAdapterForTheModelsArchitectureIsAttached(t *testing.T) {
+	files := map[string][]byte{
+		"/base.gguf":  ggufStart("general.architecture", "qwen2"),
+		"/model.gguf": ggufStart("general.architecture", "qwen2", "general.type", "model"),
+		"/llama.gguf": ggufStart("general.architecture", "llama", "general.type", "adapter"),
+		"/lora.gguf":  ggufStart("general.architecture", "qwen2", "general.type", "adapter"),
+	}
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		w.WriteHeader(http.StatusPartialContent)
+		_, _ = w.Write(files[r.URL.Path])
+	}))
+	defer srv.Close()
+
+	models := []catalog.Model{{
+		Name: "coder", Source: srv.URL + "/base.gguf",
+		Scaling:  catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 8}},
+	}}
+	for _, tc := range []struct {
+		file, want string
+		code       int
+	}{
+		{"/model.gguf", "not a LoRA adapter", http.StatusBadRequest},
+		{"/llama.gguf", "for llama models", http.StatusBadRequest},
+		{"/lora.gguf", "", http.StatusOK},
+	} {
+		a, _ := setup(t, models)
+		fm := &fakeModels{}
+		a.SetModels(fm)
+		mux := http.NewServeMux()
+		a.Register(mux)
+		req := httptest.NewRequest(http.MethodPost, "/api/v1/models/coder/adapters",
+			strings.NewReader(`{"adapters":[{"name":"fix","source":"`+srv.URL+tc.file+`"}]}`))
+		rec := httptest.NewRecorder()
+		mux.ServeHTTP(rec, req)
+		if rec.Code != tc.code || !strings.Contains(rec.Body.String(), tc.want) {
+			t.Errorf("%s: %d %s; want %d mentioning %q", tc.file, rec.Code, rec.Body.String(), tc.code, tc.want)
+		}
+	}
+}
@@ -10,16 +10,19 @@ import (
 	"context"
 	"encoding/json"
 	"fmt"
-	"gitea.mynodes.xyz/ilker/llmgw/internal/control/modelserve"
-	"gitea.mynodes.xyz/ilker/llmgw/internal/release"
 	"io"
 	"net/http"
+	"net/url"
 	"slices"
 	"strconv"
 	"strings"
 	"time"
 	"unicode"

+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/modelserve"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/release"
+
 	"gitea.mynodes.xyz/ilker/llmgw/internal/agent/discovery"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/bench"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control"
@@ -42,6 +45,7 @@ type API struct {
 	models   Models
 	capacity Capacity
 	usage    Usage
+	quotas   Quotas
 	ctl      *control.Controller
 }

@@ -57,6 +61,29 @@ func (a *API) Register(mux *http.ServeMux) {
 	mux.HandleFunc("POST /api/v1/models", a.addModel)
 	mux.HandleFunc("DELETE /api/v1/models/{name}", a.removeModel)
 	mux.HandleFunc("POST /api/v1/models/{name}/scale", a.scaleModel)
+	mux.HandleFunc("POST /api/v1/models/{name}/resources", a.setResources)
+	a.registerQuotas(mux)
+	mux.HandleFunc("POST /api/v1/models/{name}/adapters", a.setAdapters)
+	mux.HandleFunc("GET /api/v1/models/{name}/target", a.target)
+	mux.HandleFunc("POST /api/v1/models/{name}/class", func(w http.ResponseWriter, r *http.Request) {
+		if a.models == nil {
+			writeError(w, http.StatusNotImplemented, "this control plane has no store")
+			return
+		}
+		var req struct {
+			Class string `json:"class"`
+		}
+		if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
+			writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+			return
+		}
+		m, err := a.models.SetClass(r.Context(), r.PathValue("name"), req.Class, actorOf(r))
+		if err != nil {
+			writeError(w, http.StatusBadRequest, err.Error())
+			return
+		}
+		writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "class": m.EffectiveClass()})
+	})
 	mux.HandleFunc("PUT /api/v1/models/{name}/capacity", a.putCapacity)
 	mux.HandleFunc("GET /api/v1/capacity", a.listCapacity)
 	mux.HandleFunc("GET /api/v1/usage", a.usageByScope)
@@ -119,6 +146,8 @@ type NodeView struct {
 	// what decides whether a CPU profile can be placed here, so it belongs
 	// where someone looks when a model will not schedule.
 	MemoryGB int `json:"memory_gb,omitempty"`
+	// CPUCount is the machine's CPUs, 0 when the agent did not report them.
+	CPUCount int `json:"cpu_count,omitempty"`
 	// Engines are the inference engines installed here, as the agent found
 	// them. Absent means the agent is older than the probe and said nothing,
 	// which is not the same as having none — placement treats it that way too,
@@ -224,6 +253,11 @@ type LoadView struct {
 	// MemPercent is used over used plus available.
 	MemPercent float64 `json:"memory_percent"`
 	AgeSec     int64   `json:"age_seconds"`
+	// What the node's sensors say; zero is not measured.
+	CPUTempC   float64               `json:"cpu_temp_c,omitempty"`
+	CPUPowerW  float64               `json:"cpu_power_w,omitempty"`
+	InletTempC float64               `json:"inlet_temp_c,omitempty"`
+	GPUs       []inventory.GPUSensor `json:"gpu_sensors,omitempty"`
 }

 // loadFresh is how old a measurement may be and still be shown as current.
@@ -237,7 +271,8 @@ func loadView(l *inventory.Load, now time.Time) *LoadView {
 		CPUPercent: l.CPUPercent, CPUs: l.CPUCount,
 		Load1: l.Load1, Load5: l.Load5, Load15: l.Load15,
 		MemUsed: l.MemUsedBytes, MemTotal: l.MemUsedBytes + l.MemAvailBytes,
-		AgeSec: int64(now.Sub(l.At).Seconds()),
+		AgeSec:   int64(now.Sub(l.At).Seconds()),
+		CPUTempC: l.CPUTempC, CPUPowerW: l.CPUPowerW, InletTempC: l.InletTempC, GPUs: l.GPUs,
 	}
 	if v.MemTotal > 0 {
 		v.MemPercent = float64(v.MemUsed) * 100 / float64(v.MemTotal)
@@ -270,6 +305,26 @@ type ModelView struct {
 	Unplaceable bool     `json:"unplaceable,omitempty"`
 	Fallbacks   []string `json:"fallbacks,omitempty"`
 	Profiles    []string `json:"profiles"`
+	// MemoryGB and CPUCores are what one replica reserves, from its first
+	// profile; MeasuredMemoryGB is the most one of its replicas uses now, 0
+	// when none is measured.
+	MemoryGB         int     `json:"memory_gb,omitempty"`
+	CPUCores         int     `json:"cpu_cores,omitempty"`
+	MeasuredMemoryGB float64 `json:"measured_memory_gb,omitempty"`
+	// Warnings are what makes its reservation wrong: nothing declared where
+	// placement needs a figure, or less declared than it uses.
+	Warnings []string `json:"warnings,omitempty"`
+	// Class is its capacity class; Preempted the replicas it lost to a higher
+	// class; HeldFor the model it is giving room to now, if any.
+	Class string `json:"class"`
+	Owner string `json:"owner,omitempty"`
+	// External is where a model served outside the cluster goes, provider
+	// host and its name for the model; empty for one served here.
+	External string `json:"external,omitempty"`
+	// Adapters are the LoRA adapters it serves.
+	Adapters  []catalog.Adapter `json:"adapters,omitempty"`
+	Preempted int               `json:"preempted,omitempty"`
+	HeldFor   string            `json:"held_for,omitempty"`
 }

 // ReplicaView is a replica as `ctl replicas` shows it.
@@ -360,6 +415,12 @@ type HeadroomView struct {
 	Limited bool `json:"limited,omitempty"`
 	// Blocked says why one more does not fit, a line per node considered.
 	Blocked []string `json:"blocked,omitempty"`
+	// TokPerSec is what one more replica was measured to serve; 0 unmeasured.
+	TokPerSec float64 `json:"tok_per_sec,omitempty"`
+	// ByPreempting is how many more would fit by stopping replicas this
+	// model's class may take; Preempts the models those belong to.
+	ByPreempting int      `json:"by_preempting,omitempty"`
+	Preempts     []string `json:"preempts,omitempty"`
 }

 // RejectionView is one rejected candidate.
@@ -396,8 +457,8 @@ func (a *API) Nodes() []NodeView {
 			ID: n.ID, Hostname: n.Hostname, MgmtAddr: n.MgmtAddr,
 			State: n.State.String(), Labels: n.Labels,
 			GPUTotal: len(n.GPUs), GPUFree: len(inv.FreeGPUs(n.ID)),
-			MemoryGB: n.MemoryTotalMB / 1024,
-			Fabrics:  n.FabricNames(), AgentVersion: n.AgentVersion, Features: n.Features,
+			MemoryGB: n.MemoryTotalMB / 1024, CPUCount: n.CPUCount,
+			Fabrics: n.FabricNames(), AgentVersion: n.AgentVersion, Features: n.Features,
 			Replicas: replicasOn[n.ID],
 		}
 		if free, total := n.LocalDisk(); total > 0 {
@@ -497,6 +558,15 @@ func (a *API) Models() []ModelView {
 	for _, h := range a.ctl.Reconciler().ModelHealth() {
 		health[h.Model] = h
 	}
+	// The most one replica of each model uses, summed over its shards.
+	measured := map[string]int64{}
+	for _, rep := range a.Replicas("") {
+		var b int64
+		for _, sh := range rep.Shards {
+			b += sh.MemoryBytes
+		}
+		measured[rep.Model] = max(measured[rep.Model], b)
+	}
 	out := make([]ModelView, 0)
 	for _, m := range a.ctl.Catalog().Models() {
 		h := health[m.Name]
@@ -507,10 +577,17 @@ func (a *API) Models() []ModelView {
 			Fallbacks: m.Fallbacks,
 			Desired:   h.Desired, Failures: h.Failures,
 			Parked: h.Parked, Reason: h.Reason, Unplaceable: h.Unplaceable,
+			Class: m.EffectiveClass(), Owner: m.Owner, Preempted: h.Preempted, HeldFor: h.HeldFor,
+			External: externalLabel(m), Adapters: m.Adapters,
 		}
 		for _, p := range m.Profiles {
 			v.Profiles = append(v.Profiles, p.Name)
 		}
+		if len(m.Profiles) > 0 {
+			v.MemoryGB, v.CPUCores = m.Profiles[0].MinMemoryGB, m.Profiles[0].CPUCores
+		}
+		v.MeasuredMemoryGB = float64(measured[m.Name]) / (1 << 30)
+		v.Warnings = reservationWarnings(m, v.MeasuredMemoryGB)
 		out = append(out, v)
 	}
 	return out
@@ -581,6 +658,18 @@ type Models interface {
 	Add(ctx context.Context, m catalog.Model, by string) error
 	Remove(ctx context.Context, name string) error
 	Scale(ctx context.Context, name string, s catalog.Scaling, by string) (catalog.Model, error)
+	Resources(ctx context.Context, name, profile string, memoryGB, cpuCores int, by string) (catalog.Model, error)
+	SetClass(ctx context.Context, name, class, by string) (catalog.Model, error)
+	SetOwner(ctx context.Context, name, owner, by string) (catalog.Model, error)
+	SetAdapters(ctx context.Context, name string, adapters []catalog.Adapter, by string) (catalog.Model, error)
+}
+
+// ResourcesRequest sets what a model's profiles reserve of a node.
+type ResourcesRequest struct {
+	// Profile is one profile's name; empty is every profile.
+	Profile  string `json:"profile,omitempty"`
+	MemoryGB int    `json:"memory_gb"`
+	CPUCores int    `json:"cpu_cores"`
 }

 // Usage reports what each group of the tenancy tree spent. The store
@@ -826,6 +915,26 @@ func (a *API) scaleModel(w http.ResponseWriter, r *http.Request) {
 	})
 }

+// setResources sets a model's reserved memory and cores. Placed replicas keep
+// what they were started with until they are restarted.
+func (a *API) setResources(w http.ResponseWriter, r *http.Request) {
+	if a.models == nil {
+		writeError(w, http.StatusNotImplemented, "this control plane has no store")
+		return
+	}
+	var req ResourcesRequest
+	if err := json.NewDecoder(io.LimitReader(r.Body, 1<<16)).Decode(&req); err != nil {
+		writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+		return
+	}
+	m, err := a.models.Resources(r.Context(), r.PathValue("name"), req.Profile, req.MemoryGB, req.CPUCores, actorOf(r))
+	if err != nil {
+		writeError(w, http.StatusBadRequest, err.Error())
+		return
+	}
+	writeJSON(w, http.StatusOK, m)
+}
+
 func (a *API) removeModel(w http.ResponseWriter, r *http.Request) {
 	if a.models == nil {
 		writeError(w, http.StatusNotImplemented, "this control plane has no store")
@@ -873,7 +982,8 @@ func (a *API) Headroom(model string, limit int) (HeadroomView, bool) {
 		return HeadroomView{}, false
 	}
 	return HeadroomView{Model: model, Current: h.Current, More: h.More, Nodes: h.Nodes,
-		Limited: h.Limited, Blocked: h.Blocked}, true
+		Limited: h.Limited, Blocked: h.Blocked, TokPerSec: h.TokPerSec,
+		ByPreempting: h.ByPreempting, Preempts: h.Preempts}, true
 }

 func (a *API) Explain(model string) ExplainView {
@@ -1307,3 +1417,107 @@ func actorOf(r *http.Request) string {
 	}
 	return b.String()
 }
+
+// NodeCPUs is the CPU counts of ready nodes with at least this much memory:
+// where a replica needing it could go.
+func (a *API) NodeCPUs(minMemoryGB int) []int {
+	var out []int
+	for _, n := range a.ctl.Inventory().Nodes() {
+		if n.State.String() != "ready" || n.CPUCount == 0 {
+			continue
+		}
+		if n.MemoryTotalMB > 0 && n.MemoryTotalMB/1024 < minMemoryGB {
+			continue
+		}
+		out = append(out, n.CPUCount)
+	}
+	return out
+}
+
+// reservationWarnings says what is wrong with what a model reserves.
+//
+// Resident memory includes the weights mapped from disk, which the kernel can
+// take back, so it overstates what a model needs by the part of the weights
+// not in use. A tenth over is let pass for that; more is a figure too low for
+// placement to be trusted with.
+func reservationWarnings(m catalog.Model, measuredGB float64) []string {
+	var out []string
+	for _, p := range m.Profiles {
+		if p.CPUOnly() && p.MinMemoryGB <= 0 {
+			out = append(out, fmt.Sprintf("profile %s declares no memory: placement cannot tell when a node is full "+
+				"(llmgw ctl model resources %s --memory GB)", p.Name, m.Name))
+		}
+	}
+	if len(m.Profiles) == 1 && m.Profiles[0].MinMemoryGB > 0 && measuredGB > float64(m.Profiles[0].MinMemoryGB)*1.1 {
+		out = append(out, fmt.Sprintf("uses %.1f GB and reserves %d: raise it, or nodes will be filled past their memory",
+			measuredGB, m.Profiles[0].MinMemoryGB))
+	}
+	return out
+}
+
+// externalLabel is "api.openai.com · gpt-4o-mini" for a model served elsewhere.
+func externalLabel(m catalog.Model) string {
+	if !m.IsExternal() {
+		return ""
+	}
+	host := m.External.URL
+	if u, err := url.Parse(host); err == nil && u.Host != "" {
+		host = u.Host
+	}
+	return host + " · " + m.External.Model
+}
+
+// AdaptersRequest replaces a model's LoRA adapters with these.
+type AdaptersRequest struct {
+	Adapters []catalog.Adapter `json:"adapters"`
+}
+
+// setAdapters replaces a model's adapters, after reading each new file to
+// check it is an adapter for the model's architecture: one that is not is a
+// replica that fails to start, found minutes later on a node.
+func (a *API) setAdapters(w http.ResponseWriter, r *http.Request) {
+	if a.models == nil {
+		writeError(w, http.StatusNotImplemented, "this control plane has no store")
+		return
+	}
+	var req AdaptersRequest
+	if err := json.NewDecoder(io.LimitReader(r.Body, 1<<16)).Decode(&req); err != nil {
+		writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+		return
+	}
+	name := r.PathValue("name")
+	m, ok := a.ctl.Catalog().Resolve(name)
+	if !ok {
+		writeError(w, http.StatusBadRequest, "no model "+name)
+		return
+	}
+	base, baseErr := gguf.KindFromURL(r.Context(), nil, m.Source)
+	have := map[string]bool{}
+	for _, ad := range m.Adapters {
+		have[ad.Source] = true
+	}
+	for _, ad := range req.Adapters {
+		if have[ad.Source] {
+			continue // already checked when it was attached
+		}
+		h, err := gguf.KindFromURL(r.Context(), nil, ad.Source)
+		switch {
+		case err != nil:
+			writeError(w, http.StatusBadRequest, fmt.Sprintf("adapter %s: %v", ad.Name, err))
+			return
+		case !h.IsAdapter():
+			writeError(w, http.StatusBadRequest, fmt.Sprintf("adapter %s: %s is a model, not a LoRA adapter", ad.Name, ad.Source))
+			return
+		case baseErr == nil && h.Architecture != base.Architecture:
+			writeError(w, http.StatusBadRequest, fmt.Sprintf("adapter %s is for %s models and %s is %s",
+				ad.Name, h.Architecture, m.Name, base.Architecture))
+			return
+		}
+	}
+	out, err := a.models.SetAdapters(r.Context(), m.Name, req.Adapters, actorOf(r))
+	if err != nil {
+		writeError(w, http.StatusBadRequest, err.Error())
+		return
+	}
+	writeJSON(w, http.StatusOK, AdaptersRequest{Adapters: out.Adapters})
+}
@@ -258,6 +258,25 @@ type fakeModels struct {
 	err     error
 }

+func (f *fakeModels) SetAdapters(_ context.Context, name string, as []catalog.Adapter, _ string) (catalog.Model, error) {
+	return catalog.Model{Name: name, Adapters: as}, f.err
+}
+
+func (f *fakeModels) SetOwner(_ context.Context, name, owner, _ string) (catalog.Model, error) {
+	return catalog.Model{Name: name, Owner: owner}, f.err
+}
+
+func (f *fakeModels) SetClass(_ context.Context, name, class, _ string) (catalog.Model, error) {
+	return catalog.Model{Name: name, Class: class}, f.err
+}
+
+func (f *fakeModels) Resources(_ context.Context, name, profile string, mem, cores int, _ string) (catalog.Model, error) {
+	if f.err != nil {
+		return catalog.Model{}, f.err
+	}
+	return catalog.Model{Name: name, Profiles: []catalog.Profile{{Name: "p", MinMemoryGB: mem, CPUCores: cores}}}, nil
+}
+
 func (f *fakeModels) Scale(_ context.Context, name string, s catalog.Scaling, by string) (catalog.Model, error) {
 	if f.err != nil {
 		return catalog.Model{}, f.err
@@ -0,0 +1,285 @@
+package adminapi
+
+import (
+	"context"
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"sort"
+	"strings"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// Quotas is where capacity quotas and the tenancy tree they name are kept.
+// The store implements it.
+type Quotas interface {
+	CapacityQuotas(ctx context.Context) ([]store.CapacityQuota, error)
+	PutCapacityQuota(ctx context.Context, q store.CapacityQuota) error
+	DeleteCapacityQuota(ctx context.Context, scope store.Scope, id string) error
+	ScopeParents(ctx context.Context) (map[string][]string, error)
+	AllOrgs(ctx context.Context) ([]store.Org, error)
+	AllTeams(ctx context.Context) ([]store.TeamPath, error)
+	AllProjects(ctx context.Context) ([]store.ProjectPath, error)
+}
+
+// SetQuotas supplies the quota store, which mounts the quota endpoints'
+// behaviour; without it they answer 501.
+func (a *API) SetQuotas(q Quotas) { a.quotas = q }
+
+// QuotaView is one scope's quota with what it holds now.
+type QuotaView struct {
+	// Scope is team:ID; Name is how people know it, "Acme / ml".
+	Scope     string          `json:"scope"`
+	Name      string          `json:"name"`
+	Guarantee store.Resources `json:"guarantee"`
+	Limit     store.Resources `json:"limit"`
+	Holds     quota.Amount    `json:"holds"`
+	UpdatedBy string          `json:"updated_by,omitempty"`
+	UpdatedAt time.Time       `json:"updated_at"`
+}
+
+// QuotaRequest sets a scope's quota. Scope may be written with an ID or a
+// name: org:Acme, team:Acme/ml, project:Acme/ml/training.
+type QuotaRequest struct {
+	Scope     string          `json:"scope"`
+	Guarantee store.Resources `json:"guarantee"`
+	Limit     store.Resources `json:"limit"`
+}
+
+func (a *API) registerQuotas(mux *http.ServeMux) {
+	mux.HandleFunc("GET /api/v1/quotas", func(w http.ResponseWriter, r *http.Request) {
+		if a.quotas == nil {
+			writeError(w, http.StatusNotImplemented, "this control plane has no store")
+			return
+		}
+		views, err := a.QuotaViews(r.Context())
+		if err != nil {
+			writeError(w, http.StatusInternalServerError, err.Error())
+			return
+		}
+		writeJSON(w, http.StatusOK, views)
+	})
+	mux.HandleFunc("PUT /api/v1/quotas", func(w http.ResponseWriter, r *http.Request) {
+		if a.quotas == nil {
+			writeError(w, http.StatusNotImplemented, "this control plane has no store")
+			return
+		}
+		var req QuotaRequest
+		if err := json.NewDecoder(io.LimitReader(r.Body, 1<<14)).Decode(&req); err != nil {
+			writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+			return
+		}
+		key, err := a.SetQuota(r.Context(), req, actorOf(r))
+		if err != nil {
+			writeError(w, http.StatusBadRequest, err.Error())
+			return
+		}
+		writeJSON(w, http.StatusOK, map[string]string{"scope": key})
+	})
+	mux.HandleFunc("DELETE /api/v1/quotas/{scope}", func(w http.ResponseWriter, r *http.Request) {
+		if a.quotas == nil {
+			writeError(w, http.StatusNotImplemented, "this control plane has no store")
+			return
+		}
+		key, err := a.ClearQuota(r.Context(), r.PathValue("scope"))
+		if err != nil {
+			writeError(w, http.StatusBadRequest, err.Error())
+			return
+		}
+		writeJSON(w, http.StatusOK, map[string]string{"removed": key})
+	})
+	mux.HandleFunc("POST /api/v1/models/{name}/owner", func(w http.ResponseWriter, r *http.Request) {
+		if a.models == nil || a.quotas == nil {
+			writeError(w, http.StatusNotImplemented, "this control plane has no store")
+			return
+		}
+		var req struct {
+			Owner string `json:"owner"`
+		}
+		if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
+			writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+			return
+		}
+		owner := ""
+		if req.Owner != "" {
+			scope, id, _, err := a.resolveScope(r.Context(), req.Owner)
+			if err != nil {
+				writeError(w, http.StatusBadRequest, err.Error())
+				return
+			}
+			owner = quota.Key(scope, id)
+		}
+		m, err := a.models.SetOwner(r.Context(), r.PathValue("name"), owner, actorOf(r))
+		if err != nil {
+			writeError(w, http.StatusBadRequest, err.Error())
+			return
+		}
+		writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "owner": m.Owner})
+	})
+}
+
+// QuotaViews lists every quota with what its scope holds now.
+func (a *API) QuotaViews(ctx context.Context) ([]QuotaView, error) {
+	if a.quotas == nil {
+		return nil, nil
+	}
+	qs, err := a.quotas.CapacityQuotas(ctx)
+	if err != nil {
+		return nil, err
+	}
+	parents, err := a.quotas.ScopeParents(ctx)
+	if err != nil {
+		return nil, err
+	}
+	names, err := a.scopeNames(ctx)
+	if err != nil {
+		return nil, err
+	}
+	usage := quota.New(qs, parents).Usage(a.ctl.Reconciler().Holdings())
+	out := make([]QuotaView, 0, len(qs))
+	for _, q := range qs {
+		key := quota.Key(q.Scope, q.ScopeID)
+		out = append(out, QuotaView{Scope: key, Name: names[key], Guarantee: q.Guarantee, Limit: q.Limit,
+			Holds: usage[key], UpdatedBy: q.UpdatedBy, UpdatedAt: q.UpdatedAt})
+	}
+	return out, nil
+}
+
+// scopeNames maps org:ID, team:ID and project:ID to their paths.
+func (a *API) scopeNames(ctx context.Context) (map[string]string, error) {
+	out := map[string]string{}
+	orgs, err := a.quotas.AllOrgs(ctx)
+	if err != nil {
+		return nil, err
+	}
+	for _, o := range orgs {
+		out["org:"+o.ID] = o.Name
+	}
+	teams, err := a.quotas.AllTeams(ctx)
+	if err != nil {
+		return nil, err
+	}
+	for _, t := range teams {
+		out["team:"+t.ID] = t.OrgName + " / " + t.Name
+	}
+	projects, err := a.quotas.AllProjects(ctx)
+	if err != nil {
+		return nil, err
+	}
+	for _, p := range projects {
+		out["project:"+p.ID] = p.Path
+	}
+	return out, nil
+}
+
+// resolveScope reads org:X, team:X or project:X, where X is an ID or the
+// scope's path — Acme, Acme/ml, Acme/ml/training — and returns its ID and
+// path. A path that could mean two scopes is refused rather than guessed.
+func (a *API) resolveScope(ctx context.Context, s string) (store.Scope, string, string, error) {
+	kind, what, ok := strings.Cut(strings.TrimSpace(s), ":")
+	scope := store.Scope(kind)
+	if !ok || what == "" || (scope != store.ScopeOrg && scope != store.ScopeTeam && scope != store.ScopeProject) {
+		return "", "", "", fmt.Errorf("%q is not org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT", s)
+	}
+	names, err := a.scopeNames(ctx)
+	if err != nil {
+		return "", "", "", err
+	}
+	norm := func(p string) string {
+		parts := strings.Split(p, "/")
+		for i := range parts {
+			parts[i] = strings.ToLower(strings.TrimSpace(parts[i]))
+		}
+		return strings.Join(parts, "/")
+	}
+	if name, ok := names[kind+":"+what]; ok {
+		return scope, what, name, nil
+	}
+	match := func(same func(name string) bool) []string {
+		var found []string
+		for key, name := range names {
+			if strings.HasPrefix(key, kind+":") && same(name) {
+				found = append(found, key)
+			}
+		}
+		return found
+	}
+	found := match(func(name string) bool { return norm(name) == norm(what) })
+	if len(found) == 0 {
+		// Its own name alone — ml rather than Acme/ml — when that is unique.
+		found = match(func(name string) bool {
+			parts := strings.Split(norm(name), "/")
+			return parts[len(parts)-1] == norm(what)
+		})
+	}
+	switch len(found) {
+	case 0:
+		return "", "", "", fmt.Errorf("there is no %s %q", kind, what)
+	case 1:
+		_, id, _ := strings.Cut(found[0], ":")
+		return scope, id, names[found[0]], nil
+	}
+	return "", "", "", fmt.Errorf("%q could be any of %d %ss; give its ID", what, len(found), kind)
+}
+
+// SetQuota sets a scope's capacity quota, named by path or ID, and returns
+// the scope as org:ID.
+func (a *API) SetQuota(ctx context.Context, req QuotaRequest, by string) (string, error) {
+	if a.quotas == nil {
+		return "", fmt.Errorf("this control plane has no store")
+	}
+	scope, id, _, err := a.resolveScope(ctx, req.Scope)
+	if err != nil {
+		return "", err
+	}
+	if !req.Guarantee.Any() && !req.Limit.Any() {
+		return "", fmt.Errorf("a quota needs a guarantee or a limit; to remove one, clear it")
+	}
+	q := store.CapacityQuota{Scope: scope, ScopeID: id, Guarantee: req.Guarantee, Limit: req.Limit, UpdatedBy: by}
+	if err := a.quotas.PutCapacityQuota(ctx, q); err != nil {
+		return "", err
+	}
+	return quota.Key(scope, id), nil
+}
+
+// ClearQuota removes a scope's capacity quota.
+func (a *API) ClearQuota(ctx context.Context, scope string) (string, error) {
+	if a.quotas == nil {
+		return "", fmt.Errorf("this control plane has no store")
+	}
+	s, id, _, err := a.resolveScope(ctx, scope)
+	if err != nil {
+		return "", err
+	}
+	if err := a.quotas.DeleteCapacityQuota(ctx, s, id); err != nil {
+		return "", err
+	}
+	return quota.Key(s, id), nil
+}
+
+// ScopeName is a part of the tenancy tree: org:ID and its path.
+type ScopeName struct {
+	Key  string `json:"scope"`
+	Name string `json:"name"`
+}
+
+// Scopes lists every organization, team and project, by path.
+func (a *API) Scopes(ctx context.Context) ([]ScopeName, error) {
+	if a.quotas == nil {
+		return nil, nil
+	}
+	names, err := a.scopeNames(ctx)
+	if err != nil {
+		return nil, err
+	}
+	out := make([]ScopeName, 0, len(names))
+	for k, n := range names {
+		out = append(out, ScopeName{Key: k, Name: n})
+	}
+	sort.Slice(out, func(i, j int) bool { return out[i].Name < out[j].Name })
+	return out, nil
+}
@@ -0,0 +1,107 @@
+package adminapi
+
+import (
+	"context"
+	"encoding/json"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// fakeQuotaStore is Acme with teams ml and web, and Scania with another ml.
+type fakeQuotaStore struct {
+	quotas []store.CapacityQuota
+}
+
+func (f *fakeQuotaStore) CapacityQuotas(context.Context) ([]store.CapacityQuota, error) {
+	return f.quotas, nil
+}
+func (f *fakeQuotaStore) PutCapacityQuota(_ context.Context, q store.CapacityQuota) error {
+	for i := range f.quotas {
+		if f.quotas[i].Scope == q.Scope && f.quotas[i].ScopeID == q.ScopeID {
+			f.quotas[i] = q
+			return nil
+		}
+	}
+	f.quotas = append(f.quotas, q)
+	return nil
+}
+func (f *fakeQuotaStore) DeleteCapacityQuota(_ context.Context, s store.Scope, id string) error {
+	for i, q := range f.quotas {
+		if q.Scope == s && q.ScopeID == id {
+			f.quotas = append(f.quotas[:i], f.quotas[i+1:]...)
+		}
+	}
+	return nil
+}
+func (f *fakeQuotaStore) ScopeParents(context.Context) (map[string][]string, error) {
+	return map[string][]string{"team:t-ml": {"org:o-acme"}, "team:t-web": {"org:o-acme"},
+		"team:t-sml": {"org:o-scania"}}, nil
+}
+func (f *fakeQuotaStore) AllOrgs(context.Context) ([]store.Org, error) {
+	return []store.Org{{ID: "o-acme", Name: "Acme"}, {ID: "o-scania", Name: "Scania"}}, nil
+}
+func (f *fakeQuotaStore) AllTeams(context.Context) ([]store.TeamPath, error) {
+	return []store.TeamPath{{ID: "t-ml", Name: "ml", OrgID: "o-acme", OrgName: "Acme"},
+		{ID: "t-web", Name: "web", OrgID: "o-acme", OrgName: "Acme"},
+		{ID: "t-sml", Name: "ml", OrgID: "o-scania", OrgName: "Scania"}}, nil
+}
+func (f *fakeQuotaStore) AllProjects(context.Context) ([]store.ProjectPath, error) { return nil, nil }
+
+func TestAScopeIsNamedByPathOrIDAndNeverGuessed(t *testing.T) {
+	a, _ := setup(t, testModels())
+	a.SetQuotas(&fakeQuotaStore{})
+	for in, want := range map[string]string{
+		"org:Acme": "o-acme", "org:o-acme": "o-acme", "team:Acme/ml": "t-ml",
+		"team: acme / ML ": "t-ml", "team:Scania/ml": "t-sml", "team:web": "t-web",
+	} {
+		_, id, _, err := a.resolveScope(t.Context(), in)
+		if err != nil || id != want {
+			t.Errorf("%q: %q %v, expected %q", in, id, err, want)
+		}
+	}
+	for _, bad := range []string{"team:ml", "team:Acme/nope", "group:Acme", "Acme", "org:"} {
+		if _, _, _, err := a.resolveScope(t.Context(), bad); err == nil {
+			t.Errorf("%q was resolved", bad)
+		}
+	}
+	if _, _, _, err := a.resolveScope(t.Context(), "team:ml"); err == nil || !strings.Contains(err.Error(), "could be any of 2") {
+		t.Errorf("an ambiguous name should say so: %v", err)
+	}
+}
+
+func TestAQuotaIsSetListedAndCleared(t *testing.T) {
+	a, _ := setup(t, testModels())
+	qs := &fakeQuotaStore{}
+	a.SetQuotas(qs)
+	mux := http.NewServeMux()
+	a.Register(mux)
+	call := func(method, path, body string) *httptest.ResponseRecorder {
+		req := httptest.NewRequest(method, path, strings.NewReader(body))
+		req.Header.Set("Content-Type", "application/json")
+		req.Header.Set("X-Llmgw-Actor", "ops")
+		rec := httptest.NewRecorder()
+		mux.ServeHTTP(rec, req)
+		return rec
+	}
+	if rec := call("PUT", "/api/v1/quotas", `{"scope":"team:Acme/ml","guarantee":{"gpus":8},"limit":{"gpus":16}}`); rec.Code != 200 {
+		t.Fatalf("set: %d %s", rec.Code, rec.Body)
+	}
+	if len(qs.quotas) != 1 || qs.quotas[0].ScopeID != "t-ml" || *qs.quotas[0].Guarantee.GPUs != 8 || qs.quotas[0].UpdatedBy != "ops" {
+		t.Errorf("stored %+v", qs.quotas)
+	}
+	rec := call("GET", "/api/v1/quotas", "")
+	var views []QuotaView
+	if err := json.Unmarshal(rec.Body.Bytes(), &views); err != nil || len(views) != 1 || views[0].Name != "Acme / ml" {
+		t.Errorf("list %d %s", rec.Code, rec.Body)
+	}
+	if rec := call("PUT", "/api/v1/quotas", `{"scope":"team:Acme/ml"}`); rec.Code != 400 {
+		t.Errorf("an empty quota was accepted: %d", rec.Code)
+	}
+	if rec := call("DELETE", "/api/v1/quotas/team:Acme%2Fml", ""); rec.Code != 200 || len(qs.quotas) != 0 {
+		t.Errorf("clear: %d %s %+v", rec.Code, rec.Body, qs.quotas)
+	}
+}
@@ -0,0 +1,69 @@
+package adminapi
+
+import (
+	"fmt"
+	"math"
+	"net/http"
+	"strconv"
+)
+
+// TargetView is how many replicas a model needs to serve a throughput, from
+// what one replica was measured to serve, and whether the cluster has room.
+type TargetView struct {
+	Model string `json:"model"`
+	// TokPerSec is what was asked for; PerReplica what one replica was
+	// measured to serve at its SLO, on Profile.
+	TokPerSec  float64 `json:"tok_per_sec"`
+	PerReplica float64 `json:"per_replica_tok_per_sec"`
+	Profile    string  `json:"profile"`
+	// Needed replicas in all; Current hold capacity now; More is how many of
+	// Needed are not running, and Room how many more fit.
+	Needed  int      `json:"needed"`
+	Current int      `json:"current"`
+	More    int      `json:"more"`
+	Room    int      `json:"room"`
+	Fits    bool     `json:"fits"`
+	Blocked []string `json:"blocked,omitempty"`
+	// Preempts is whose replicas would be stopped to make the room, when
+	// free room alone is not enough; Room counts that room too.
+	Preempts []string `json:"preempts,omitempty"`
+}
+
+// Target sizes a model for a throughput.
+func (a *API) Target(model string, tokPerSec float64) (TargetView, error) {
+	meas, profile, ok := a.ctl.Reconciler().MeasuredFor(model)
+	if !ok || meas.TokPerSec <= 0 {
+		return TargetView{}, fmt.Errorf("%s has not been measured; run `llmgw ctl bench %s --save` first", model, model)
+	}
+	v := TargetView{Model: model, TokPerSec: tokPerSec, PerReplica: meas.TokPerSec, Profile: profile,
+		Needed: max(int(math.Ceil(tokPerSec/meas.TokPerSec)), 1)}
+	h, ok := a.Headroom(model, v.Needed)
+	if !ok {
+		return TargetView{}, fmt.Errorf("no model %q", model)
+	}
+	v.Current = h.Current
+	v.More = max(v.Needed-h.Current, 0)
+	v.Room = h.More + h.ByPreempting
+	v.Fits = v.More <= v.Room || h.Limited
+	if v.More > h.More {
+		v.Preempts = h.Preempts
+	}
+	if !v.Fits {
+		v.Blocked = h.Blocked
+	}
+	return v, nil
+}
+
+func (a *API) target(w http.ResponseWriter, r *http.Request) {
+	tps, err := strconv.ParseFloat(r.URL.Query().Get("tokens_per_sec"), 64)
+	if err != nil || tps <= 0 {
+		writeError(w, http.StatusBadRequest, "tokens_per_sec must be a positive number")
+		return
+	}
+	v, err := a.Target(r.PathValue("name"), tps)
+	if err != nil {
+		writeError(w, http.StatusBadRequest, err.Error())
+		return
+	}
+	writeJSON(w, http.StatusOK, v)
+}
@@ -0,0 +1,30 @@
+package adminapi
+
+import (
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
+)
+
+func TestATargetIsSizedFromTheMeasurementAndTheRoom(t *testing.T) {
+	// Three 8-GPU nodes; one tp8 replica was measured at 500 tokens/s.
+	a, ctl := setup(t, testModels(), gpuNode("1", 8), gpuNode("2", 8), gpuNode("3", 8))
+	if _, err := a.Target("llama", 1000); err == nil || !strings.Contains(err.Error(), "ctl bench llama --save") {
+		t.Errorf("an unmeasured model should say how to measure it: %v", err)
+	}
+	ctl.Reconciler().SetMeasurements(func() map[string]reconciler.Measured {
+		return map[string]reconciler.Measured{"llama/tp8": {MaxConcurrency: 16, TokPerSec: 500}}
+	})
+	v, err := a.Target("llama", 1200)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if v.Needed != 3 || v.PerReplica != 500 || v.Profile != "tp8" || v.More != 3 || !v.Fits {
+		t.Errorf("1200 tok/s at 500 each is 3 replicas on 3 empty nodes: %+v", v)
+	}
+	v, _ = a.Target("llama", 2000)
+	if v.Needed != 4 || v.Fits || v.Room != 3 || len(v.Blocked) == 0 {
+		t.Errorf("a fourth does not fit on three nodes, and should say why: %+v", v)
+	}
+}
@@ -31,6 +31,11 @@ func (a *API) Orphans() []OrphanView {
 	var wanted []string
 	for _, m := range a.ctl.Catalog().Models() {
 		wanted = append(wanted, m.Source)
+		// A model's adapters are its weights too; collecting them would
+		// leave replicas that cannot restart.
+		for _, ad := range m.Adapters {
+			wanted = append(wanted, ad.Source)
+		}
 	}

 	out := make([]OrphanView, 0)
@@ -4,6 +4,7 @@ package agentsvc
 import (
 	"fmt"
 	"maps"
+	"runtime"
 	"time"

 	llmgwv1 "gitea.mynodes.xyz/ilker/llmgw/api/gen/llmgw/v1"
@@ -27,6 +28,7 @@ func NodeFromRegister(r *llmgwv1.Register) inventory.Node {
 		EngineVersions: r.GetEngineVersions(),
 		EnginesProbed:  r.GetEnginesProbed(),
 		MemoryTotalMB:  int(r.GetMemoryTotalMb()),
+		CPUCount:       int(r.GetCpuCount()),
 	}
 	if h := r.GetHost(); h != nil {
 		n.Host = discovery.Host{
@@ -90,6 +92,7 @@ func RegisterFromInventory(nodeID, hostname, mgmtAddr, agentVersion string,
 		NodeId: nodeID, Hostname: hostname, MgmtAddr: mgmtAddr,
 		Labels: labels, AgentVersion: agentVersion,
 		MemoryTotalMb: int64(inv.MemoryTotalMB),
+		CpuCount:      int32(runtime.NumCPU()),
 	}
 	for _, w := range inv.Weights {
 		r.Weights = append(r.Weights, &llmgwv1.LocalWeights{
@@ -186,12 +189,14 @@ func StartShardFromCommand(cmd reconciler.ShardCommand) *llmgwv1.StartShard {
 			GpusPerNode: int32(p.GPUsPerNode), Nodes: int32(p.Nodes),
 			Args: p.Args, Env: p.Env,
 			ModelPath: p.ModelPath, ServedModelName: p.ServedModelName,
+			Adapters: adaptersToWire(p.Adapters),
 		},
 		StartupTimeoutMs: cmd.StartupTimeout.Milliseconds(),
 		Pool:             cmd.Pool,
 		Fabric:           cmd.Fabric,
 		ProfileFp:        cmd.ProfileFP,
 		ApiKey:           cmd.Spec.APIKey,
+		CpuThreads:       int32(cmd.Spec.Threads),
 	}
 }

@@ -213,12 +218,14 @@ func ShardSpecFromStart(m *llmgwv1.StartShard) (engines.ShardSpec, time.Duration
 		Fabric:        m.GetFabric(),
 		ProfileFP:     m.GetProfileFp(),
 		APIKey:        m.GetApiKey(),
+		Threads:       int(m.GetCpuThreads()),
 		Profile: engines.Profile{
 			Name: p.GetName(), Engine: p.GetEngine(), GPUClass: p.GetGpuClass(),
 			TP: int(p.GetTp()), PP: int(p.GetPp()),
 			GPUsPerNode: int(p.GetGpusPerNode()), Nodes: int(p.GetNodes()),
 			Args: p.GetArgs(), Env: p.GetEnv(),
 			ModelPath: p.GetModelPath(), ServedModelName: p.GetServedModelName(),
+			Adapters: adaptersFromWire(p.GetAdapters()),
 		},
 	}
 	return spec, time.Duration(m.GetStartupTimeoutMs()) * time.Millisecond
@@ -287,3 +294,19 @@ func fabricFrom(state, status, cluster, clique, bandwidth string) *discovery.GPU
 	}
 	return &discovery.GPUFabric{State: state, Status: status, ClusterUUID: cluster, CliqueID: clique, Bandwidth: bandwidth}
 }
+
+func adaptersToWire(as []engines.Adapter) []*llmgwv1.LoraAdapter {
+	var out []*llmgwv1.LoraAdapter
+	for _, a := range as {
+		out = append(out, &llmgwv1.LoraAdapter{Name: a.Name, Source: a.Path})
+	}
+	return out
+}
+
+func adaptersFromWire(as []*llmgwv1.LoraAdapter) []engines.Adapter {
+	var out []engines.Adapter
+	for _, a := range as {
+		out = append(out, engines.Adapter{Name: a.GetName(), Path: a.GetSource()})
+	}
+	return out
+}
@@ -0,0 +1,20 @@
+package agentsvc
+
+import (
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/agent/engines"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
+)
+
+func TestAdaptersTravelToTheAgentInOrder(t *testing.T) {
+	cmd := reconciler.ShardCommand{ReplicaID: "r", ShardID: "s", Spec: engines.ShardSpec{
+		Profile: engines.Profile{Name: "cpu", Engine: "llamacpp", ModelPath: "/m.gguf",
+			Adapters: []engines.Adapter{{Name: "sql", Path: "http://mirror/sql.gguf"}, {Name: "docs", Path: "http://mirror/docs.gguf"}}},
+	}}
+	spec, _ := ShardSpecFromStart(StartShardFromCommand(cmd))
+	got := spec.Profile.Adapters
+	if len(got) != 2 || got[0].Name != "sql" || got[1].Path != "http://mirror/docs.gguf" {
+		t.Errorf("adapters %+v", got)
+	}
+}
@@ -345,8 +345,9 @@ func (s *Service) applyHeartbeat(nodeID string, hb *llmgwv1.Heartbeat) {
 	problems := hb.GetNewProblems()
 	storage := hb.GetStorage()
 	memMB := int(hb.GetMemoryTotalMb())
+	cpus := int(hb.GetCpuCount())
 	if len(used) == 0 && len(problems) == 0 && len(storage) == 0 &&
-		!hb.GetWeightsChanged() && memMB == 0 && !hb.GetEnginesChanged() &&
+		!hb.GetWeightsChanged() && memMB == 0 && cpus == 0 && !hb.GetEnginesChanged() &&
 		!hb.GetProvisionChanged() && hb.GetLoad() == nil && len(hb.GetIbPorts()) == 0 &&
 		!hb.GetFabricsChanged() && len(hb.GetGpuFabric()) == 0 && hb.GetImex() == nil {
 		return
@@ -406,6 +407,12 @@ func (s *Service) applyHeartbeat(nodeID string, hb *llmgwv1.Heartbeat) {
 			"from_mb", n.MemoryTotalMB, "to_mb", memMB)
 		n.MemoryTotalMB = memMB
 	}
+	if cpus > 0 && cpus != n.CPUCount {
+		if n.CPUCount > 0 {
+			s.log.Info("node CPUs changed", "node", nodeID, "from", n.CPUCount, "to", cpus)
+		}
+		n.CPUCount = cpus
+	}
 	if hb.GetEnginesChanged() {
 		vers := maps.Clone(hb.GetEngineVersions())
 		if !maps.Equal(vers, n.EngineVersions) {
@@ -538,6 +545,13 @@ func loadFrom(l *llmgwv1.NodeLoad, at time.Time) *inventory.Load {
 			CPUCores: sh.GetCpuCores(), RSSBytes: sh.GetMemoryRssBytes(),
 		}
 	}
+	out.CPUTempC, out.CPUPowerW, out.InletTempC = l.GetCpuTempC(), l.GetCpuPowerW(), l.GetInletTempC()
+	for _, g := range l.GetGpus() {
+		out.GPUs = append(out.GPUs, inventory.GPUSensor{
+			Index: int(g.GetIndex()), TempC: g.GetTempC(), MemoryTempC: g.GetMemoryTempC(),
+			PowerW: g.GetPowerW(), PowerLimitW: g.GetPowerLimitW(), Throttle: g.GetThrottle(),
+		})
+	}
 	return out
 }

@@ -7,6 +7,7 @@ import (
 	"encoding/hex"
 	"encoding/json"
 	"fmt"
+	"net/url"
 	"slices"
 	"strings"
 	"time"
@@ -74,6 +75,17 @@ type Profile struct {
 	// the control plane.
 	MinMemoryGB int `json:"min_memory_gb,omitempty" koanf:"min_memory_gb"`

+	// CPUCores is the cores a replica of a profile with no GPU is given:
+	// reserved out of its node's CPUs by placement, as memory is, and what
+	// its engine computes on (llama.cpp's --threads).
+	//
+	// Reserved because an engine takes every core it is told about, and two
+	// on one node that each think they have all of them spin against each
+	// other: on an 8-vCPU node, two 7B replicas fell from 10 tokens a second
+	// to under one each. Zero reserves nothing and leaves the thread count to
+	// the node, which is every profile from before this.
+	CPUCores int `json:"cpu_cores,omitempty" koanf:"cpu_cores"`
+
 	// MinDiskGB is the free space a node needs before it can hold these
 	// weights.
 	//
@@ -138,9 +150,30 @@ type Model struct {
 	// choice by guesswork.
 	Description string `json:"description,omitempty" koanf:"description"`
 	// Source is the weights path or a HuggingFace id.
-	Source    string    `json:"source,omitempty" koanf:"source"`
-	Profiles  []Profile `json:"profiles,omitempty" koanf:"profiles"`
-	Scaling   Scaling   `json:"scaling,omitempty" koanf:"scaling"`
+	Source   string    `json:"source,omitempty" koanf:"source"`
+	Profiles []Profile `json:"profiles,omitempty" koanf:"profiles"`
+	Scaling  Scaling   `json:"scaling,omitempty" koanf:"scaling"`
+	// Class is how much this model's capacity is worth against another's,
+	// when there is not enough for both: critical, standard or opportunistic.
+	// Empty is standard, or opportunistic for a model loaded on demand. See
+	// EffectiveClass.
+	Class string `json:"class,omitempty" koanf:"class"`
+	// Owner is the part of the tenancy tree whose capacity quota this model
+	// counts against, written org:ID, team:ID or project:ID. Empty is shared:
+	// held by the cluster, counted against nobody.
+	Owner string `json:"owner,omitempty" koanf:"owner"`
+	// External is a model served by someone else — an OpenAI-compatible API
+	// outside the cluster — rather than by replicas placed here. Such a model
+	// has no profiles and nothing is placed for it; the router sends its
+	// requests to the provider. Most useful named in another model's
+	// fallbacks: used only when the cluster has no capacity left. Its
+	// prompts leave the cluster, so which keys may name it is worth an
+	// allowlist.
+	External *External `json:"external,omitempty" koanf:"external"`
+	// Adapters are LoRA adapters served on this model's replicas, each asked
+	// for by its own name as though it were a model. Every replica loads
+	// every adapter, applying one only to a request that names it.
+	Adapters  []Adapter `json:"adapters,omitempty" koanf:"adapters"`
 	Fallbacks []string  `json:"fallbacks,omitempty" koanf:"fallbacks"`
 	// RoutingProfile selects how the router scores endpoints for this model.
 	RoutingProfile string `json:"routing_profile,omitempty" koanf:"routing_profile"`
@@ -184,14 +217,17 @@ type Capabilities struct {
 type Catalog struct {
 	models  map[string]Model
 	aliases map[string]string
-	order   []string
+	// adapters maps an adapter's name to the model it runs on.
+	adapters map[string]string
+	order    []string
 }

 // New builds a catalog, rejecting definitions that would fail confusingly later.
 func New(models []Model) (*Catalog, error) {
 	c := &Catalog{
-		models:  make(map[string]Model, len(models)),
-		aliases: map[string]string{},
+		models:   make(map[string]Model, len(models)),
+		aliases:  map[string]string{},
+		adapters: map[string]string{},
 	}

 	for i := range models {
@@ -220,6 +256,18 @@ func New(models []Model) (*Catalog, error) {
 		}
 	}

+	// Adapters are named like models, so they share the namespace too.
+	for _, name := range c.order {
+		for _, a := range c.models[name].Adapters {
+			_, isModel := c.models[a.Name]
+			_, isAlias := c.aliases[a.Name]
+			if _, isAdapter := c.adapters[a.Name]; isModel || isAlias || isAdapter {
+				return nil, fmt.Errorf("model %q: adapter %q collides with a model, an alias or another adapter", name, a.Name)
+			}
+			c.adapters[a.Name] = name
+		}
+	}
+
 	// Fallbacks are resolved now rather than at request time: a typo in a
 	// fallback should stop a deployment, not surface during an outage when the
 	// fallback is finally needed.
@@ -285,9 +333,22 @@ func (m Model) validate() error {
 	if m.Source == "" {
 		return fmt.Errorf("model %q: source is required", m.Name)
 	}
-	if len(m.Profiles) == 0 {
+	if err := validateAdapters(m); err != nil {
+		return err
+	}
+	if m.External != nil {
+		if err := m.External.validate(m.Name, len(m.Profiles)); err != nil {
+			return err
+		}
+	} else if len(m.Profiles) == 0 {
 		return fmt.Errorf("model %q: at least one profile is required", m.Name)
 	}
+	if m.Owner != "" && !validOwner(m.Owner) {
+		return fmt.Errorf("model %q: owner %q is not org:ID, team:ID or project:ID", m.Name, m.Owner)
+	}
+	if !validClass(m.Class) {
+		return fmt.Errorf("model %q: class %q is not one of critical, standard, opportunistic", m.Name, m.Class)
+	}
 	if m.Scaling.Max < m.Scaling.Min {
 		return fmt.Errorf("model %q: scaling max %d is below min %d", m.Name, m.Scaling.Max, m.Scaling.Min)
 	}
@@ -331,6 +392,23 @@ func (m Model) validate() error {
 }

 // Resolve maps a client-supplied name or alias onto a model.
+// ResolveAdapter finds an adapter by its name: the model it runs on, the
+// adapter, and its index among the model's adapters — which is the id the
+// engine knows it by.
+func (c *Catalog) ResolveAdapter(name string) (Model, Adapter, int, bool) {
+	base, ok := c.adapters[name]
+	if !ok {
+		return Model{}, Adapter{}, 0, false
+	}
+	m := c.models[base]
+	for i, a := range m.Adapters {
+		if a.Name == name {
+			return m, a, i, true
+		}
+	}
+	return Model{}, Adapter{}, 0, false
+}
+
 func (c *Catalog) Resolve(name string) (Model, bool) {
 	if m, ok := c.models[name]; ok {
 		return m, true
@@ -380,6 +458,7 @@ func (m Model) PlacementProfiles() []placement.Profile {
 			NodeSelector: p.NodeSelector,
 			MinMemoryGB:  p.MinMemoryGB,
 			MinDiskGB:    p.MinDiskGB,
+			CPUCores:     p.CPUCores,
 			// The model's source, so placement can prefer nodes that already
 			// hold the weights rather than one that must first move them.
 			Source: m.Source,
@@ -406,6 +485,7 @@ func (m Model) EngineProfile(name string) (engines.Profile, bool) {
 			Env:             p.Env,
 			ModelPath:       m.Source,
 			ServedModelName: m.Name,
+			Adapters:        m.engineAdapters(),
 		}, true
 	}
 	return engines.Profile{}, false
@@ -475,3 +555,152 @@ func (p Profile) Fingerprint() string {
 	sum := sha256.Sum256(b)
 	return hex.EncodeToString(sum[:8])
 }
+
+// SuggestCores is the cores to give a replica with no GPU on nodes of these
+// CPU counts: half of the smallest, so two replicas share any of them without
+// competing. Zero when no node says, which reserves nothing.
+func SuggestCores(cpuCounts []int) int {
+	smallest := 0
+	for _, c := range cpuCounts {
+		if c > 0 && (smallest == 0 || c < smallest) {
+			smallest = c
+		}
+	}
+	if smallest == 0 {
+		return 0
+	}
+	return max(smallest/2, 1)
+}
+
+// The capacity classes, lowest first.
+const (
+	ClassOpportunistic = "opportunistic"
+	ClassStandard      = "standard"
+	ClassCritical      = "critical"
+)
+
+func validClass(c string) bool {
+	return c == "" || c == ClassOpportunistic || c == ClassStandard || c == ClassCritical
+}
+
+// EffectiveClass is the class a model is treated as: what it says, or, when
+// it says nothing, opportunistic for one loaded on demand — it holds capacity
+// only while asked for, and has always been the first to give it up — and
+// standard for everything else.
+func (m Model) EffectiveClass() string {
+	if m.Class != "" {
+		return m.Class
+	}
+	if m.Scaling.OnDemand() {
+		return ClassOpportunistic
+	}
+	return ClassStandard
+}
+
+// ClassRank orders classes: a higher rank may take capacity from a lower one.
+func ClassRank(c string) int {
+	switch c {
+	case ClassCritical:
+		return 3
+	case ClassStandard, "":
+		return 2
+	default:
+		return 1
+	}
+}
+
+func validOwner(o string) bool {
+	for _, p := range []string{"org:", "team:", "project:"} {
+		if strings.HasPrefix(o, p) && len(o) > len(p) {
+			return true
+		}
+	}
+	return false
+}
+
+// External is a provider outside the cluster that serves a model.
+type External struct {
+	// URL is the provider's base address, without /v1: https://api.openai.com.
+	URL string `json:"url" koanf:"url"`
+	// Model is what the provider calls it; requests are rewritten to it.
+	Model string `json:"model" koanf:"model"`
+	// APIKeyFile holds the provider's key, on the control plane. A file
+	// rather than the key itself, so the key is in neither the catalog, the
+	// store nor the audit log; empty for a provider that asks for none.
+	APIKeyFile string `json:"api_key_file,omitempty" koanf:"api_key_file"`
+}
+
+func (e *External) validate(model string, profiles int) error {
+	if profiles > 0 {
+		return fmt.Errorf("model %q: an external model is served elsewhere and has no profiles", model)
+	}
+	u, err := url.Parse(e.URL)
+	if err != nil || (u.Scheme != "https" && u.Scheme != "http") || u.Host == "" {
+		return fmt.Errorf("model %q: external url %q is not an http(s) address", model, e.URL)
+	}
+	if strings.TrimSpace(e.Model) == "" {
+		return fmt.Errorf("model %q: external model needs the name the provider knows it by", model)
+	}
+	return nil
+}
+
+// IsExternal reports whether a model is served outside the cluster.
+func (m Model) IsExternal() bool { return m.External != nil }
+
+// Adapter is a LoRA adapter served on a model's replicas.
+type Adapter struct {
+	// Name is what clients ask for, like a model name.
+	Name string `json:"name" koanf:"name"`
+	// Source is the adapter file (GGUF for llama.cpp), fetched by each node
+	// as the model's weights are.
+	Source string `json:"source" koanf:"source"`
+	// Scale is how strongly it is applied; zero takes 1.
+	Scale float64 `json:"scale,omitempty" koanf:"scale"`
+}
+
+// EffectiveScale is the scale an adapter is applied at.
+func (a Adapter) EffectiveScale() float64 {
+	if a.Scale <= 0 {
+		return 1
+	}
+	return a.Scale
+}
+
+func validateAdapters(m Model) error {
+	if len(m.Adapters) == 0 {
+		return nil
+	}
+	if m.External != nil {
+		return fmt.Errorf("model %q: an external model cannot carry adapters", m.Name)
+	}
+	for _, p := range m.Profiles {
+		// llama.cpp loads adapters at start and applies them per request;
+		// the other engines are not driven that way yet.
+		if p.Engine != "llamacpp" && p.Engine != "sim" {
+			return fmt.Errorf("model %q profile %q: adapters are served by llamacpp profiles only", m.Name, p.Name)
+		}
+	}
+	seen := map[string]bool{}
+	for _, a := range m.Adapters {
+		switch {
+		case a.Name == "" || a.Source == "":
+			return fmt.Errorf("model %q: an adapter needs a name and a source", m.Name)
+		case a.Name == m.Name:
+			return fmt.Errorf("model %q: an adapter cannot have the model's own name", m.Name)
+		case seen[a.Name]:
+			return fmt.Errorf("model %q: adapter %q is declared twice", m.Name, a.Name)
+		case a.Scale < 0:
+			return fmt.Errorf("model %q: adapter %q has a negative scale", m.Name, a.Name)
+		}
+		seen[a.Name] = true
+	}
+	return nil
+}
+
+func (m Model) engineAdapters() []engines.Adapter {
+	var out []engines.Adapter
+	for _, a := range m.Adapters {
+		out = append(out, engines.Adapter{Name: a.Name, Path: a.Source})
+	}
+	return out
+}
@@ -9,6 +9,7 @@ import (
 	"encoding/hex"
 	"fmt"
 	"log/slog"
+	"os"
 	"strings"
 	"sync"
 	"time"
@@ -338,6 +339,34 @@ func (c *Controller) endpoints() []endpoint.Info {
 			Priority: prof.Priority,
 		})
 	}
+	return append(out, c.externalEndpoints()...)
+}
+
+// externalEndpoints are the models served outside the cluster, as endpoints:
+// one each, with the provider's key read from its file every time, so a key
+// rotated on disk is used from the next publish on.
+func (c *Controller) externalEndpoints() []endpoint.Info {
+	var out []endpoint.Info
+	for _, m := range c.cat.Load().Models() {
+		if !m.IsExternal() {
+			continue
+		}
+		key := ""
+		if f := m.External.APIKeyFile; f != "" {
+			b, err := os.ReadFile(f) //nolint:gosec // the operator's own path
+			if err != nil {
+				c.log.Warn("an external model's key could not be read; not offered", "model", m.Name, "err", err)
+				continue
+			}
+			key = strings.TrimSpace(string(b))
+		}
+		out = append(out, endpoint.Info{
+			ID: "external:" + m.Name, URL: strings.TrimRight(m.External.URL, "/"),
+			Models: append([]string{m.Name}, m.Aliases...), APIKey: key,
+			Engine: endpoint.EngineExternal, Pool: "external", Weight: 1,
+			UpstreamModel: m.External.Model,
+		})
+	}
 	return out
 }

@@ -345,7 +374,8 @@ func (c *Controller) endpoints() []endpoint.Info {
 func fingerprint(infos []endpoint.Info) string {
 	var b strings.Builder
 	for _, i := range infos {
-		fmt.Fprintf(&b, "%s|%s|%s|%s|%v|%g|%d;", i.ID, i.URL, i.Engine, i.Pool, i.Models, i.Weight, i.Priority)
+		fmt.Fprintf(&b, "%s|%s|%s|%s|%v|%g|%d|%s|%x;", i.ID, i.URL, i.Engine, i.Pool, i.Models, i.Weight, i.Priority,
+			i.UpstreamModel, sha256.Sum256([]byte(i.APIKey)))
 	}
 	return b.String()
 }
@@ -375,3 +405,6 @@ func EngineKeys(secret []byte) func(replicaID string) string {
 		return "llmgw-" + hex.EncodeToString(m.Sum(nil))[:40]
 	}
 }
+
+// QuotaUse is every capacity quota beside what its scope holds.
+func (c *Controller) QuotaUse() []reconciler.QuotaUse { return c.rec.QuotaUse() }
@@ -0,0 +1,75 @@
+package control
+
+import (
+	"context"
+	"log/slog"
+	"os"
+	"path/filepath"
+	"sync"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
+)
+
+type recordSink struct {
+	mu    sync.Mutex
+	infos []endpoint.Info
+	times int
+}
+
+func (s *recordSink) SetEndpoints(_ context.Context, infos []endpoint.Info) {
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	s.infos, s.times = infos, s.times+1
+}
+
+type noDispatch struct{}
+
+func (noDispatch) StartShard(context.Context, reconciler.ShardCommand) error { return nil }
+func (noDispatch) StopShard(context.Context, string, string) error           { return nil }
+func (noDispatch) DrainShard(context.Context, string, string) error          { return nil }
+
+func TestAnExternalModelIsPublishedWithItsKeyAndNothingIsPlacedForIt(t *testing.T) {
+	dir := t.TempDir()
+	keyFile := filepath.Join(dir, "provider.key")
+	if err := os.WriteFile(keyFile, []byte("sk-provider-1\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	models := []catalog.Model{
+		{Name: "cloud", Source: "external", Aliases: []string{"gpt"}, Scaling: catalog.Scaling{Min: 3, Max: 3},
+			External: &catalog.External{URL: "https://api.example.com/", Model: "gpt-4o-mini", APIKeyFile: keyFile}},
+		{Name: "gone", Source: "external",
+			External: &catalog.External{URL: "https://api.example.com", Model: "m", APIKeyFile: filepath.Join(dir, "missing")}},
+	}
+	cat, err := catalog.New(models)
+	if err != nil {
+		t.Fatal(err)
+	}
+	sink := &recordSink{}
+	c := New(DefaultConfig(), catalog.NewLive(cat), noDispatch{}, sink, slog.New(slog.DiscardHandler))
+	c.Publish(t.Context())
+
+	if len(sink.infos) != 1 {
+		t.Fatalf("published %+v; the model whose key is missing must not be offered", sink.infos)
+	}
+	e := sink.infos[0]
+	if e.ID != "external:cloud" || e.URL != "https://api.example.com" || e.APIKey != "sk-provider-1" ||
+		e.UpstreamModel != "gpt-4o-mini" || e.Engine != endpoint.EngineExternal || len(e.Models) != 2 {
+		t.Errorf("endpoint %+v", e)
+	}
+	// A key rotated on disk is published on the next pass.
+	_ = os.WriteFile(keyFile, []byte("sk-provider-2"), 0o600)
+	c.Publish(t.Context())
+	if sink.times != 2 || sink.infos[0].APIKey != "sk-provider-2" {
+		t.Errorf("the rotated key was not published: %d pushes, key %q", sink.times, sink.infos[0].APIKey)
+	}
+	// Scaling of 3 or not, nothing is wanted of the cluster for a model
+	// served elsewhere, and so it never reads as short or unplaceable.
+	for _, h := range c.Reconciler().ModelHealth() {
+		if h.Model == "cloud" && (h.Desired != 0 || h.Unplaceable) {
+			t.Errorf("an external model wants replicas: %+v", h)
+		}
+	}
+}
@@ -51,12 +51,16 @@ type HealthSource interface {
 type Collector struct {
 	src Source

-	modelDesired     *prometheus.Desc
-	modelFailures    *prometheus.Desc
-	modelParked      *prometheus.Desc
-	modelUnplaceable *prometheus.Desc
-	engineInstall    *prometheus.Desc
-	leader           *prometheus.Desc
+	modelDesired      *prometheus.Desc
+	modelFailures     *prometheus.Desc
+	modelParked       *prometheus.Desc
+	modelUnplaceable  *prometheus.Desc
+	modelPreempted    *prometheus.Desc
+	capacityHolds     *prometheus.Desc
+	capacityGuarantee *prometheus.Desc
+	capacityLimit     *prometheus.Desc
+	engineInstall     *prometheus.Desc
+	leader            *prometheus.Desc

 	nodeUp       *prometheus.Desc
 	nodeGPUs     *prometheus.Desc
@@ -70,6 +74,13 @@ type Collector struct {
 	gpuMemoryUsed  *prometheus.Desc
 	gpuUtil        *prometheus.Desc
 	gpuTemp        *prometheus.Desc
+	gpuMemTemp     *prometheus.Desc
+	gpuPower       *prometheus.Desc
+	gpuPowerLimit  *prometheus.Desc
+	gpuThrottled   *prometheus.Desc
+	nodeCPUTemp    *prometheus.Desc
+	nodeCPUPower   *prometheus.Desc
+	nodeInletTemp  *prometheus.Desc
 	gpuHealthy     *prometheus.Desc
 	gpuNVLink      *prometheus.Desc
 	gpuECC         *prometheus.Desc
@@ -138,6 +149,21 @@ func New(src Source) *Collector {
 			"How busy a GPU was over the last sample period. Absent when the driver does not "+
 				"report it, which is not the same as idle. Memory used says a model is loaded; "+
 				"this says whether it is doing anything.", gpu, nil),
+		gpuMemTemp: prometheus.NewDesc("llmgw_gpu_memory_temperature_celsius",
+			"A GPU's memory (HBM) temperature, where the driver reports it.", gpu, nil),
+		gpuPower: prometheus.NewDesc("llmgw_gpu_power_watts",
+			"A GPU's power draw now.", gpu, nil),
+		gpuPowerLimit: prometheus.NewDesc("llmgw_gpu_power_limit_watts",
+			"The power limit a GPU is held to.", gpu, nil),
+		gpuThrottled: prometheus.NewDesc("llmgw_gpu_throttled",
+			"1 while something holds a GPU's clocks down: sw_power_cap, hw_slowdown, sw_thermal, hw_thermal, power_brake.",
+			append(gpu, "reason"), nil),
+		nodeCPUTemp: prometheus.NewDesc("llmgw_node_cpu_temperature_celsius",
+			"A node's hottest CPU sensor.", node, nil),
+		nodeCPUPower: prometheus.NewDesc("llmgw_node_cpu_power_watts",
+			"A node's CPU packages' power draw, from RAPL.", node, nil),
+		nodeInletTemp: prometheus.NewDesc("llmgw_node_inlet_temperature_celsius",
+			"A node's inlet or ambient air sensor, where its board has one.", node, nil),
 		gpuTemp: prometheus.NewDesc("llmgw_gpu_temperature_celsius",
 			"A GPU's reported temperature.", gpu, nil),
 		gpuHealthy: prometheus.NewDesc("llmgw_gpu_healthy",
@@ -168,6 +194,18 @@ func New(src Source) *Collector {
 		modelUnplaceable: prometheus.NewDesc("llmgw_model_unplaceable",
 			"1 when a model is short of replicas and the last attempt found no node to place one on.",
 			[]string{"model"}, nil),
+		capacityHolds: prometheus.NewDesc("llmgw_capacity_holds",
+			"What a part of the tenancy tree holds of a resource, its own models' replicas and those beneath it.",
+			[]string{"scope", "resource"}, nil),
+		capacityGuarantee: prometheus.NewDesc("llmgw_capacity_guarantee",
+			"What a part of the tenancy tree is guaranteed of a resource.",
+			[]string{"scope", "resource"}, nil),
+		capacityLimit: prometheus.NewDesc("llmgw_capacity_limit",
+			"The most of a resource a part of the tenancy tree may hold.",
+			[]string{"scope", "resource"}, nil),
+		modelPreempted: prometheus.NewDesc("llmgw_model_preempted_total",
+			"Replicas of a model stopped to make room for a model of a higher capacity class.",
+			[]string{"model", "class"}, nil),
 		engineInstall: prometheus.NewDesc("llmgw_node_engine_install",
 			"Where installing an engine stands on a node: 1 for the current state.",
 			[]string{"node", "engine", "version", "state"}, nil),
@@ -234,9 +272,11 @@ func (c *Collector) Describe(ch chan<- *prometheus.Desc) {
 		c.ibActive, c.ibRate, c.ibCounter, c.fabricSMs, c.fabricMaster, c.fabricProblems,
 		c.gpuNVLJoined, c.imexUp,
 		c.gpuMemoryTotal, c.gpuMemoryUsed, c.gpuUtil, c.gpuTemp, c.gpuHealthy,
+		c.gpuMemTemp, c.gpuPower, c.gpuPowerLimit, c.gpuThrottled, c.nodeCPUTemp, c.nodeCPUPower, c.nodeInletTemp,
 		c.gpuNVLink, c.gpuECC,
 		c.replicaState, c.replicaReady, c.replicaIdle,
-		c.modelDesired, c.modelFailures, c.modelParked, c.modelUnplaceable,
+		c.modelDesired, c.modelFailures, c.modelParked, c.modelUnplaceable, c.modelPreempted,
+		c.capacityHolds, c.capacityGuarantee, c.capacityLimit,
 		c.engineInstall, c.leader,
 	} {
 		ch <- d
@@ -246,6 +286,7 @@ func (c *Collector) Describe(ch chan<- *prometheus.Desc) {
 // Collect implements prometheus.Collector.
 func (c *Collector) Collect(ch chan<- prometheus.Metric) {
 	now := time.Now()
+	c.collectQuotas(ch)

 	loads := map[string]*inventory.Load{}
 	for _, n := range c.src.Nodes() {
@@ -279,6 +320,16 @@ func (c *Collector) Collect(ch chan<- prometheus.Metric) {
 		}

 		if l := n.Load; l != nil && now.Sub(l.At) < loadFresh {
+			// Each only when measured: a VM has none of these, and a zero
+			// would read as a machine at 0 °C drawing nothing.
+			for _, v := range []struct {
+				d   *prometheus.Desc
+				val float64
+			}{{c.nodeCPUTemp, l.CPUTempC}, {c.nodeCPUPower, l.CPUPowerW}, {c.nodeInletTemp, l.InletTempC}} {
+				if v.val > 0 {
+					ch <- prometheus.MustNewConstMetric(v.d, prometheus.GaugeValue, v.val, n.ID)
+				}
+			}
 			ch <- prometheus.MustNewConstMetric(c.nodeCPU, prometheus.GaugeValue, l.CPUPercent, n.ID)
 			if l.CPUCount > 0 {
 				ch <- prometheus.MustNewConstMetric(c.nodeCPUs, prometheus.GaugeValue, float64(l.CPUCount), n.ID)
@@ -348,7 +399,11 @@ func (c *Collector) Collect(ch chan<- prometheus.Metric) {
 				ch <- prometheus.MustNewConstMetric(c.gpuUtil, prometheus.GaugeValue,
 					float64(g.UtilizationPercent), labels...)
 			}
-			if g.Health.TemperatureC > 0 {
+			// The sensors' fresh reading when there is one; the temperature
+			// read at registration otherwise, which is all an older agent sends.
+			if s, ok := sensorFor(n.Load, g.Index, now); ok {
+				c.gpuSensor(ch, s, labels)
+			} else if g.Health.TemperatureC > 0 {
 				ch <- prometheus.MustNewConstMetric(c.gpuTemp, prometheus.GaugeValue,
 					float64(g.Health.TemperatureC), labels...)
 			}
@@ -455,6 +510,7 @@ func (c *Collector) Collect(ch chan<- prometheus.Metric) {
 		ch <- prometheus.MustNewConstMetric(c.modelFailures, prometheus.GaugeValue, float64(h.Failures), h.Model)
 		ch <- prometheus.MustNewConstMetric(c.modelParked, prometheus.GaugeValue, b2f(h.Parked), h.Model)
 		ch <- prometheus.MustNewConstMetric(c.modelUnplaceable, prometheus.GaugeValue, b2f(h.Unplaceable), h.Model)
+		ch <- prometheus.MustNewConstMetric(c.modelPreempted, prometheus.CounterValue, float64(h.Preempted), h.Model, h.Class)
 	}
 }

@@ -464,3 +520,65 @@ func b2f(b bool) float64 {
 	}
 	return 0
 }
+
+// QuotaSource adds the tenancy tree's capacity quotas. Asked for by type
+// assertion, like HealthSource.
+type QuotaSource interface {
+	QuotaUse() []reconciler.QuotaUse
+}
+
+func (c *Collector) collectQuotas(ch chan<- prometheus.Metric) {
+	qs, ok := c.src.(QuotaSource)
+	if !ok {
+		return
+	}
+	for _, q := range qs.QuotaUse() {
+		for _, r := range []struct {
+			name     string
+			holds    int
+			gua, lim *int
+		}{
+			{"gpus", q.Holds.GPUs, q.Guarantee.GPUs, q.Limit.GPUs},
+			{"cpu_cores", q.Holds.CPUCores, q.Guarantee.CPUCores, q.Limit.CPUCores},
+			{"memory_gb", q.Holds.MemoryGB, q.Guarantee.MemoryGB, q.Limit.MemoryGB},
+		} {
+			if r.gua == nil && r.lim == nil {
+				continue
+			}
+			ch <- prometheus.MustNewConstMetric(c.capacityHolds, prometheus.GaugeValue, float64(r.holds), q.Scope, r.name)
+			if r.gua != nil {
+				ch <- prometheus.MustNewConstMetric(c.capacityGuarantee, prometheus.GaugeValue, float64(*r.gua), q.Scope, r.name)
+			}
+			if r.lim != nil {
+				ch <- prometheus.MustNewConstMetric(c.capacityLimit, prometheus.GaugeValue, float64(*r.lim), q.Scope, r.name)
+			}
+		}
+	}
+}
+
+// sensorFor is a GPU's fresh sensor reading, if the node sent one.
+func sensorFor(l *inventory.Load, index int, now time.Time) (inventory.GPUSensor, bool) {
+	if l == nil || now.Sub(l.At) >= loadFresh {
+		return inventory.GPUSensor{}, false
+	}
+	for _, g := range l.GPUs {
+		if g.Index == index {
+			return g, true
+		}
+	}
+	return inventory.GPUSensor{}, false
+}
+
+func (c *Collector) gpuSensor(ch chan<- prometheus.Metric, s inventory.GPUSensor, labels []string) {
+	for _, v := range []struct {
+		d   *prometheus.Desc
+		val float64
+	}{{c.gpuTemp, s.TempC}, {c.gpuMemTemp, s.MemoryTempC}, {c.gpuPower, s.PowerW}, {c.gpuPowerLimit, s.PowerLimitW}} {
+		if v.val > 0 {
+			ch <- prometheus.MustNewConstMetric(v.d, prometheus.GaugeValue, v.val, labels...)
+		}
+	}
+	for _, reason := range s.Throttle {
+		ch <- prometheus.MustNewConstMetric(c.gpuThrottled, prometheus.GaugeValue, 1, append(labels, reason)...)
+	}
+}
@@ -0,0 +1,41 @@
+package fleetmetrics
+
+import (
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/agent/discovery"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
+)
+
+func TestSensorsArePublishedFreshAndOnlyWhenMeasured(t *testing.T) {
+	gpuNode := inventory.Node{ID: "g1", State: inventory.Ready,
+		GPUs: []discovery.GPU{{Index: 0, Name: "NVIDIA H100 80GB HBM3", MemoryTotalMB: 81559, Health: discovery.GPUHealth{TemperatureC: 40}}},
+		Load: &inventory.Load{At: time.Now(), CPUTempC: 58, CPUPowerW: 65,
+			GPUs: []inventory.GPUSensor{{Index: 0, TempC: 88, PowerW: 698, PowerLimitW: 700,
+				Throttle: []string{"sw_power_cap", "sw_thermal"}}}}}
+	vm := inventory.Node{ID: "vm1", State: inventory.Ready, Load: &inventory.Load{At: time.Now(), CPUPercent: 3}}
+	out := gather(t, fakeSource{nodes: []inventory.Node{gpuNode, vm}})
+
+	for _, want := range []string{
+		"llmgw_node_cpu_temperature_celsius node=g1 -> 58",
+		"llmgw_node_cpu_power_watts node=g1 -> 65",
+		"llmgw_gpu_power_watts class=h100-80g gpu=0 node=g1 -> 698",
+		"llmgw_gpu_power_limit_watts class=h100-80g gpu=0 node=g1 -> 700",
+		"llmgw_gpu_throttled class=h100-80g gpu=0 node=g1 reason=sw_thermal -> 1",
+		// The fresh reading, not the 40 °C from registration.
+		"llmgw_gpu_temperature_celsius class=h100-80g gpu=0 node=g1 -> 88",
+	} {
+		if !strings.Contains(out, want) {
+			t.Errorf("missing %q", want)
+		}
+	}
+	for _, absent := range []string{"node=vm1 -> 0", "llmgw_node_inlet_temperature_celsius", "llmgw_gpu_memory_temperature_celsius"} {
+		for _, line := range strings.Split(out, "\n") {
+			if strings.Contains(line, absent) && (strings.Contains(line, "temperature") || strings.Contains(line, "power")) {
+				t.Errorf("published what was not measured: %s", line)
+			}
+		}
+	}
+}
@@ -79,6 +79,9 @@ type Node struct {
 	// it, which is treated as no constraint — the same as every node before
 	// this field existed.
 	MemoryTotalMB int `json:"memory_total_mb,omitempty"`
+	// CPUCount is the CPUs the agent may run on. Zero is an agent too old to
+	// say, which, like memory, constrains nothing.
+	CPUCount int `json:"cpu_count,omitempty"`

 	// Weights are the model files this node can see, however they got there.
 	// Empty means the node was not asked to look, which is not the same as
@@ -301,6 +304,21 @@ type Load struct {
 	MemAvailBytes int64
 	// Shards is each engine's use, by shard ID.
 	Shards map[string]ShardLoad
+	// What the node's sensors say; zero is not measured.
+	CPUTempC   float64
+	CPUPowerW  float64
+	InletTempC float64
+	GPUs       []GPUSensor
+}
+
+// GPUSensor is one card's heat and power, and what is throttling it.
+type GPUSensor struct {
+	Index       int      `json:"index"`
+	TempC       float64  `json:"temp_c,omitempty"`
+	MemoryTempC float64  `json:"memory_temp_c,omitempty"`
+	PowerW      float64  `json:"power_w,omitempty"`
+	PowerLimitW float64  `json:"power_limit_w,omitempty"`
+	Throttle    []string `json:"throttle,omitempty"`
 }

 // ShardLoad is one engine's use: cores busy, and resident memory.
@@ -373,6 +391,9 @@ type Allocation struct {
 	// Zero for a profile that declared nothing, which constrains nothing and
 	// is counted as nothing.
 	MemoryGB int
+	// CPUCores is the cores this shard was given to compute on, for a shard
+	// with no GPU; zero for one that shares whatever is free.
+	CPUCores int
 }

 // Store holds the cluster inventory. It is safe for concurrent use.
@@ -91,6 +91,9 @@ type Request struct {
 	// use. They default to 16384 and 4096, what four parallel slots divide.
 	CtxPool       int `json:"ctx_pool,omitempty"`
 	ContextLength int `json:"context_length,omitempty"`
+	// CPUCores is what a replica computes on; zero takes half the smallest
+	// node it fits on.
+	CPUCores int `json:"cpu_cores,omitempty"`
 	// By is who asked, for the catalog's record.
 	By string `json:"-"`
 }
@@ -278,13 +281,20 @@ func (s *Server) run(j *Job) {
 	}
 	s.step(j, "check", Done, fmt.Sprintf("%s, needs about %d GB (%s)", orUnknown(header.Architecture), memGB, basis))

+	cores := req.CPUCores
+	if cores <= 0 {
+		if nc, ok := s.deps.Fleet.(interface{ NodeCPUs(int) []int }); ok {
+			cores = catalog.SuggestCores(nc.NodeCPUs(memGB))
+		}
+	}
+
 	// 4. The catalog.
 	m := catalog.Model{
 		Name: name, Aliases: req.Aliases, Description: req.Description, Source: source,
 		RoutingProfile: "chat", ContextLength: req.ContextLength, MaxOutput: 1024,
 		Profiles: []catalog.Profile{{
 			Name: "cpu", Engine: "llamacpp", Weight: 1, Priority: 100,
-			Args: []string{"--ctx-size", strconv.Itoa(req.CtxPool)}, MinMemoryGB: memGB,
+			Args: []string{"--ctx-size", strconv.Itoa(req.CtxPool)}, MinMemoryGB: memGB, CPUCores: cores,
 		}},
 		Scaling: catalog.Scaling{Min: req.Replicas, Max: max(req.Replicas, 1)},
 	}
@@ -18,6 +18,7 @@ package modelsvc
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"fmt"
 	"log/slog"
 	"sync"
@@ -86,6 +87,9 @@ func (s *Service) Sources(ctx context.Context) (map[string][]catalog.Model, erro
 		if m.Source != "" {
 			out[m.Source] = append(out[m.Source], m)
 		}
+		for _, ad := range m.Adapters {
+			out[ad.Source] = append(out[ad.Source], m)
+		}
 	}
 	return out, nil
 }
@@ -104,6 +108,16 @@ func (s *Service) Add(ctx context.Context, m catalog.Model, by string) error {
 	if s.Declared(m.Name) {
 		return fmt.Errorf("model %q is declared in configuration; edit that rather than adding a second definition", m.Name)
 	}
+	// A model with no GPU is all host memory, and one that declares none
+	// fits everywhere as far as placement can tell — however many of it
+	// there already are. Required of what is added now; models stored before
+	// this still load, and say what they are missing.
+	for _, p := range m.Profiles {
+		if p.CPUOnly() && p.MinMemoryGB <= 0 {
+			return fmt.Errorf("model %q profile %q runs without a GPU and declares no memory (min_memory_gb): "+
+				"placement could not tell when a node is full", m.Name, p.Name)
+		}
+	}
 	if s.check != nil {
 		if err := s.check(ctx, m); err != nil {
 			return err
@@ -189,36 +203,115 @@ func (s *Service) Remove(ctx context.Context, name string) error {
 	return nil
 }

-// Scale changes how many replicas a model runs, without a restart.
-//
-// Only the scaling block moves. Everything else about the model — its source,
-// its profiles, the engine arguments somebody tuned — is read back from what
-// was stored and written out again untouched, so changing a number cannot
-// quietly become a redefinition.
-//
-// A model declared in configuration is refused, for the same reason Remove
-// refuses one: the YAML is read on every start, so the new number would hold
-// until the next restart and then silently revert. A cluster that disagrees
-// with its own config file is worse than an error message saying where to edit.
+// Scale changes how many replicas a model added at runtime runs.
 func (s *Service) Scale(ctx context.Context, name string, sc catalog.Scaling, by string) (catalog.Model, error) {
+	final, err := s.update(ctx, name, by, "change its scaling", func(m *catalog.Model) error {
+		m.Scaling = sc
+		return nil
+	})
+	if err == nil {
+		s.log.Info("model scaled", "model", name,
+			"min", final.Scaling.Min, "max", final.Scaling.Max, "by", by)
+	}
+	return final, err
+}
+
+// Resources sets what a model's profile reserves of a node: host memory, and
+// the CPU cores a replica without a GPU computes on. An empty profile name
+// sets every profile.
+func (s *Service) Resources(ctx context.Context, name, profile string, memoryGB, cpuCores int, by string) (catalog.Model, error) {
+	if memoryGB < 0 || cpuCores < 0 {
+		return catalog.Model{}, errors.New("memory and cores cannot be negative")
+	}
+	final, err := s.update(ctx, name, by, "change its resources", func(m *catalog.Model) error {
+		found := false
+		for i := range m.Profiles {
+			if profile != "" && m.Profiles[i].Name != profile {
+				continue
+			}
+			found = true
+			m.Profiles[i].MinMemoryGB = memoryGB
+			m.Profiles[i].CPUCores = cpuCores
+		}
+		if !found {
+			return fmt.Errorf("model %q has no profile %q", name, profile)
+		}
+		return nil
+	})
+	if err == nil {
+		s.log.Info("model resources set", "model", name, "profile", profile,
+			"memory_gb", memoryGB, "cpu_cores", cpuCores, "by", by)
+	}
+	return final, err
+}
+
+// SetClass sets a model's capacity class: critical, standard or
+// opportunistic, or empty to go back to what its scaling implies.
+func (s *Service) SetClass(ctx context.Context, name, class, by string) (catalog.Model, error) {
+	final, err := s.update(ctx, name, by, "change its class", func(m *catalog.Model) error {
+		m.Class = class
+		return nil
+	})
+	if err == nil {
+		s.log.Info("model class set", "model", name, "class", final.EffectiveClass(), "by", by)
+	}
+	return final, err
+}
+
+// SetOwner sets the part of the tenancy tree a model's capacity counts
+// against, org:ID, team:ID or project:ID; empty makes it shared.
+func (s *Service) SetOwner(ctx context.Context, name, owner, by string) (catalog.Model, error) {
+	final, err := s.update(ctx, name, by, "change its owner", func(m *catalog.Model) error {
+		m.Owner = owner
+		return nil
+	})
+	if err == nil {
+		s.log.Info("model owner set", "model", name, "owner", owner, "by", by)
+	}
+	return final, err
+}
+
+// SetAdapters replaces the LoRA adapters a model serves. Replicas load the
+// new set when they next start.
+func (s *Service) SetAdapters(ctx context.Context, name string, adapters []catalog.Adapter, by string) (catalog.Model, error) {
+	final, err := s.update(ctx, name, by, "change its adapters", func(m *catalog.Model) error {
+		m.Adapters = adapters
+		return nil
+	})
+	if err == nil {
+		names := make([]string, 0, len(adapters))
+		for _, a := range adapters {
+			names = append(names, a.Name)
+		}
+		s.log.Info("model adapters set", "model", name, "adapters", names, "by", by)
+	}
+	return final, err
+}
+
+// update changes one model added at runtime and keeps the change.
+//
+// Built before anything is written, like Add: catalog.New is what applies the
+// defaults and refuses what does not validate, so the model that comes back
+// is the one that will run rather than the one that was asked for.
+func (s *Service) update(ctx context.Context, name, by, what string, change func(*catalog.Model) error) (catalog.Model, error) {
 	s.mu.Lock()
 	defer s.mu.Unlock()

 	if s.Declared(name) {
 		return catalog.Model{}, fmt.Errorf(
-			"model %q is declared in configuration; change its scaling there, or it reverts on the next restart", name)
+			"model %q is declared in configuration; %s there, or it reverts on the next restart", name, what)
 	}
-
 	stored, err := s.storedModels(ctx)
 	if err != nil {
 		return catalog.Model{}, err
 	}
-
 	next := append([]catalog.Model{}, s.declared...)
 	found := false
 	for _, e := range stored {
 		if e.Name == name {
-			e.Scaling = sc
+			if err := change(&e); err != nil {
+				return catalog.Model{}, err
+			}
 			found = true
 		}
 		next = append(next, e)
@@ -226,20 +319,14 @@ func (s *Service) Scale(ctx context.Context, name string, sc catalog.Scaling, by
 	if !found {
 		return catalog.Model{}, fmt.Errorf("model %q was not added at runtime", name)
 	}
-
-	// Built before anything is written, like Add: catalog.New is what applies
-	// the defaults (max follows min when it is left out) and what refuses a
-	// max below a min, so the model that comes back is the one that will run
-	// rather than the one that was asked for.
 	built, err := catalog.New(next)
 	if err != nil {
-		return catalog.Model{}, fmt.Errorf("scaling %q that way would make the catalog invalid: %w", name, err)
+		return catalog.Model{}, fmt.Errorf("that would make the catalog invalid: %w", err)
 	}
 	final, ok := built.Resolve(name)
 	if !ok {
 		return catalog.Model{}, fmt.Errorf("model %q vanished from the catalog it was just built into", name)
 	}
-
 	def, err := json.Marshal(final)
 	if err != nil {
 		return catalog.Model{}, err
@@ -249,10 +336,7 @@ func (s *Service) Scale(ctx context.Context, name string, sc catalog.Scaling, by
 	}); err != nil {
 		return catalog.Model{}, err
 	}
-
 	s.apply.ReplaceCatalog(ctx, built)
-	s.log.Info("model scaled", "model", name,
-		"min", final.Scaling.Min, "max", final.Scaling.Max, "by", by)
 	return final, nil
 }

@@ -48,7 +48,7 @@ func model(name string) catalog.Model {
 	return catalog.Model{
 		Name: name, Source: "/models/" + name,
 		Scaling:  catalog.Scaling{Min: 1, Max: 1},
-		Profiles: []catalog.Profile{{Name: "p", Engine: "llamacpp", Weight: 1}},
+		Profiles: []catalog.Profile{{Name: "p", Engine: "llamacpp", Weight: 1, MinMemoryGB: 2}},
 	}
 }

@@ -337,3 +337,43 @@ func TestMinZeroWithAMaxIsOnDemand(t *testing.T) {
 		t.Errorf("the idle timeout was lost; got %v", got.Scaling.IdleTimeout)
 	}
 }
+
+func TestACPUModelAddedNowMustSayHowMuchMemoryItNeeds(t *testing.T) {
+	s, st, _ := newService(t)
+	m := model("bare")
+	m.Profiles[0].MinMemoryGB = 0
+	err := s.Add(t.Context(), m, "tester")
+	if err == nil || !strings.Contains(err.Error(), "declares no memory") {
+		t.Fatalf("a CPU model with no memory was added: %v", err)
+	}
+	if len(st.rows) != 0 {
+		t.Error("it was stored anyway")
+	}
+}
+
+func TestResourcesAreSetOnAStoredModelAndSurviveARestart(t *testing.T) {
+	s, _, _ := newService(t)
+	if err := s.Add(t.Context(), model("m"), "tester"); err != nil {
+		t.Fatal(err)
+	}
+	got, err := s.Resources(t.Context(), "m", "", 6, 3, "tester")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if p := got.Profiles[0]; p.MinMemoryGB != 6 || p.CPUCores != 3 {
+		t.Errorf("profile %+v", p)
+	}
+	if _, err := s.Resources(t.Context(), "m", "nope", 6, 3, "tester"); err == nil {
+		t.Error("a profile the model does not have was accepted")
+	}
+	if _, err := s.Resources(t.Context(), "m", "", -1, 0, "tester"); err == nil {
+		t.Error("negative memory was accepted")
+	}
+	cat, err := s.Load(t.Context())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m, _ := cat.Resolve("m"); m.Profiles[0].CPUCores != 3 {
+		t.Error("the cores were lost on reload")
+	}
+}
@@ -1,6 +1,7 @@
 package placement

 import (
+	"strings"
 	"testing"

 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
@@ -100,3 +101,38 @@ func TestCPUProfileRespectsAvoid(t *testing.T) {
 		t.Errorf("placed on %q despite being told to avoid it", got)
 	}
 }
+
+func TestCPUCoresAreReservedAsMemoryIs(t *testing.T) {
+	n := node("n1", "", 0)
+	n.CPUCount = 8
+	st := store(n)
+	s := New(st)
+	p := cpuProfile("cpu")
+	p.CPUCores = 3
+
+	for i := range 2 {
+		res := s.Place([]Profile{p}, nil)
+		if res.Placement == nil {
+			t.Fatalf("replica %d did not fit in 8 cores:\n%s", i+1, res.Explain())
+		}
+		st.Allocate("n1", inventory.Allocation{ShardID: string(rune('a' + i)), CPUCores: 3})
+	}
+	// Six of eight given away: a third would need three.
+	res := s.Place([]Profile{p}, nil)
+	if res.Placement != nil {
+		t.Fatal("a third replica was given cores the node does not have")
+	}
+	if got := res.Explain(); !strings.Contains(got, "has 8 CPUs with 2 free, needs 3") {
+		t.Errorf("the refusal should say how many cores are left:\n%s", got)
+	}
+
+	// Unknown counts constrain nothing: an older agent, or a profile that
+	// reserves no cores.
+	old := node("n2", "", 0)
+	if res := New(store(old)).Place([]Profile{p}, nil); res.Placement == nil {
+		t.Error("a node that does not report its CPUs was excluded")
+	}
+	if res := s.Place([]Profile{cpuProfile("cpu")}, nil); res.Placement == nil {
+		t.Error("a profile reserving no cores was refused")
+	}
+}
@@ -39,6 +39,9 @@ type Profile struct {
 	// MinDiskGB is the free space needed to hold the weights, checked only on
 	// nodes that do not already have them. Zero is unconstrained.
 	MinDiskGB int
+	// CPUCores is the cores a profile with no GPU reserves. Zero is
+	// unconstrained.
+	CPUCores int
 	// Source is the model's weights, as the engine would be given it. Used to
 	// prefer nodes that already hold them; empty asks for no preference.
 	Source string
@@ -582,6 +585,18 @@ func (s *Scheduler) eligibleNodes(p Profile, fabric string, avoid []string, reje
 			}
 		}

+		// Cores, for a replica that computes on them: what is left after the
+		// cores already given to the node's other replicas. Same exemptions
+		// as memory: zero declares nothing, and a node that does not report
+		// its CPUs is an older agent, not one with none.
+		if p.CPUOnly() && p.CPUCores > 0 && n.CPUCount > 0 {
+			free := n.CPUCount - s.claimedCores(n.ID)
+			if free < p.CPUCores {
+				reject(fmt.Sprintf("has %d CPUs with %d free, needs %d", n.CPUCount, free, p.CPUCores))
+				continue
+			}
+		}
+
 		if p.CPUOnly() {
 			// Every ready node can run it; there is no card to be free.
 			out = append(out, candidate{node: n})
@@ -723,6 +738,15 @@ func (s *Scheduler) claimedMemory(nodeID string) int {
 	return total
 }

+// claimedCores is the CPU cores already given to replicas on a node.
+func (s *Scheduler) claimedCores(nodeID string) int {
+	total := 0
+	for _, a := range s.inv.Allocations(nodeID) {
+		total += a.CPUCores
+	}
+	return total
+}
+
 // installing says why an engine is missing, when the node has said.
 func installing(n inventory.Node, engine string) string {
 	for _, p := range n.Provision {
@@ -0,0 +1,192 @@
+// Package quota decides what the tenancy tree is guaranteed of the cluster
+// and how much of it each part may hold.
+//
+// A scope — an organization, team or project — may be guaranteed GPUs, CPU
+// cores and memory, and limited to at most some amount of each. What a scope
+// holds is what the replicas of the models it owns, and of the models its
+// teams and projects own, have reserved. Holding more than its guarantee is
+// borrowing: allowed while the room is free, and taken back when another
+// scope within its own guarantee needs it.
+package quota
+
+import (
+	"fmt"
+	"sort"
+	"strings"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// Amount is how much of each resource something holds or needs.
+type Amount struct {
+	GPUs, CPUCores, MemoryGB int
+}
+
+// Plus adds two amounts.
+func (a Amount) Plus(b Amount) Amount {
+	return Amount{a.GPUs + b.GPUs, a.CPUCores + b.CPUCores, a.MemoryGB + b.MemoryGB}
+}
+
+// Over names the first resource in which a is above the governed amounts of
+// l, with both figures; empty when it is above none.
+func Over(a Amount, l store.Resources) string {
+	switch {
+	case l.GPUs != nil && a.GPUs > *l.GPUs:
+		return fmt.Sprintf("%d GPUs of %d", a.GPUs, *l.GPUs)
+	case l.CPUCores != nil && a.CPUCores > *l.CPUCores:
+		return fmt.Sprintf("%d cores of %d", a.CPUCores, *l.CPUCores)
+	case l.MemoryGB != nil && a.MemoryGB > *l.MemoryGB:
+		return fmt.Sprintf("%d GB of memory of %d", a.MemoryGB, *l.MemoryGB)
+	}
+	return ""
+}
+
+// Holding is one replica's hold on the cluster, for working out who holds
+// what.
+type Holding struct {
+	Replica string
+	Owner   string
+	Amount  Amount
+	Since   time.Time
+}
+
+// Policy is the quotas and the tree they sit in, read at one moment.
+type Policy struct {
+	quotas  map[string]store.CapacityQuota
+	parents map[string][]string
+}
+
+// New builds a policy from stored quotas and each scope's parents.
+func New(quotas []store.CapacityQuota, parents map[string][]string) *Policy {
+	p := &Policy{quotas: map[string]store.CapacityQuota{}, parents: parents}
+	for _, q := range quotas {
+		p.quotas[Key(q.Scope, q.ScopeID)] = q
+	}
+	return p
+}
+
+// Key writes a scope the way owners are written: team:ID.
+func Key(s store.Scope, id string) string { return string(s) + ":" + id }
+
+// Chain is an owner and every scope above it, nearest first.
+func (p *Policy) Chain(owner string) []string {
+	if owner == "" {
+		return nil
+	}
+	return append([]string{owner}, p.parents[owner]...)
+}
+
+// Governing is the quotas along an owner's chain, nearest first.
+func (p *Policy) Governing(owner string) []store.CapacityQuota {
+	if p == nil {
+		return nil
+	}
+	var out []store.CapacityQuota
+	for _, s := range p.Chain(owner) {
+		if q, ok := p.quotas[s]; ok {
+			out = append(out, q)
+		}
+	}
+	return out
+}
+
+// Quotas lists every quota.
+func (p *Policy) Quotas() []store.CapacityQuota {
+	if p == nil {
+		return nil
+	}
+	out := make([]store.CapacityQuota, 0, len(p.quotas))
+	for _, q := range p.quotas {
+		out = append(out, q)
+	}
+	sort.Slice(out, func(i, j int) bool {
+		return Key(out[i].Scope, out[i].ScopeID) < Key(out[j].Scope, out[j].ScopeID)
+	})
+	return out
+}
+
+// Usage is what every scope holds, its own models' replicas and those of
+// everything beneath it.
+func (p *Policy) Usage(hs []Holding) map[string]Amount {
+	out := map[string]Amount{}
+	if p == nil {
+		return out
+	}
+	for _, h := range hs {
+		for _, s := range p.Chain(h.Owner) {
+			out[s] = out[s].Plus(h.Amount)
+		}
+	}
+	return out
+}
+
+// Borrowed is the replicas held beyond a guarantee: of each governed scope's
+// replicas, oldest first, the ones that take it past what it is guaranteed.
+// A replica beyond any guarantee along its chain is borrowed.
+func (p *Policy) Borrowed(hs []Holding) map[string]bool {
+	out := map[string]bool{}
+	if p == nil || len(p.quotas) == 0 {
+		return out
+	}
+	sorted := append([]Holding(nil), hs...)
+	sort.Slice(sorted, func(i, j int) bool { return sorted[i].Since.Before(sorted[j].Since) })
+	for key, q := range p.quotas {
+		if !q.Guarantee.Any() {
+			continue
+		}
+		var sum Amount
+		for _, h := range sorted {
+			if !contains(p.Chain(h.Owner), key) {
+				continue
+			}
+			sum = sum.Plus(h.Amount)
+			if Over(sum, q.Guarantee) != "" {
+				out[h.Replica] = true
+			}
+		}
+	}
+	return out
+}
+
+// Guaranteed reports whether an owner, holding what it holds, is still within
+// every guarantee along its chain with need added — so it may take back what
+// others borrowed. An owner with no guarantee anywhere is never guaranteed.
+func (p *Policy) Guaranteed(owner string, usage map[string]Amount, need Amount) bool {
+	gov := p.Governing(owner)
+	any := false
+	for _, q := range gov {
+		if !q.Guarantee.Any() {
+			continue
+		}
+		any = true
+		if Over(usage[Key(q.Scope, q.ScopeID)].Plus(need), q.Guarantee) != "" {
+			return false
+		}
+	}
+	return any
+}
+
+// AtLimit says which limit along an owner's chain need would take it past,
+// empty when none.
+func (p *Policy) AtLimit(owner string, usage map[string]Amount, need Amount) string {
+	for _, q := range p.Governing(owner) {
+		if !q.Limit.Any() {
+			continue
+		}
+		key := Key(q.Scope, q.ScopeID)
+		if over := Over(usage[key].Plus(need), q.Limit); over != "" {
+			return fmt.Sprintf("%s is at its capacity limit (%s)", key, strings.TrimSpace(over))
+		}
+	}
+	return ""
+}
+
+func contains(list []string, v string) bool {
+	for _, x := range list {
+		if x == v {
+			return true
+		}
+	}
+	return false
+}
@@ -0,0 +1,83 @@
+package quota
+
+import (
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+func n(v int) *int { return &v }
+
+// acme has two teams; ml is guaranteed 8 GPUs, the org as a whole 12 and
+// no more than 16.
+func policy() *Policy {
+	return New([]store.CapacityQuota{
+		{Scope: store.ScopeTeam, ScopeID: "ml", Guarantee: store.Resources{GPUs: n(8)}},
+		{Scope: store.ScopeOrg, ScopeID: "acme", Guarantee: store.Resources{GPUs: n(12)}, Limit: store.Resources{GPUs: n(16)}},
+	}, map[string][]string{
+		"team:ml": {"org:acme"}, "team:web": {"org:acme"},
+		"project:train": {"team:ml", "org:acme"},
+	})
+}
+
+func at(min int) time.Time { return time.Date(2026, 1, 1, 0, min, 0, 0, time.UTC) }
+
+func TestAScopeHoldsWhatEverythingBeneathItHolds(t *testing.T) {
+	p := policy()
+	u := p.Usage([]Holding{
+		{Replica: "a", Owner: "project:train", Amount: Amount{GPUs: 4}},
+		{Replica: "b", Owner: "team:web", Amount: Amount{GPUs: 2, CPUCores: 3}},
+		{Replica: "c", Owner: "", Amount: Amount{GPUs: 8}}, // shared: nobody's
+	})
+	if u["team:ml"].GPUs != 4 || u["org:acme"].GPUs != 6 || u["org:acme"].CPUCores != 3 || u["project:train"].GPUs != 4 {
+		t.Errorf("usage %+v", u)
+	}
+	if _, ok := u[""]; ok {
+		t.Error("a shared model was counted against somebody")
+	}
+}
+
+func TestWhatGoesPastAGuaranteeIsBorrowedNewestFirst(t *testing.T) {
+	p := policy()
+	b := p.Borrowed([]Holding{
+		{Replica: "old", Owner: "team:ml", Amount: Amount{GPUs: 8}, Since: at(1)},
+		{Replica: "new", Owner: "team:ml", Amount: Amount{GPUs: 4}, Since: at(2)},
+		{Replica: "web", Owner: "team:web", Amount: Amount{GPUs: 4}, Since: at(3)},
+	})
+	// ml: 8 within its 8, then 4 past it. acme: 8 + 4 = 12 within, web's 4
+	// takes the org to 16, past its 12.
+	if b["old"] || !b["new"] || !b["web"] {
+		t.Errorf("borrowed %v", b)
+	}
+}
+
+func TestAGuaranteeIsOnlyForWhoHasOne(t *testing.T) {
+	p := policy()
+	u := map[string]Amount{"team:ml": {GPUs: 4}, "org:acme": {GPUs: 4}}
+	if !p.Guaranteed("project:train", u, Amount{GPUs: 4}) {
+		t.Error("8 of ml's 8 and 8 of acme's 12 is within both")
+	}
+	if p.Guaranteed("project:train", u, Amount{GPUs: 5}) {
+		t.Error("9 GPUs is past ml's 8")
+	}
+	if p.Guaranteed("", u, Amount{GPUs: 1}) || New(nil, nil).Guaranteed("team:ml", u, Amount{}) {
+		t.Error("an owner with no guarantee is not guaranteed anything")
+	}
+}
+
+func TestALimitStopsGrowthWhereverItIsSet(t *testing.T) {
+	p := policy()
+	u := map[string]Amount{"org:acme": {GPUs: 14}}
+	if why := p.AtLimit("team:web", u, Amount{GPUs: 2}); why != "" {
+		t.Errorf("16 of 16 is at the limit, not past it: %s", why)
+	}
+	why := p.AtLimit("team:web", u, Amount{GPUs: 4})
+	if !strings.Contains(why, "org:acme is at its capacity limit (18 GPUs of 16)") {
+		t.Errorf("the refusal should name the scope and the figures: %q", why)
+	}
+	if p.AtLimit("", u, Amount{GPUs: 99}) != "" {
+		t.Error("a shared model was limited")
+	}
+}
@@ -0,0 +1,95 @@
+package control
+
+import (
+	"context"
+	"encoding/json"
+	"log/slog"
+	"sync"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/bench"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// QuotaStore is where capacity quotas and the tenancy tree are kept.
+type QuotaStore interface {
+	CapacityQuotas(ctx context.Context) ([]store.CapacityQuota, error)
+	ScopeParents(ctx context.Context) (map[string][]string, error)
+}
+
+// quotaRefresh is how stale the reconciler's view of quotas may get. A quota
+// changed by an operator takes effect within this; the reconciler asks many
+// times a pass and the store is not asked each time.
+const quotaRefresh = 30 * time.Second
+
+// QuotaPolicy returns what the reconciler asks for its quotas: the last
+// policy read from the store, read again once it is older than quotaRefresh.
+// A store that fails keeps the last good policy, so a database hiccup does
+// not drop every guarantee and limit at once.
+func QuotaPolicy(st QuotaStore, log *slog.Logger) func() *quota.Policy {
+	var mu sync.Mutex
+	var cur *quota.Policy
+	var at time.Time
+	return func() *quota.Policy {
+		mu.Lock()
+		defer mu.Unlock()
+		if cur != nil && time.Since(at) < quotaRefresh {
+			return cur
+		}
+		ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+		defer cancel()
+		qs, err := st.CapacityQuotas(ctx)
+		if err == nil {
+			var parents map[string][]string
+			if parents, err = st.ScopeParents(ctx); err == nil {
+				cur, at = quota.New(qs, parents), time.Now()
+				return cur
+			}
+		}
+		log.Warn("could not read capacity quotas; keeping the last ones", "err", err)
+		at = time.Now() // not again on every call while the store is down
+		return cur
+	}
+}
+
+// MeasurementStore is where benchmark results are kept.
+type MeasurementStore interface {
+	Capacities(ctx context.Context) ([]store.StoredCapacity, error)
+}
+
+// Measurements returns what the reconciler asks for benchmark results, keyed
+// model/profile, read again once older than quotaRefresh; like QuotaPolicy it
+// keeps the last good answer when the store fails.
+func Measurements(st MeasurementStore, log *slog.Logger) func() map[string]reconciler.Measured {
+	var mu sync.Mutex
+	var cur map[string]reconciler.Measured
+	var at time.Time
+	return func() map[string]reconciler.Measured {
+		mu.Lock()
+		defer mu.Unlock()
+		if cur != nil && time.Since(at) < quotaRefresh {
+			return cur
+		}
+		ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+		defer cancel()
+		at = time.Now()
+		rows, err := st.Capacities(ctx)
+		if err != nil {
+			log.Warn("could not read benchmark results; keeping the last ones", "err", err)
+			return cur
+		}
+		next := map[string]reconciler.Measured{}
+		for _, row := range rows {
+			var c bench.Capacity
+			if json.Unmarshal(row.Measurement, &c) != nil {
+				continue
+			}
+			next[row.Model+"/"+row.Profile] = reconciler.Measured{
+				MaxConcurrency: c.MaxConcurrencyAtSLO, TokPerSec: c.ThroughputTokPerSec}
+		}
+		cur = next
+		return cur
+	}
+}
@@ -144,6 +144,7 @@ func (r *Reconciler) mergeAdopted(rep *Replica, group []AdoptedShard) {
 		r.inv.Allocate(s.NodeID, inventory.Allocation{
 			ShardID: s.ShardID, ReplicaID: rep.ID, GPUIndices: s.GPUIndices,
 			MemoryGB: r.profileMemory(rep.Model, rep.Profile),
+			CPUCores: r.adoptedCores(rep.Model, rep.Profile),
 		})
 		// The endpoint comes from the leader, which may be in any of the
 		// groups — the rank 0 node is not necessarily the first to register.
@@ -253,3 +254,9 @@ type AdoptedShard struct {
 	// Endpoint is the leader's base URL, set on the rank 0 shard.
 	Endpoint string
 }
+
+// adoptedCores is what an adopted shard is taken to hold of its node's CPUs.
+func (r *Reconciler) adoptedCores(model, profile string) int {
+	p, _ := r.profileOf(model, profile)
+	return cpuCores(p)
+}
@@ -94,6 +94,9 @@ func (r *Reconciler) desiredFor(m catalog.Model) int {
 // baseFor is what a model runs without load: its minimum, or for an on-demand
 // model one while somebody is asking and none once they stop.
 func (r *Reconciler) baseFor(m catalog.Model) int {
+	if m.IsExternal() {
+		return 0 // served elsewhere: nothing to place, whatever its scaling says
+	}
 	if !m.Scaling.OnDemand() {
 		return m.Scaling.Min
 	}
@@ -157,8 +160,31 @@ type Load struct {
 // the last one is still starting: on a small CPU fleet each replica is minutes
 // of weights and gigabytes of memory, and a spike that is over by the time it
 // loads should not leave three behind.
+//
+// With a benchmark stored for the profile a model runs on, its measured
+// capacity is used too: more requests per replica than were measured to meet
+// the SLO is busy, even with the engines' queues empty — an engine will take
+// requests into its slots long after latency has gone — and a replica is only
+// taken back when the rest would still be under scaleDownHeadroom of what they
+// were measured to hold, so scaling down does not land on the edge.
 func (r *Reconciler) RecordLoad(loads map[string]Load) {
 	now := r.now()
+	// The profile each model's replicas run on, for its measurement; read
+	// before the loop, which holds the lock.
+	running := map[string]string{}
+	r.mu.Lock()
+	for _, rep := range r.replicas {
+		if rep.State.Counts() {
+			running[rep.Model] = rep.Profile
+		}
+	}
+	r.mu.Unlock()
+	measured := map[string]Measured{}
+	for model, profile := range running {
+		if m, ok := r.measured(model, profile); ok {
+			measured[model] = m
+		}
+	}
 	for _, m := range r.cat.Load().Models() {
 		base := r.baseFor(m)
 		r.mu.Lock()
@@ -178,8 +204,13 @@ func (r *Reconciler) RecordLoad(loads map[string]Load) {
 		l := loads[m.Name]
 		perReplica := float64(l.Waiting) / float64(max(l.Replicas, 1))
 		extra := r.extra[m.Name]
+		meas, hasMeas := measured[m.Name]
+		overMeasured := hasMeas &&
+			float64(l.Running+l.Waiting) > float64(meas.MaxConcurrency*max(l.Replicas, 1))
+		fitsInFewer := !hasMeas ||
+			float64(l.Running) <= scaleDownHeadroom*float64(meas.MaxConcurrency*max(l.Replicas-1, 0))
 		switch {
-		case perReplica >= r.cfg.ScaleUpWaiting:
+		case perReplica >= r.cfg.ScaleUpWaiting || overMeasured:
 			delete(r.quietSince, m.Name)
 			since, ok := r.busySince[m.Name]
 			if !ok {
@@ -190,9 +221,10 @@ func (r *Reconciler) RecordLoad(loads map[string]Load) {
 				r.extra[m.Name] = extra + 1
 				r.busySince[m.Name] = now
 				r.log.Info("scaling up for load", "model", m.Name, "replicas", base+extra+1,
-					"waiting_per_replica", perReplica, "for", now.Sub(since).Round(time.Second))
+					"waiting_per_replica", perReplica, "in_flight", l.Running+l.Waiting,
+					"measured_per_replica", meas.MaxConcurrency, "for", now.Sub(since).Round(time.Second))
 			}
-		case l.Waiting == 0 && extra > 0:
+		case l.Waiting == 0 && extra > 0 && fitsInFewer:
 			delete(r.busySince, m.Name)
 			since, ok := r.quietSince[m.Name]
 			if !ok {
@@ -212,3 +244,8 @@ func (r *Reconciler) RecordLoad(loads map[string]Load) {
 		r.mu.Unlock()
 	}
 }
+
+// scaleDownHeadroom is how full, of their measured capacity, the replicas left
+// after a scale-down may be. Below one so that the load that was there a
+// minute ago does not put them straight back over.
+const scaleDownHeadroom = 0.7
@@ -5,6 +5,7 @@ import (

 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/placement"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
 )

 // Headroom is how many more replicas of a model the cluster has room for now.
@@ -24,6 +25,14 @@ type Headroom struct {
 	// Blocked is why one more than More does not fit, a line per node that
 	// was considered. Empty when Limited.
 	Blocked []string
+	// TokPerSec is what one more replica was measured to serve, on the
+	// profile placement would give it; zero when it was never measured.
+	TokPerSec float64
+	// ByPreempting is how many more, beyond More, would fit by stopping
+	// replicas the model's class may take, and Preempts the models those
+	// replicas belong to: what "room" means for a model that outranks others.
+	ByPreempting int
+	Preempts     []string
 }

 // Headroom tries placing up to limit more replicas of a model on a copy of
@@ -52,11 +61,42 @@ func (r *Reconciler) Headroom(model string, limit int) (Headroom, bool) {

 	inv := r.inv.Clone()
 	sched := placement.New(inv)
-	for h.More < limit {
-		res := sched.Place(m.PlacementProfiles(), avoid)
+	pol := r.policy()
+	var usage map[string]quota.Amount
+	if pol != nil {
+		usage = pol.Usage(r.Holdings())
+	}
+	place := func() placement.Result {
+		res := sched.Place(r.orderedProfiles(m), avoid)
 		if res.Placement == nil && len(avoid) > 0 {
 			// As scaleUp does: spreading is a preference, not a rule.
-			res = sched.Place(m.PlacementProfiles(), nil)
+			res = sched.Place(r.orderedProfiles(m), nil)
+		}
+		return res
+	}
+	// What the model's class may take, in the order preemption would, each
+	// released on the copy only when free room has run out.
+	victims, nextVictim, preempting := r.victims(m), 0, false
+	for h.More+h.ByPreempting < limit {
+		res := place()
+		// Named only once they have made room: candidates tried on the way
+		// to finding there is none would otherwise read as whose replicas a
+		// placement would stop.
+		var tried []string
+		for res.Placement == nil && nextVictim < len(victims) {
+			v := victims[nextVictim]
+			nextVictim++
+			inv.ReleaseReplica(v.ID)
+			tried = append(tried, v.Model)
+			res = place()
+		}
+		if res.Placement != nil && len(tried) > 0 {
+			preempting = true
+			for _, m := range tried {
+				if !slices.Contains(h.Preempts, m) {
+					h.Preempts = append(h.Preempts, m)
+				}
+			}
 		}
 		if res.Placement == nil {
 			for _, rej := range res.Rejections {
@@ -66,20 +106,42 @@ func (r *Reconciler) Headroom(model string, limit int) (Headroom, bool) {
 			h.Blocked = slices.Compact(h.Blocked)
 			return h, true
 		}
-		mem := 0
+		// A quota limit is as much an edge as a full node.
+		if pol != nil && m.Owner != "" {
+			need := placementNeed(m, res.Placement)
+			if why := pol.AtLimit(m.Owner, usage, need); why != "" {
+				h.Blocked = []string{why}
+				return h, true
+			}
+			for _, s := range pol.Chain(m.Owner) {
+				usage[s] = usage[s].Plus(need)
+			}
+		}
+		mem, cores := 0, 0
 		if prof, ok := m.ProfileByName(res.Placement.Profile.Name); ok {
-			mem = prof.MinMemoryGB
+			mem, cores = prof.MinMemoryGB, cpuCores(prof)
+		}
+		if h.More == 0 {
+			if meas, ok := r.measured(m.Name, res.Placement.Profile.Name); ok {
+				h.TokPerSec = meas.TokPerSec
+			}
 		}
 		var nodes []string
 		for _, a := range res.Placement.Nodes {
 			inv.Allocate(a.NodeID, inventory.Allocation{
-				ShardID: "headroom", ReplicaID: "headroom", GPUIndices: a.GPUIndices, MemoryGB: mem,
+				ShardID: "headroom", ReplicaID: "headroom", GPUIndices: a.GPUIndices, MemoryGB: mem, CPUCores: cores,
 			})
 			nodes = append(nodes, a.NodeID)
 		}
 		avoid = append(avoid, nodes...)
-		h.Nodes = append(h.Nodes, nodes)
-		h.More++
+		// Once anything was stopped, what fits after is counted as fitting
+		// by preempting: it may be sitting in the room that was freed.
+		if preempting {
+			h.ByPreempting++
+		} else {
+			h.Nodes = append(h.Nodes, nodes)
+			h.More++
+		}
 	}
 	h.Limited = true
 	return h, true
@@ -57,3 +57,24 @@ func TestHeadroomCountsMemoryAlreadyClaimed(t *testing.T) {
 		t.Errorf("headroom %+v, expected room for 3 more", got)
 	}
 }
+
+func TestReservedCoresAreTheEnginesThreadsAndBoundTheRoom(t *testing.T) {
+	cpu := inventory.Node{ID: "a", MgmtAddr: "10.0.0.1", State: inventory.Ready, MemoryTotalMB: 64 * 1024, CPUCount: 8}
+	m := catalog.Model{Name: "coder", Source: "/m.gguf", Scaling: catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 6, CPUCores: 3}}}
+	h := newHarness(t, m, cpu)
+	h.r.Reconcile(t.Context())
+
+	started := h.disp.startedShards()
+	if len(started) != 1 || started[0].Spec.Threads != 3 {
+		t.Fatalf("the engine was not told its 3 cores: %+v", started)
+	}
+	if a := h.inv.Allocations("a"); len(a) != 1 || a[0].CPUCores != 3 {
+		t.Errorf("the cores were not reserved: %+v", a)
+	}
+	// Memory would allow nine more; cores allow one: 8 - 3 = 5, then 2.
+	got, _ := h.r.Headroom("coder", 10)
+	if got.More != 1 {
+		t.Errorf("room for %d more, expected 1 by cores: %+v", got.More, got)
+	}
+}
@@ -83,6 +83,12 @@ type ModelHealth struct {
 	// place one found nowhere; Placement says why.
 	Unplaceable bool
 	Placement   string
+	// Class is the capacity class it is treated as. Preempted counts the
+	// replicas it has lost to models of a higher class since the control
+	// plane started; HeldFor names the model it is giving room to now.
+	Class     string
+	Preempted int
+	HeldFor   string
 }

 // ModelHealth reports every model in the catalog.
@@ -106,7 +112,10 @@ func (r *Reconciler) ModelHealth() []ModelHealth {
 	out := make([]ModelHealth, 0, len(models))
 	for _, m := range models {
 		h := ModelHealth{Model: m.Name, Desired: desired[m.Name], Ready: ready[m.Name],
-			Failures: r.attempts[m.Name]}
+			Failures: r.attempts[m.Name], Class: m.EffectiveClass(), Preempted: r.preempted[m.Name]}
+		if hd, ok := r.held[m.Name]; ok && r.now().Before(hd.Until) {
+			h.HeldFor = hd.For
+		}
 		if p, ok := r.parked[m.Name]; ok {
 			h.Parked, h.ParkedSince, h.Reason = true, p.Since, p.Reason
 		}
@@ -0,0 +1,108 @@
+package reconciler
+
+import (
+	"sort"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/placement"
+)
+
+// Measured is what one replica of a model on a profile was measured to serve
+// while meeting its SLO, by `llmgw ctl bench --save`.
+type Measured struct {
+	// MaxConcurrency is the most requests at once that still met the SLO.
+	MaxConcurrency int
+	// TokPerSec is the throughput at that concurrency.
+	TokPerSec float64
+}
+
+// SetMeasurements supplies benchmark results, keyed model/profile, read
+// afresh each time it is called. Without them scaling goes by queue alone and
+// profiles are chosen by priority alone.
+func (r *Reconciler) SetMeasurements(f func() map[string]Measured) {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	r.measurements = f
+}
+
+// measured is a model's measurement on one profile.
+func (r *Reconciler) measured(model, profile string) (Measured, bool) {
+	r.mu.Lock()
+	f := r.measurements
+	r.mu.Unlock()
+	if f == nil {
+		return Measured{}, false
+	}
+	m, ok := f()[model+"/"+profile]
+	return m, ok && m.MaxConcurrency > 0
+}
+
+// orderedProfiles is a model's profiles as placement should try them: by
+// priority, and among equal priorities the one measured to serve the most per
+// GPU — or per core, for a profile with none — first. Unmeasured profiles keep
+// their catalog order behind measured ones of the same priority.
+//
+// Placement sorts by priority itself, so the order found here is written back
+// as priorities, descending, for it to keep.
+func (r *Reconciler) orderedProfiles(m catalog.Model) []placement.Profile {
+	ps := m.PlacementProfiles()
+	eff := make([]float64, len(ps))
+	for i, p := range ps {
+		meas, ok := r.measured(m.Name, p.Name)
+		if !ok {
+			continue
+		}
+		units := float64(max(p.GPUsPerNode, 0) * max(p.Nodes, 1))
+		if units == 0 {
+			units = float64(max(p.CPUCores, 1))
+		}
+		eff[i] = meas.TokPerSec / units
+	}
+	idx := make([]int, len(ps))
+	for i := range idx {
+		idx[i] = i
+	}
+	sort.SliceStable(idx, func(a, b int) bool {
+		pa, pb := ps[idx[a]], ps[idx[b]]
+		if pa.Priority != pb.Priority {
+			return pa.Priority > pb.Priority
+		}
+		return eff[idx[a]] > eff[idx[b]]
+	})
+	out := make([]placement.Profile, len(ps))
+	for rank, i := range idx {
+		out[rank] = ps[i]
+		out[rank].Priority = len(ps) - rank
+	}
+	return out
+}
+
+// MeasuredFor is the measurement that sizes a model: that of the profile its
+// replicas run on, or, with none running, of the first profile placement
+// would try that was measured.
+func (r *Reconciler) MeasuredFor(model string) (Measured, string, bool) {
+	r.mu.Lock()
+	profile := ""
+	for _, rep := range r.replicas {
+		if rep.Model == model && rep.State.Counts() {
+			profile = rep.Profile
+			break
+		}
+	}
+	r.mu.Unlock()
+	if profile != "" {
+		if m, ok := r.measured(model, profile); ok {
+			return m, profile, true
+		}
+	}
+	cm, ok := r.cat.Load().Resolve(model)
+	if !ok {
+		return Measured{}, "", false
+	}
+	for _, p := range r.orderedProfiles(cm) {
+		if m, ok := r.measured(cm.Name, p.Name); ok {
+			return m, p.Name, true
+		}
+	}
+	return Measured{}, "", false
+}
@@ -0,0 +1,96 @@
+package reconciler
+
+import (
+	"testing"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+)
+
+func measure(h *harness, byKey map[string]Measured) {
+	h.r.SetMeasurements(func() map[string]Measured { return byKey })
+}
+
+func TestMoreRequestsThanWereMeasuredToMeetTheSLOIsBusy(t *testing.T) {
+	m := testModel(1)
+	m.Scaling.Max = 2
+	h := newHarness(t, m, gpuNode("1", 8), gpuNode("2", 8))
+	h.r.Reconcile(t.Context())
+	h.markReady()
+	// Six in flight, nothing waiting at the engine: by the queue it is fine.
+	load := map[string]Load{"llama": {Replicas: 1, Running: 6}}
+	hold := func() {
+		h.r.RecordLoad(load)
+		h.advance(61 * time.Second)
+		h.r.RecordLoad(load)
+		h.r.Reconcile(t.Context())
+	}
+	hold()
+	if n := liveCount(h); n != 1 {
+		t.Fatalf("scaled up with no measurement and an empty queue: %d", n)
+	}
+	// Measured to meet its SLO at four at once: six is past it.
+	measure(h, map[string]Measured{"llama/tp4": {MaxConcurrency: 4, TokPerSec: 200}})
+	hold()
+	if n := liveCount(h); n != 2 {
+		t.Errorf("%d replicas with 6 in flight against a measured 4", n)
+	}
+}
+
+func TestAReplicaIsTakenBackOnlyIfTheRestWouldHaveRoomToSpare(t *testing.T) {
+	m := testModel(1)
+	m.Scaling.Max = 2
+	h := newHarness(t, m, gpuNode("1", 8), gpuNode("2", 8))
+	measure(h, map[string]Measured{"llama/tp4": {MaxConcurrency: 4, TokPerSec: 200}})
+	h.r.Reconcile(t.Context())
+	h.markReady()
+	h.r.mu.Lock()
+	h.r.extra["llama"] = 1
+	h.r.mu.Unlock()
+	h.r.Reconcile(t.Context())
+	h.markReady()
+	if liveCount(h) != 2 {
+		t.Fatal("setup: not at two replicas")
+	}
+	quietFor := func(l Load) {
+		for range 3 {
+			h.r.RecordLoad(map[string]Load{"llama": l})
+			h.advance(DefaultConfig().ScaleDownAfter + time.Second)
+			h.r.RecordLoad(map[string]Load{"llama": l})
+			h.r.Reconcile(t.Context())
+		}
+	}
+	// Nothing waiting, but three running: one replica left would be at 3 of
+	// its 4, past 70%.
+	quietFor(Load{Replicas: 2, Running: 3})
+	if n := liveCount(h); n != 2 {
+		t.Errorf("scaled down to the edge of what one replica holds: %d", n)
+	}
+	quietFor(Load{Replicas: 2, Running: 2})
+	if n := liveCount(h); n != 1 {
+		t.Errorf("two running fits one replica's 70%% of 4; still %d", n)
+	}
+}
+
+func TestTheProfileMeasuredToServeMostPerGPUIsTriedFirst(t *testing.T) {
+	m := catalog.Model{Name: "llama", Source: "/models/llama", Scaling: catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{
+			{Name: "tp4", GPUClass: "h100-80g", TP: 4},
+			{Name: "tp2", GPUClass: "h100-80g", TP: 2},
+			{Name: "tp8", GPUClass: "h100-80g", TP: 8, Priority: 10},
+		}}
+	h := newHarness(t, m, gpuNode("1", 8))
+	// tp4: 400/4 = 100 per GPU; tp2: 300/2 = 150. tp8 is unmeasured but has
+	// the higher priority, which an operator set and a benchmark does not
+	// overrule.
+	measure(h, map[string]Measured{"llama/tp4": {MaxConcurrency: 8, TokPerSec: 400},
+		"llama/tp2": {MaxConcurrency: 6, TokPerSec: 300}})
+	cm, _ := h.r.cat.Load().Resolve("llama")
+	var order []string
+	for _, p := range h.r.orderedProfiles(cm) {
+		order = append(order, p.Name)
+	}
+	if len(order) != 3 || order[0] != "tp8" || order[1] != "tp2" || order[2] != "tp4" {
+		t.Errorf("order %v, expected tp8 (priority), then tp2 (150/GPU), then tp4 (100/GPU)", order)
+	}
+}
@@ -0,0 +1,324 @@
+package reconciler
+
+import (
+	"context"
+	"slices"
+	"sort"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/placement"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
+)
+
+// hold keeps a model that gave up room from taking it back.
+type hold struct {
+	Until time.Time
+	For   string
+}
+
+// maxVictims bounds how many replicas one placement may stop. Past this the
+// model is better reported as not fitting than paid for with half a cluster.
+const maxVictims = 3
+
+// victim is a replica that could be stopped for another model.
+type victim struct {
+	ID       string
+	Model    string
+	Class    string
+	AboveMin bool
+	Tier     int
+	Idle     time.Duration
+	// Size is what stopping it frees, for choosing the smaller of two
+	// otherwise equal victims.
+	Size quota.Amount
+}
+
+// preemptFor stops replicas of lower classes when a model cannot otherwise be
+// placed, and says whether it did.
+//
+// What may be taken from whom:
+//
+//   - A lower class gives way to a higher one: opportunistic to standard,
+//     standard to critical.
+//   - Within a class, only a replica above its model's minimum, and only for
+//     a model still below its own. Room a model grew into under load is not a
+//     promise; its minimum is.
+//   - A critical model's minimum is never taken.
+//
+// Victims only ever come from a lower rank, or from above a minimum for a
+// model below one, so two models cannot take turns stopping each other.
+//
+// The set is tried on a copy of the inventory first, and only stopped if it
+// would let the model be placed: stopping replicas that do not make room is
+// the one outcome worse than not preempting.
+func (r *Reconciler) preemptFor(ctx context.Context, m catalog.Model, avoid []string) bool {
+	cands := r.victims(m)
+	if len(cands) == 0 {
+		return false
+	}
+	// Candidates in order until the model fits; then every one whose
+	// stopping turns out not to be needed is kept after all. Without that
+	// second step an idle replica on the wrong node, tried first, would be
+	// stopped along with the one that actually made the room.
+	var chosen []victim
+	for _, v := range cands {
+		if len(chosen) == maxVictims {
+			break
+		}
+		chosen = append(chosen, v)
+		if !r.fitsWithout(m, avoid, chosen) {
+			continue
+		}
+		for i := len(chosen) - 1; i >= 0 && len(chosen) > 1; i-- {
+			fewer := append(append([]victim(nil), chosen[:i]...), chosen[i+1:]...)
+			if r.fitsWithout(m, avoid, fewer) {
+				chosen = fewer
+			}
+		}
+		r.preempt(ctx, m, chosen)
+		return true
+	}
+	return false
+}
+
+// fitsWithout reports whether m could be placed with these replicas gone,
+// tried on a copy of the inventory.
+func (r *Reconciler) fitsWithout(m catalog.Model, avoid []string, gone []victim) bool {
+	trial := r.inv.Clone()
+	for _, v := range gone {
+		trial.ReleaseReplica(v.ID)
+	}
+	sched := placement.New(trial)
+	res := sched.Place(r.orderedProfiles(m), avoid)
+	if res.Placement == nil && len(avoid) > 0 {
+		res = sched.Place(r.orderedProfiles(m), nil)
+	}
+	return res.Placement != nil
+}
+
+// preempt drains the chosen replicas and holds their models back.
+func (r *Reconciler) preempt(ctx context.Context, m catalog.Model, chosen []victim) {
+	now := r.now()
+	var ids []string
+	r.mu.Lock()
+	for _, v := range chosen {
+		if rep, ok := r.replicas[v.ID]; ok {
+			rep.PreemptedFor = m.Name
+		}
+		ids = append(ids, v.ID)
+		r.held[v.Model] = hold{Until: now.Add(r.cfg.PreemptHold), For: m.Name}
+		r.preempted[v.Model]++
+	}
+	r.preempting[m.Name] = ids
+	r.mu.Unlock()
+
+	for _, v := range chosen {
+		r.log.Warn("preempting a replica to make room",
+			"replica", v.ID, "model", v.Model, "class", v.Class, "above_min", v.AboveMin,
+			"for", m.Name, "for_class", m.EffectiveClass())
+	}
+	r.scaleDown(ctx, ids)
+}
+
+// awaitingPreemption reports whether replicas stopped for a model are still
+// on their way out, so it waits for them rather than stopping more.
+func (r *Reconciler) awaitingPreemption(model string) bool {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	ids := r.preempting[model]
+	ids = slices.DeleteFunc(slices.Clone(ids), func(id string) bool {
+		_, ok := r.replicas[id]
+		return !ok
+	})
+	if len(ids) == 0 {
+		delete(r.preempting, model)
+		return false
+	}
+	r.preempting[model] = ids
+	return true
+}
+
+// isHeld reports whether a model gave up room and may not take it back yet.
+// The hold ends early once the model it was given to has what it asked for.
+//
+// Today a preempted replica is only removed by its own model's drain loop,
+// which runs after that model has decided not to scale up, so the room it
+// frees reaches the model it was freed for first anyway. The hold is what
+// keeps that true for any other way a replica can go — a stop reported
+// between passes, a reap at the start of one — rather than it resting on the
+// order of lines in reconcileModel.
+func (r *Reconciler) isHeld(model string) bool {
+	r.mu.Lock()
+	h, ok := r.held[model]
+	r.mu.Unlock()
+	if !ok {
+		return false
+	}
+	if r.now().After(h.Until) {
+		r.mu.Lock()
+		delete(r.held, model)
+		r.mu.Unlock()
+		return false
+	}
+	if to, ok := r.cat.Load().Resolve(h.For); ok && r.liveCount(h.For) >= r.desiredFor(to) {
+		r.mu.Lock()
+		delete(r.held, model)
+		r.mu.Unlock()
+		return false
+	}
+	return true
+}
+
+// liveCount is how many replicas of a model hold capacity.
+func (r *Reconciler) liveCount(model string) int {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	n := 0
+	for _, rep := range r.replicas {
+		if rep.Model == model && rep.State.Counts() && rep.RetiringAt.IsZero() {
+			n++
+		}
+	}
+	return n
+}
+
+// victims lists the replicas that may be stopped for m, in the order they
+// should be: idle on-demand ones first, then what other tenants borrowed past
+// their guarantees when m's owner is within its own, then what lower classes
+// grew into under load, then the same within m's class, then lower classes'
+// minimums; within each, the longest unused first.
+//
+// A replica within its owner's guarantee is taken only for a model of the
+// same owner: a guarantee that another tenant's higher class could take is
+// not one.
+func (r *Reconciler) victims(m catalog.Model) []victim {
+	cat := r.cat.Load()
+	now := r.now()
+	myRank := catalog.ClassRank(m.EffectiveClass())
+	belowMin := r.liveCount(m.Name) < m.Scaling.Min
+
+	pol := r.policy()
+	var borrowed map[string]bool
+	reclaim := false
+	if pol != nil {
+		hs := r.Holdings()
+		borrowed = pol.Borrowed(hs)
+		reclaim = pol.Guaranteed(m.Owner, pol.Usage(hs), profileNeed(m))
+	}
+
+	r.mu.Lock()
+	defer r.mu.Unlock()
+
+	// Per model, its live replicas newest first: the newest are the ones
+	// above its minimum.
+	byModel := map[string][]*Replica{}
+	for _, rep := range r.replicas {
+		if rep.Model == m.Name || !rep.State.Counts() || !rep.RetiringAt.IsZero() || rep.PreemptedFor != "" {
+			continue
+		}
+		byModel[rep.Model] = append(byModel[rep.Model], rep)
+	}
+
+	var out []victim
+	for model, reps := range byModel {
+		other, ok := cat.Resolve(model)
+		if !ok {
+			continue
+		}
+		class := other.EffectiveClass()
+		rank := catalog.ClassRank(class)
+		sort.Slice(reps, func(i, j int) bool { return reps[i].CreatedAt.After(reps[j].CreatedAt) })
+		above := max(len(reps)-other.Scaling.Min, 0)
+		for i, rep := range reps {
+			aboveMin := i < above
+			tier := -1
+			switch {
+			case rank < myRank && other.Scaling.OnDemand() && idleFor(rep, now) >= r.cfg.EvictionGrace:
+				tier = 0
+			case rank < myRank && aboveMin:
+				tier = 2
+			case rank == myRank && aboveMin && belowMin:
+				tier = 3
+			case rank < myRank && class != catalog.ClassCritical:
+				tier = 4
+			}
+			if pol != nil && other.Owner != m.Owner {
+				switch {
+				case borrowed[rep.ID] && reclaim && (class != catalog.ClassCritical || aboveMin):
+					// Past its owner's guarantee, wanted by an owner within
+					// its own: taken back whatever the classes.
+					if tier != 0 {
+						tier = 1
+					}
+				case !borrowed[rep.ID] && guaranteed(pol, other.Owner):
+					continue // held within its owner's guarantee
+				}
+			}
+			if tier < 0 {
+				continue
+			}
+			// Just started: stopping it throws away a model load that has
+			// only now been paid for.
+			if !rep.ReadyAt.IsZero() && now.Sub(rep.ReadyAt) < r.cfg.MinLifetime {
+				continue
+			}
+			if len(nodesOf(rep)) > 1 {
+				continue // see makeRoomFor: a multi-node replica is its own decision
+			}
+			prof, _ := other.ProfileByName(rep.Profile)
+			var size quota.Amount
+			for _, sh := range rep.Shards {
+				size = size.Plus(quota.Amount{GPUs: len(sh.GPUIndices), CPUCores: cpuCores(prof), MemoryGB: prof.MinMemoryGB})
+			}
+			out = append(out, victim{ID: rep.ID, Model: model, Class: class,
+				AboveMin: aboveMin, Tier: tier, Idle: idleFor(rep, now), Size: size})
+		}
+	}
+	sort.SliceStable(out, func(i, j int) bool {
+		a, b := out[i], out[j]
+		if a.Tier != b.Tier {
+			return a.Tier < b.Tier
+		}
+		if a.Idle != b.Idle {
+			return a.Idle > b.Idle
+		}
+		// The smaller first: the least served is lost if it is enough, and
+		// the pruning after keeps a larger one only when it is needed.
+		if a.Size.GPUs != b.Size.GPUs {
+			return a.Size.GPUs < b.Size.GPUs
+		}
+		if a.Size.MemoryGB != b.Size.MemoryGB {
+			return a.Size.MemoryGB < b.Size.MemoryGB
+		}
+		return a.ID < b.ID
+	})
+	return out
+}
+
+// idleFor is how long a replica has gone unused; zero when nobody has said,
+// which is not the same as idle.
+func idleFor(rep *Replica, now time.Time) time.Duration {
+	if rep.LastUsedAt.IsZero() {
+		return 0
+	}
+	return now.Sub(rep.LastUsedAt)
+}
+
+func nodesOf(rep *Replica) map[string]struct{} {
+	out := map[string]struct{}{}
+	for _, sh := range rep.Shards {
+		out[sh.NodeID] = struct{}{}
+	}
+	return out
+}
+
+// guaranteed reports whether an owner has a guarantee anywhere up its tree.
+func guaranteed(pol *quota.Policy, owner string) bool {
+	for _, q := range pol.Governing(owner) {
+		if q.Guarantee.Any() {
+			return true
+		}
+	}
+	return false
+}
@@ -0,0 +1,418 @@
+package reconciler
+
+import (
+	"log/slog"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
+)
+
+// newFleetHarness is newHarness with a catalog of several models.
+func newFleetHarness(t *testing.T, models []catalog.Model, nodes ...inventory.Node) *harness {
+	t.Helper()
+	cat, err := catalog.New(models)
+	if err != nil {
+		t.Fatal(err)
+	}
+	inv := inventory.NewStore()
+	c := newClock()
+	for _, n := range nodes {
+		inv.Upsert(n)
+	}
+	disp := &fakeDispatcher{}
+	cfg := DefaultConfig()
+	cfg.UnreachableAfter = 15 * time.Second
+	cfg.LostAfter = 60 * time.Second
+	r := New(cfg, catalog.NewLive(cat), inv, disp, slog.New(slog.DiscardHandler))
+	r.SetClock(c.now)
+	c.advance(cfg.SettleFor)
+	for _, n := range nodes {
+		touchAt(inv, n.ID, c.now())
+	}
+	return &harness{r: r, inv: inv, disp: disp, clock: c}
+}
+
+// gpuModel runs on one whole 8-GPU node.
+func gpuModel(name, class string, min int) catalog.Model {
+	return catalog.Model{Name: name, Source: "/models/" + name, Class: class,
+		Scaling:  catalog.Scaling{Min: min, Max: min},
+		Profiles: []catalog.Profile{{Name: "tp8", Engine: "vllm", GPUClass: "h100-80g", TP: 8}}}
+}
+
+// settle runs passes until nothing changes, marking what starts ready and
+// letting preempted replicas run out their grace.
+func (h *harness) settle(t *testing.T, passes int) {
+	t.Helper()
+	for range passes {
+		h.r.Reconcile(t.Context())
+		h.markReady()
+		h.advance(h.r.cfg.PreemptGrace + time.Second)
+	}
+}
+
+func replicasOf(h *harness, model string) (live int) {
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == model && rep.State.Counts() {
+			live++
+		}
+	}
+	return live
+}
+
+// fill places the standard model, then ages it past MinLifetime so it may be
+// taken from, and only then adds the model that wants its node.
+func fillThenWant(t *testing.T, want catalog.Model, have catalog.Model, nodes ...inventory.Node) *harness {
+	t.Helper()
+	h := newFleetHarness(t, []catalog.Model{have, withMin(want, 0)}, nodes...)
+	h.settle(t, 3)
+	if got := replicasOf(h, have.Name); got != have.Scaling.Min {
+		t.Fatalf("%s has %d replicas before anything wanted its room", have.Name, got)
+	}
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	cat, err := catalog.New([]catalog.Model{have, want})
+	if err != nil {
+		t.Fatal(err)
+	}
+	h.r.cat.Store(cat)
+	return h
+}
+
+func withMin(m catalog.Model, n int) catalog.Model {
+	m.Scaling = catalog.Scaling{Min: n, Max: n}
+	return m
+}
+
+func TestACriticalModelTakesAStandardModelsNode(t *testing.T) {
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+
+	h.r.Reconcile(t.Context())
+	var draining int
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == "chat" && rep.State == Draining && rep.PreemptedFor == "vital" {
+			draining++
+		}
+	}
+	if draining != 1 {
+		t.Fatalf("the standard replica was not drained for the critical one: %+v", h.r.Replicas())
+	}
+	if hs := health(h, "chat"); hs.HeldFor != "vital" || hs.Preempted != 1 {
+		t.Errorf("the victim's health does not say it gave way: %+v", hs)
+	}
+
+	h.settle(t, 4)
+	if replicasOf(h, "vital") != 1 {
+		t.Fatalf("the critical model was not placed once room was made: %+v", h.r.Replicas())
+	}
+	// Held back until the critical model had its replica, then short again
+	// because there is no room: it must not have taken the node back — not
+	// even for a moment, which would load a model only to preempt it again.
+	if replicasOf(h, "chat") != 0 {
+		t.Error("the standard model took the node back")
+	}
+	started := 0
+	for _, cmd := range h.disp.startedShards() {
+		if strings.HasPrefix(cmd.ReplicaID, "chat-") {
+			started++
+		}
+	}
+	if started != 1 {
+		t.Errorf("the standard model started %d replicas; it should not have tried again", started)
+	}
+}
+
+func TestAStandardModelDoesNotTakeAnothersMinimum(t *testing.T) {
+	h := fillThenWant(t, gpuModel("second", "", 1), gpuModel("first", "", 1), gpuNode("1", 8))
+	h.settle(t, 3)
+	if replicasOf(h, "first") != 1 || replicasOf(h, "second") != 0 {
+		t.Errorf("a model of the same class took another's minimum: %+v", h.r.Replicas())
+	}
+	if hs := health(h, "second"); !hs.Unplaceable {
+		t.Error("the model left without room should say it cannot be placed")
+	}
+}
+
+func TestWithinAClassOnlyWhatWasGrownIntoIsTaken(t *testing.T) {
+	grown := gpuModel("busy", "", 1)
+	grown.Scaling = catalog.Scaling{Min: 1, Max: 2}
+	h := newFleetHarness(t, []catalog.Model{grown, withMin(gpuModel("new", "", 1), 0)}, gpuNode("1", 8), gpuNode("2", 8))
+	h.settle(t, 2)
+	// Load grows it to its max: the second replica is above its minimum.
+	h.r.mu.Lock()
+	h.r.extra["busy"] = 1
+	h.r.mu.Unlock()
+	h.settle(t, 3)
+	if replicasOf(h, "busy") != 2 {
+		t.Fatalf("setup: busy did not grow to 2: %+v", h.r.Replicas())
+	}
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	cat, _ := catalog.New([]catalog.Model{grown, gpuModel("new", "", 1)})
+	h.r.cat.Store(cat)
+
+	h.settle(t, 5)
+	if replicasOf(h, "new") != 1 || replicasOf(h, "busy") != 1 {
+		t.Errorf("the grown replica should give way to a minimum: busy %d, new %d",
+			replicasOf(h, "busy"), replicasOf(h, "new"))
+	}
+}
+
+func TestNothingIsStoppedThatWouldNotMakeRoom(t *testing.T) {
+	// Two nodes, each half used by an opportunistic model; the critical one
+	// needs a whole node. Stopping one frees half a node, which is no use —
+	// and both together are on two nodes, which is no use either.
+	half := catalog.Model{Name: "half", Source: "/m", Class: catalog.ClassOpportunistic,
+		Scaling:  catalog.Scaling{Min: 2, Max: 2},
+		Profiles: []catalog.Profile{{Name: "tp4", Engine: "vllm", GPUClass: "h100-80g", TP: 4}}}
+	five := func(id string) inventory.Node { return gpuNode(id, 5) }
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), half, five("1"), five("2"))
+	h.r.Reconcile(t.Context())
+	for _, rep := range h.r.Replicas() {
+		if rep.PreemptedFor != "" || rep.State == Draining {
+			t.Fatalf("a replica was stopped without making room: %+v", rep)
+		}
+	}
+}
+
+func TestACriticalMinimumIsNeverTaken(t *testing.T) {
+	h := fillThenWant(t, gpuModel("also-vital", catalog.ClassCritical, 1), gpuModel("vital", catalog.ClassCritical, 1), gpuNode("1", 8))
+	h.settle(t, 3)
+	if replicasOf(h, "vital") != 1 {
+		t.Error("a critical model's minimum was taken")
+	}
+}
+
+func TestAReplicaThatJustStartedIsLeftAlone(t *testing.T) {
+	h := newFleetHarness(t, []catalog.Model{gpuModel("chat", "", 1), withMin(gpuModel("vital", catalog.ClassCritical, 1), 0)}, gpuNode("1", 8))
+	h.settle(t, 2)
+	cat, _ := catalog.New([]catalog.Model{gpuModel("chat", "", 1), gpuModel("vital", catalog.ClassCritical, 1)})
+	h.r.cat.Store(cat)
+	h.r.Reconcile(t.Context())
+	for _, rep := range h.r.Replicas() {
+		if rep.State == Draining {
+			t.Fatal("a replica that had only just loaded was preempted")
+		}
+	}
+}
+
+func health(h *harness, model string) ModelHealth {
+	for _, x := range h.r.ModelHealth() {
+		if x.Model == model {
+			return x
+		}
+	}
+	return ModelHealth{}
+}
+
+func TestAHoldLastsUntilTheRoomIsUsedOrItExpires(t *testing.T) {
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+	h.r.Reconcile(t.Context())
+	if !h.r.isHeld("chat") {
+		t.Fatal("the model that gave way is not held")
+	}
+	if h.r.isHeld("vital") {
+		t.Error("the model room was made for is held")
+	}
+	// Released as soon as the model it was given to has what it asked for.
+	h.settle(t, 3)
+	if replicasOf(h, "vital") != 1 {
+		t.Fatalf("setup: vital was not placed: %+v", h.r.Replicas())
+	}
+	if h.r.isHeld("chat") {
+		t.Error("the hold outlasted its purpose")
+	}
+
+	// And, whatever happens, after PreemptHold.
+	h.r.mu.Lock()
+	h.r.held["other"] = hold{Until: h.r.now().Add(time.Minute), For: "never-placed"}
+	h.r.mu.Unlock()
+	if !h.r.isHeld("other") {
+		t.Fatal("a fresh hold is not in force")
+	}
+	h.advance(2 * time.Minute)
+	if h.r.isHeld("other") {
+		t.Error("a hold outlived its time")
+	}
+}
+
+func TestOnlyTheReplicasThatMakeTheRoomAreStopped(t *testing.T) {
+	// Two half-used nodes. "idle" on node 1 is on demand and long unused, so
+	// it is first in line; but the critical model needs node 2's other half,
+	// which only stopping "chat" frees.
+	idle := catalog.Model{Name: "idle", Source: "/m", Scaling: catalog.Scaling{Min: 0, Max: 1},
+		Profiles: []catalog.Profile{{Name: "tp4", Engine: "vllm", GPUClass: "h100-80g", TP: 4}}}
+	chat := catalog.Model{Name: "chat", Source: "/m", Scaling: catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "tp6", Engine: "vllm", GPUClass: "h100-80g", TP: 6}}}
+	vital := catalog.Model{Name: "vital", Source: "/m", Class: catalog.ClassCritical,
+		Scaling:  catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "tp8", Engine: "vllm", GPUClass: "h100-80g", TP: 8}}}
+	h := newFleetHarness(t, []catalog.Model{idle, chat}, gpuNode("1", 6), gpuNode("2", 8))
+	h.r.Want("idle")
+	h.settle(t, 4)
+	var idleNode, chatNode string
+	for _, rep := range h.r.Replicas() {
+		switch rep.Model {
+		case "idle":
+			idleNode = rep.Shards[0].NodeID
+		case "chat":
+			chatNode = rep.Shards[0].NodeID
+		}
+	}
+	if idleNode == "" || chatNode != "2" {
+		t.Fatalf("setup: idle on %q, chat on %q; chat must hold node 2: %+v", idleNode, chatNode, h.r.Replicas())
+	}
+	h.r.mu.Lock()
+	for _, rep := range h.r.replicas {
+		rep.LastUsedAt = h.r.now().Add(-time.Hour)
+	}
+	h.r.mu.Unlock()
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	cat, _ := catalog.New([]catalog.Model{idle, chat, vital})
+	h.r.cat.Store(cat)
+
+	h.r.Reconcile(t.Context())
+	var stopped []string
+	for _, rep := range h.r.Replicas() {
+		if rep.PreemptedFor == "vital" {
+			stopped = append(stopped, rep.Model)
+		}
+	}
+	if len(stopped) != 1 || stopped[0] != "chat" {
+		t.Errorf("stopped %v; only chat's replica makes room for a whole node", stopped)
+	}
+}
+
+func TestOfTwoThatWouldEachMakeRoomTheSmallerIsStopped(t *testing.T) {
+	cpu := inventory.Node{ID: "a", MgmtAddr: "10.0.0.1", State: inventory.Ready, MemoryTotalMB: 31 * 1024, CPUCount: 8}
+	big := catalog.Model{Name: "big", Source: "/m", Scaling: catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 12, CPUCores: 4}}}
+	small := catalog.Model{Name: "small", Source: "/m", Scaling: catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 2, CPUCores: 1}}}
+	vital := catalog.Model{Name: "vital", Source: "/m", Class: catalog.ClassCritical,
+		Scaling:  catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 15, CPUCores: 4}}}
+	// 8 cores: big 4 + small 1 leaves 3, and vital needs 4. Stopping either
+	// is enough; stopping big loses far more.
+	for range 5 { // the choice must not depend on map order
+		h := newFleetHarness(t, []catalog.Model{big, small}, cpu)
+		h.settle(t, 3)
+		h.advance(h.r.cfg.MinLifetime + time.Minute)
+		cat, _ := catalog.New([]catalog.Model{big, small, vital})
+		h.r.cat.Store(cat)
+		h.r.Reconcile(t.Context())
+		var stopped []string
+		for _, rep := range h.r.Replicas() {
+			if rep.PreemptedFor == "vital" {
+				stopped = append(stopped, rep.Model)
+			}
+		}
+		if len(stopped) != 1 || stopped[0] != "small" {
+			t.Fatalf("stopped %v; the small replica alone makes room", stopped)
+		}
+	}
+}
+
+func TestHeadroomCountsWhatAHigherClassCouldTake(t *testing.T) {
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+	got, _ := h.r.Headroom("vital", 3)
+	if got.More != 0 || got.ByPreempting != 1 || len(got.Preempts) != 1 || got.Preempts[0] != "chat" {
+		t.Errorf("a critical model should see one more by stopping chat: %+v", got)
+	}
+	// Tried on a copy: nothing was stopped.
+	if replicasOf(h, "chat") != 1 {
+		t.Fatal("asking about room stopped a replica")
+	}
+	// A model of chat's own class may not take its minimum, so has no such room.
+	peer := newFleetHarness(t, []catalog.Model{gpuModel("chat", "", 1), withMin(gpuModel("peer", "", 1), 0)}, gpuNode("1", 8))
+	peer.settle(t, 3)
+	peer.advance(peer.r.cfg.MinLifetime + time.Minute)
+	if got, _ := peer.r.Headroom("peer", 3); got.ByPreempting != 0 {
+		t.Errorf("a standard model counted room in another's minimum: %+v", got)
+	}
+}
+
+func TestARestartBorrowsALowerClassesRoomAndGivesItBack(t *testing.T) {
+	// chat fills node 1; the only other node holds an opportunistic replica.
+	chat := gpuModel("chat", "", 1)
+	extra := gpuModel("extra", catalog.ClassOpportunistic, 1)
+	h := newFleetHarness(t, []catalog.Model{chat, extra}, gpuNode("1", 8), gpuNode("2", 8))
+	h.settle(t, 3)
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	var old string
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == "chat" {
+			old = rep.ID
+		}
+	}
+	if old == "" || replicasOf(h, "extra") != 1 {
+		t.Fatalf("setup: %+v", h.r.Replicas())
+	}
+
+	if err := h.r.Restart(old); err != nil {
+		t.Fatal(err)
+	}
+	// Replacement first, beside the old one, in the room extra gives up;
+	// the old one keeps serving until then.
+	servedThroughout := true
+	for range 8 {
+		h.r.Reconcile(t.Context())
+		h.markReady()
+		ready := 0
+		for _, rep := range h.r.Replicas() {
+			if rep.Model == "chat" && rep.State == Ready {
+				ready++
+			}
+		}
+		if ready == 0 {
+			servedThroughout = false
+		}
+		h.advance(h.r.cfg.PreemptGrace + time.Second)
+	}
+	if !servedThroughout {
+		t.Error("chat had no ready replica during its restart")
+	}
+	var newChat bool
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == "chat" && rep.ID != old && rep.State == Ready {
+			newChat = true
+		}
+		if rep.ID == old {
+			t.Errorf("the old replica is still there: %+v", rep)
+		}
+	}
+	if !newChat {
+		t.Fatalf("no new chat replica: %+v", h.r.Replicas())
+	}
+	// And extra has its replica back, in the room the old chat left.
+	h.settle(t, 4)
+	if replicasOf(h, "extra") != 1 {
+		t.Errorf("the opportunistic model did not get its room back: %+v", h.r.Replicas())
+	}
+}
+
+func TestHeadroomNamesOnlyTheReplicasThatWouldMakeRoom(t *testing.T) {
+	// vital needs a whole node and chat is on the only one; a second
+	// critical can never fit. Asking for 3: one by stopping chat, then no
+	// candidates left — nothing else may be named.
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+	got, _ := h.r.Headroom("vital", 3)
+	if got.ByPreempting != 1 || len(got.Preempts) != 1 {
+		t.Errorf("%+v", got)
+	}
+	// Free room for one, and candidates that cannot make room for a second:
+	// none of them may be listed.
+	small := catalog.Model{Name: "small", Source: "/m", Class: catalog.ClassOpportunistic,
+		Scaling:  catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "tp2", Engine: "vllm", GPUClass: "h100-80g", TP: 2}}}
+	h2 := fillThenWant(t, gpuModel("big", "", 1), small, gpuNode("1", 8), gpuNode("2", 8))
+	got2, _ := h2.r.Headroom("big", 3)
+	if got2.More != 1 || got2.ByPreempting != 1 || len(got2.Preempts) != 1 {
+		t.Errorf("one free, one by stopping small: %+v", got2)
+	}
+	h3 := fillThenWant(t, gpuModel("big", "", 1), small, gpuNode("1", 8), gpuNode("2", 6))
+	got3, _ := h3.r.Headroom("big", 3)
+	if got3.ByPreempting != 0 || len(got3.Preempts) != 0 {
+		t.Errorf("stopping small frees a 6-GPU node, too small for big; nobody may be named: %+v", got3)
+	}
+}
@@ -0,0 +1,105 @@
+package reconciler
+
+import (
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/placement"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// SetQuotas supplies the tenancy tree's capacity quotas, read afresh each
+// time it is called. Without it nothing is guaranteed or limited, and
+// preemption goes by class alone.
+func (r *Reconciler) SetQuotas(f func() *quota.Policy) {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	r.quotas = f
+}
+
+func (r *Reconciler) policy() *quota.Policy {
+	r.mu.Lock()
+	f := r.quotas
+	r.mu.Unlock()
+	if f == nil {
+		return nil
+	}
+	return f()
+}
+
+// Holdings is what every replica holds, and for whom.
+func (r *Reconciler) Holdings() []quota.Holding {
+	cat := r.cat.Load()
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	out := make([]quota.Holding, 0, len(r.replicas))
+	for _, rep := range r.replicas {
+		m, ok := cat.Resolve(rep.Model)
+		if !ok {
+			continue
+		}
+		prof, _ := m.ProfileByName(rep.Profile)
+		var a quota.Amount
+		for _, sh := range rep.Shards {
+			a = a.Plus(quota.Amount{GPUs: len(sh.GPUIndices), CPUCores: cpuCores(prof), MemoryGB: prof.MinMemoryGB})
+		}
+		out = append(out, quota.Holding{Replica: rep.ID, Owner: m.Owner, Amount: a, Since: rep.CreatedAt})
+	}
+	return out
+}
+
+// placementNeed is what a placement would hold.
+func placementNeed(m catalog.Model, pl *placement.Placement) quota.Amount {
+	prof, _ := m.ProfileByName(pl.Profile.Name)
+	var a quota.Amount
+	for _, n := range pl.Nodes {
+		a = a.Plus(quota.Amount{GPUs: len(n.GPUIndices), CPUCores: cpuCores(prof), MemoryGB: prof.MinMemoryGB})
+	}
+	return a
+}
+
+// profileNeed is what a replica of a model's first profile would hold, for
+// deciding before there is a placement.
+func profileNeed(m catalog.Model) quota.Amount {
+	if len(m.Profiles) == 0 {
+		return quota.Amount{}
+	}
+	p := m.Profiles[0]
+	nodes := max(p.Nodes, 1)
+	gpus := p.GPUsPerNode
+	if gpus == 0 && !p.CPUOnly() {
+		gpus = max(p.TP, 1)
+	}
+	return quota.Amount{GPUs: gpus * nodes, CPUCores: cpuCores(p) * nodes, MemoryGB: p.MinMemoryGB * nodes}
+}
+
+// overLimit says why a placement would take its model's owner past a limit.
+func (r *Reconciler) overLimit(m catalog.Model, pl *placement.Placement) string {
+	pol := r.policy()
+	if pol == nil || m.Owner == "" {
+		return ""
+	}
+	return pol.AtLimit(m.Owner, pol.Usage(r.Holdings()), placementNeed(m, pl))
+}
+
+// QuotaUse is one scope's quota beside what it holds.
+type QuotaUse struct {
+	Scope     string
+	Guarantee store.Resources
+	Limit     store.Resources
+	Holds     quota.Amount
+}
+
+// QuotaUse lists every quota with what its scope holds now.
+func (r *Reconciler) QuotaUse() []QuotaUse {
+	pol := r.policy()
+	if pol == nil {
+		return nil
+	}
+	usage := pol.Usage(r.Holdings())
+	var out []QuotaUse
+	for _, q := range pol.Quotas() {
+		key := quota.Key(q.Scope, q.ScopeID)
+		out = append(out, QuotaUse{Scope: key, Guarantee: q.Guarantee, Limit: q.Limit, Holds: usage[key]})
+	}
+	return out
+}
@@ -0,0 +1,81 @@
+package reconciler
+
+import (
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+func owned(m catalog.Model, owner string) catalog.Model { m.Owner = owner; return m }
+
+func gpus(n int) store.Resources { return store.Resources{GPUs: &n} }
+
+// Two teams in one org; each 8-GPU node holds one tp8 replica.
+func withQuotas(h *harness, qs ...store.CapacityQuota) {
+	pol := quota.New(qs, map[string][]string{"team:ml": {"org:acme"}, "team:web": {"org:acme"}})
+	h.r.SetQuotas(func() *quota.Policy { return pol })
+}
+
+func TestATenantWithinItsGuaranteeTakesBackWhatAnotherBorrowed(t *testing.T) {
+	// web is guaranteed one node and borrowed the second; ml is guaranteed
+	// one node and wants it. Same class, and web's replica is at its minimum
+	// — by class alone nothing could be taken.
+	web := owned(gpuModel("web-chat", "", 2), "team:web")
+	ml := owned(gpuModel("ml-train", "", 1), "team:ml")
+	h := newFleetHarness(t, []catalog.Model{web, withMin(ml, 0)}, gpuNode("1", 8), gpuNode("2", 8))
+	withQuotas(h,
+		store.CapacityQuota{Scope: store.ScopeTeam, ScopeID: "web", Guarantee: gpus(8)},
+		store.CapacityQuota{Scope: store.ScopeTeam, ScopeID: "ml", Guarantee: gpus(8)})
+	h.settle(t, 4)
+	if replicasOf(h, "web-chat") != 2 {
+		t.Fatalf("setup: web did not borrow the second node: %+v", h.r.Replicas())
+	}
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	cat, _ := catalog.New([]catalog.Model{web, ml})
+	h.r.cat.Store(cat)
+
+	h.settle(t, 5)
+	if replicasOf(h, "ml-train") != 1 || replicasOf(h, "web-chat") != 1 {
+		t.Errorf("the borrowed node was not taken back: web %d, ml %d",
+			replicasOf(h, "web-chat"), replicasOf(h, "ml-train"))
+	}
+}
+
+func TestAReplicaWithinItsGuaranteeIsNotTakenByAHigherClassElsewhere(t *testing.T) {
+	web := owned(gpuModel("web-chat", "", 1), "team:web")
+	vital := owned(gpuModel("vital", catalog.ClassCritical, 1), "team:ml")
+	h := fillThenWant(t, vital, web, gpuNode("1", 8))
+	withQuotas(h, store.CapacityQuota{Scope: store.ScopeTeam, ScopeID: "web", Guarantee: gpus(8)})
+	h.settle(t, 3)
+	if replicasOf(h, "web-chat") != 1 || replicasOf(h, "vital") != 0 {
+		t.Errorf("a guaranteed replica was preempted by another tenant's class: %+v", h.r.Replicas())
+	}
+	// Without the guarantee, critical takes it: the guarantee was the reason.
+	h2 := fillThenWant(t, vital, web, gpuNode("1", 8))
+	h2.settle(t, 3)
+	if replicasOf(h2, "vital") != 1 {
+		t.Error("setup: without a guarantee the critical model should have taken the node")
+	}
+}
+
+func TestALimitStopsAModelBeforeItStarts(t *testing.T) {
+	web := owned(gpuModel("web-chat", "", 2), "team:web")
+	h := newFleetHarness(t, []catalog.Model{web}, gpuNode("1", 8), gpuNode("2", 8))
+	withQuotas(h, store.CapacityQuota{Scope: store.ScopeOrg, ScopeID: "acme", Limit: gpus(8)})
+	h.settle(t, 4)
+	if replicasOf(h, "web-chat") != 1 {
+		t.Errorf("a second replica went past the org's 8-GPU limit: %+v", h.r.Replicas())
+	}
+	hs := health(h, "web-chat")
+	if !hs.Unplaceable || !strings.Contains(hs.Placement, "org:acme is at its capacity limit (16 GPUs of 8)") {
+		t.Errorf("the refusal should name the limit: %+v", hs)
+	}
+	room, _ := h.r.Headroom("web-chat", 5)
+	if room.More != 0 || len(room.Blocked) != 1 || !strings.Contains(room.Blocked[0], "capacity limit") {
+		t.Errorf("headroom should stop at the limit, not the empty node: %+v", room)
+	}
+}
@@ -15,6 +15,7 @@ import (
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/placement"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
 )

 // ShardCommand is everything an agent needs to launch one shard.
@@ -65,6 +66,12 @@ type Config struct {
 	// Without a bound, draining a node whose model cannot be placed anywhere
 	// else would never finish.
 	DrainGrace time.Duration
+	// PreemptGrace is how long a replica stopped to make room for another
+	// is given to finish what it is serving: a request, not a replacement.
+	PreemptGrace time.Duration
+	// PreemptHold is how long a model that gave up room may not take it back,
+	// so the replica it was given up for is placed first.
+	PreemptHold time.Duration
 	// ScaleUpWaiting is the average number of requests waiting per ready
 	// replica, held for ScaleUpAfter, that adds a replica — up to the model's
 	// max. ScaleDownAfter is how long with nothing waiting before one of the
@@ -141,6 +148,8 @@ func DefaultConfig() Config {
 		EvictionGrace:    2 * time.Minute,
 		MinLifetime:      5 * time.Minute,
 		DrainGrace:       5 * time.Minute,
+		PreemptGrace:     30 * time.Second,
+		PreemptHold:      5 * time.Minute,
 		ParkAfter:        5,
 		ScaleUpWaiting:   1,
 		ScaleUpAfter:     time.Minute,
@@ -158,6 +167,12 @@ func (c *Config) normalize() {
 	if c.BaseBackoff <= 0 {
 		c.BaseBackoff = d.BaseBackoff
 	}
+	if c.PreemptGrace <= 0 {
+		c.PreemptGrace = d.PreemptGrace
+	}
+	if c.PreemptHold <= 0 {
+		c.PreemptHold = d.PreemptHold
+	}
 	if c.MaxBackoff < c.BaseBackoff {
 		c.MaxBackoff = max(c.BaseBackoff, d.MaxBackoff)
 	}
@@ -250,6 +265,16 @@ type Reconciler struct {
 	// placementLogged remembers the last "no placement" said per model, so it
 	// is said when it changes rather than on every five-second pass.
 	placementLogged map[string]placementNote
+	// preempting is, per model, the replicas stopped to make room for it and
+	// not gone yet; held is, per model that gave them up, until when it may
+	// not take the room back; preempted counts, per model, replicas it lost.
+	preempting map[string][]string
+	held       map[string]hold
+	preempted  map[string]int
+	// quotas is the tenancy tree's capacity quotas as last read; nil is none.
+	quotas func() *quota.Policy
+	// measurements are benchmark results by model/profile; nil is none.
+	measurements func() map[string]Measured

 	wake chan struct{}
 	stop chan struct{}
@@ -285,6 +310,9 @@ func New(cfg Config, cat *catalog.Live, inv *inventory.Store, disp Dispatcher, l
 		quietSince:      map[string]time.Time{},
 		placementLogged: map[string]placementNote{},
 		lastExplain:     map[string]placement.Result{},
+		preempting:      map[string][]string{},
+		held:            map[string]hold{},
+		preempted:       map[string]int{},
 		wake:            make(chan struct{}, 1),
 		stop:            make(chan struct{}),
 		done:            make(chan struct{}),
@@ -704,6 +732,9 @@ func (r *Reconciler) reconcileModel(ctx context.Context, m catalog.Model) {
 	switch {
 	case len(liveIDs) < desired && isParked:
 		// Not placed. Its failure is on every status the operator reads.
+	case len(liveIDs) < desired && r.isHeld(m.Name):
+		// It gave up room to a model of a higher class, which is placed
+		// first; taking it back now would undo the preemption.
 	case len(liveIDs) < desired:
 		if r.now().Before(blockedUntil) {
 			// Still in backoff. Rebuilding immediately would hot-loop on a
@@ -724,8 +755,12 @@ func (r *Reconciler) reconcileModel(ctx context.Context, m catalog.Model) {
 		r.mu.Lock()
 		rep, ok := r.replicas[id]
 		var expired bool
+		grace, preemptedFor := r.cfg.DrainGrace, ""
 		if ok {
-			expired = !rep.DrainingAt.IsZero() && r.now().Sub(rep.DrainingAt) > r.cfg.DrainGrace
+			if rep.PreemptedFor != "" {
+				grace, preemptedFor = r.cfg.PreemptGrace, rep.PreemptedFor
+			}
+			expired = !rep.DrainingAt.IsZero() && r.now().Sub(rep.DrainingAt) > grace
 		}
 		r.mu.Unlock()

@@ -733,11 +768,15 @@ func (r *Reconciler) reconcileModel(ctx context.Context, m catalog.Model) {
 		case !ok:
 		case replaced:
 			r.stopReplica(ctx, id)
+		case expired && preemptedFor != "":
+			r.log.Info("stopping a preempted replica", "replica", id,
+				"for", preemptedFor, "after", grace)
+			r.stopReplica(ctx, id)
 		case expired:
 			// The replacement never arrived — often because nowhere else can
 			// host it. Draining forever would leave the node undrainable.
 			r.log.Warn("stopping a drained replica without a replacement",
-				"replica", id, "after", r.cfg.DrainGrace)
+				"replica", id, "after", grace)
 			r.stopReplica(ctx, id)
 		}
 	}
@@ -751,12 +790,13 @@ func (r *Reconciler) reconcileModel(ctx context.Context, m catalog.Model) {
 // allocated, and starting several at once from the same inventory snapshot
 // would hand them the same GPUs.
 func (r *Reconciler) scaleUp(ctx context.Context, m catalog.Model, avoid []string) {
-	res := r.sched.Place(m.PlacementProfiles(), avoid)
+	profiles := r.orderedProfiles(m)
+	res := r.sched.Place(profiles, avoid)
 	if res.Placement == nil && len(avoid) > 0 {
 		// Spreading across failure domains is a preference, not a rule. If it
 		// is the only thing standing in the way, take the placement anyway:
 		// a second replica on one node still survives an engine crash.
-		res = r.sched.Place(m.PlacementProfiles(), nil)
+		res = r.sched.Place(profiles, nil)
 	}

 	r.mu.Lock()
@@ -775,9 +815,22 @@ func (r *Reconciler) scaleUp(ctx context.Context, m catalog.Model, avoid []strin
 		if m.Scaling.OnDemand() && r.makeRoomFor(ctx, m) {
 			return
 		}
+		if r.awaitingPreemption(m.Name) || r.preemptFor(ctx, m, avoid) {
+			return
+		}
 		r.notePlacementFailure(m.Name, res.Explain())
 		return
 	}
+	if why := r.overLimit(m, res.Placement); why != "" {
+		// Recorded as the placement's answer, so the model reads as not
+		// placeable, and says why, wherever that is shown.
+		r.mu.Lock()
+		r.lastExplain[m.Name] = placement.Result{Rejections: []placement.Rejection{
+			{Profile: res.Placement.Profile.Name, Reason: why}}}
+		r.mu.Unlock()
+		r.notePlacementFailure(m.Name, why)
+		return
+	}
 	if err := r.startReplica(ctx, m, res.Placement); err != nil {
 		r.log.Error("starting replica failed", "model", m.Name, "err", err)
 	}
@@ -825,7 +878,7 @@ func (r *Reconciler) startReplica(ctx context.Context, m catalog.Model, pl *plac
 		})
 		r.inv.Allocate(a.NodeID, inventory.Allocation{
 			ShardID: shardID, ReplicaID: id, GPUIndices: a.GPUIndices,
-			MemoryGB: prof.MinMemoryGB,
+			MemoryGB: prof.MinMemoryGB, CPUCores: cpuCores(prof),
 		})
 	}
 	rep.Endpoint = fmt.Sprintf("http://%s:%d", addrOf(leaderNode), basePort)
@@ -869,6 +922,7 @@ func (r *Reconciler) startReplica(ctx context.Context, m catalog.Model, pl *plac
 				HTTPPort:      s.HTTPPort,
 				Profile:       espec,
 				APIKey:        r.engineKey(id),
+				Threads:       cpuCores(prof),
 			},
 			StartupTimeout: prof.StartupTimeout,
 			Pool:           pl.Pool,
@@ -1108,16 +1162,31 @@ func addrOf(n inventory.Node) string {
 // claim on restart would place a second large model onto a node already full of
 // the first.
 func (r *Reconciler) profileMemory(model, profile string) int {
+	p, _ := r.profileOf(model, profile)
+	return p.MinMemoryGB
+}
+
+// profileOf is a replica's profile as the catalog holds it now.
+func (r *Reconciler) profileOf(model, profile string) (catalog.Profile, bool) {
 	m, ok := r.cat.Load().Resolve(model)
 	if !ok {
-		return 0
+		return catalog.Profile{}, false
 	}
 	for _, p := range m.Profiles {
 		if p.Name == profile {
-			return p.MinMemoryGB
+			return p, true
 		}
 	}
-	return 0
+	return catalog.Profile{}, false
+}
+
+// cpuCores is what a profile reserves of a node's CPUs: its cores when it
+// computes on them, nothing when a GPU does the work.
+func cpuCores(p catalog.Profile) int {
+	if !p.CPUOnly() {
+		return 0
+	}
+	return p.CPUCores
 }

 func (r *Reconciler) engineKey(replicaID string) string {
@@ -156,6 +156,8 @@ type Replica struct {
 	// DrainingAt is when the replica stopped taking new work, so the wait for a
 	// replacement can be bounded.
 	DrainingAt time.Time
+	// PreemptedFor is the model this replica was stopped to make room for.
+	PreemptedFor string
 	// RetiringAt is set while the replica is being restarted: it keeps serving
 	// but no longer counts toward the model's replicas, so a replacement is
 	// placed beside it, and it is drained only once that replacement is ready.
@@ -25,6 +25,10 @@ type Header struct {
 	Version      uint32
 	Architecture string
 	Name         string
+	// Type is general.type: "model", or "adapter" for a LoRA adapter, which
+	// is applied to a model rather than run on its own. Empty in files made
+	// before the key existed, which are models.
+	Type string

 	// The shape the KV cache is sized from, from the architecture's own keys
 	// (llama.block_count, qwen2.attention.head_count_kv, ...). Zero when the
@@ -235,6 +239,8 @@ func Read(r io.Reader) (Header, error) {
 			h.Architecture = v
 		case "general.name":
 			h.Name = v
+		case "general.type":
+			h.Type = v
 		}
 		if h.Architecture != "" {
 			switch strings.TrimPrefix(key, h.Architecture+".") {
@@ -290,6 +296,41 @@ func FromURL(ctx context.Context, client *http.Client, url string) (Header, erro
 	return Read(io.LimitReader(resp.Body, headerBytes))
 }

+// IsAdapter reports whether the file is a LoRA adapter rather than a model.
+func (h Header) IsAdapter() bool { return h.Type == "adapter" }
+
+// adapterProbe is how much of a file is read to tell an adapter from a model:
+// general.type comes before the tokenizer, which is what makes a model's
+// header megabytes long.
+const adapterProbe = 256 << 10
+
+// KindFromURL reads just enough of a GGUF file to say what it is — its type
+// and architecture — over HTTP. A header cut short by the probe is fine: the
+// keys asked for come first.
+func KindFromURL(ctx context.Context, client *http.Client, url string) (Header, error) {
+	if client == nil {
+		client = &http.Client{Timeout: 30 * time.Second}
+	}
+	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
+	if err != nil {
+		return Header{}, err
+	}
+	req.Header.Set("Range", fmt.Sprintf("bytes=0-%d", adapterProbe-1))
+	resp, err := client.Do(req)
+	if err != nil {
+		return Header{}, err
+	}
+	defer resp.Body.Close()
+	if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusPartialContent {
+		return Header{}, fmt.Errorf("%s answered %s", url, resp.Status)
+	}
+	h, err := Read(io.LimitReader(resp.Body, adapterProbe))
+	if h.Architecture != "" {
+		return h, nil // what was asked for was read, whatever came after
+	}
+	return h, err
+}
+
 // IsGGUF reports whether a source names a GGUF file, or a directory of them.
 func IsGGUF(source string) bool {
 	s := strings.ToLower(strings.TrimSuffix(source, "/"))
@@ -0,0 +1,31 @@
+package gguf
+
+import (
+	"os"
+	"testing"
+)
+
+// Run by hand against the real adapter this was written for:
+//
+//	LLMGW_LIVE_GGUF=1 go test ./internal/gguf -run LiveAdapter -v
+func TestLiveAdapterIsAnAdapter(t *testing.T) {
+	if os.Getenv("LLMGW_LIVE_GGUF") == "" {
+		t.Skip("set LLMGW_LIVE_GGUF to read a real adapter from Hugging Face")
+	}
+	h, err := KindFromURL(t.Context(), nil,
+		"https://huggingface.co/ggml-org/LoRA-Qwen2.5-Coder-7B-Instruct-F16-GGUF/resolve/main/LoRA-Qwen2.5-Coder-7B-Instruct-f16.gguf")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !h.IsAdapter() || h.Architecture != "qwen2" {
+		t.Errorf("%+v", h)
+	}
+	m, err := KindFromURL(t.Context(), nil,
+		"https://huggingface.co/bartowski/Qwen2.5-0.5B-Instruct-GGUF/resolve/main/Qwen2.5-0.5B-Instruct-Q4_K_M.gguf")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m.IsAdapter() || m.Architecture != "qwen2" {
+		t.Errorf("a model read as %+v", m)
+	}
+}
@@ -0,0 +1,58 @@
+package gguf
+
+import (
+	"bytes"
+	"encoding/binary"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+)
+
+// adapterHeader is the start of a LoRA adapter as llama.cpp's converter
+// writes it, followed by padding that stands in for its tensors.
+func adapterHeader(kind string, pad int) []byte {
+	var b bytes.Buffer
+	w := func(v any) { _ = binary.Write(&b, binary.LittleEndian, v) }
+	str := func(s string) { w(uint64(len(s))); b.WriteString(s) }
+	b.WriteString("GGUF")
+	w(uint32(3))
+	w(uint64(0))
+	w(uint64(3))
+	str("general.architecture")
+	w(uint32(tString))
+	str("qwen2")
+	str("general.type")
+	w(uint32(tString))
+	str(kind)
+	str("adapter.type")
+	w(uint32(tString))
+	str("lora")
+	b.Write(make([]byte, pad))
+	return b.Bytes()
+}
+
+func TestAnAdapterIsToldFromAModelByItsFirstBytes(t *testing.T) {
+	var asked string
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		asked = r.Header.Get("Range")
+		w.WriteHeader(http.StatusPartialContent)
+		if r.URL.Path == "/adapter.gguf" {
+			_, _ = w.Write(adapterHeader("adapter", 1<<20))
+			return
+		}
+		_, _ = w.Write(header("qwen2"))
+	}))
+	defer srv.Close()
+
+	h, err := KindFromURL(t.Context(), srv.Client(), srv.URL+"/adapter.gguf")
+	if err != nil || !h.IsAdapter() || h.Architecture != "qwen2" {
+		t.Errorf("an adapter read as %+v, %v", h, err)
+	}
+	if asked != "bytes=0-262143" {
+		t.Errorf("asked for %q: the probe should read the start only", asked)
+	}
+	m, err := KindFromURL(t.Context(), srv.Client(), srv.URL+"/model.gguf")
+	if err != nil || m.IsAdapter() || m.Architecture != "qwen2" {
+		t.Errorf("a model read as %+v, %v", m, err)
+	}
+}
@@ -156,6 +156,10 @@ type Query struct {
 	// applies it, so a page of small models is a full page.
 	MaxParams int64
 	Limit     int
+	// Filters are further registry tags every result must carry, such as
+	// "lora", or "base_model:adapter:Qwen/Qwen2.5-7B-Instruct" for the
+	// adapters made for one model.
+	Filters []string
 }

 // exploreFields is everything a listing shows. Asking for any field with
@@ -188,7 +192,10 @@ func (c *Client) Explore(ctx context.Context, q Query) ([]Model, error) {
 		v.Set("pipeline_tag", q.Task)
 	}
 	if q.GGUF {
-		v.Set("filter", "gguf")
+		v.Add("filter", "gguf")
+	}
+	for _, f := range q.Filters {
+		v.Add("filter", f)
 	}
 	if q.MaxParams > 0 {
 		v.Set("num_parameters", fmt.Sprintf("min:0,max:%d", q.MaxParams))
@@ -9,6 +9,7 @@ package release

 import (
 	"bufio"
+	"bytes"
 	"context"
 	"crypto/sha256"
 	"encoding/hex"
@@ -51,7 +52,7 @@ func Install(ctx context.Context, client *http.Client, base, version, goos, goar
 	asset := AssetName(version, goos, goarch)
 	root := strings.TrimSuffix(base, "/") + "/" + version + "/"

-	want, err := checksum(ctx, client, root+"checksums.txt", asset)
+	want, err := checksum(ctx, client, root, asset)
 	if err != nil {
 		return "", err
 	}
@@ -85,13 +86,35 @@ func Install(ctx context.Context, client *http.Client, base, version, goos, goar
 	return backup, nil
 }

-func checksum(ctx context.Context, client *http.Client, url, asset string) (string, error) {
-	body, err := get(ctx, client, url)
+// checksum reads the release's checksum for asset — after checking the
+// checksums' signature, when this build trusts a signing key. A release with
+// no signature is then refused rather than taken on its checksums: a missing
+// signature is exactly what replacing a release would look like.
+func checksum(ctx context.Context, client *http.Client, root, asset string) (string, error) {
+	body, err := get(ctx, client, root+"checksums.txt")
 	if err != nil {
 		return "", err
 	}
 	defer body.Close()
-	sc := bufio.NewScanner(io.LimitReader(body, 1<<20))
+	sums, err := io.ReadAll(io.LimitReader(body, 1<<20))
+	if err != nil {
+		return "", err
+	}
+	if key := TrustedKey(); key != "" {
+		sigBody, err := get(ctx, client, root+SignatureName)
+		if err != nil {
+			return "", fmt.Errorf("this build only installs signed releases, and %s could not be read: %w", SignatureName, err)
+		}
+		sig, err := io.ReadAll(io.LimitReader(sigBody, 4096))
+		sigBody.Close()
+		if err != nil {
+			return "", err
+		}
+		if err := Verify(key, sums, string(sig)); err != nil {
+			return "", err
+		}
+	}
+	sc := bufio.NewScanner(bytes.NewReader(sums))
 	for sc.Scan() {
 		f := strings.Fields(sc.Text())
 		if len(f) == 2 && f[1] == asset && len(f[0]) == 64 {
@@ -0,0 +1,72 @@
+package release
+
+import (
+	"crypto/ed25519"
+	"crypto/rand"
+	_ "embed"
+	"encoding/base64"
+	"errors"
+	"fmt"
+	"os"
+	"strings"
+)
+
+// trustedPub is the public key releases are signed with, base64, built into
+// every binary from the repository. Empty — as it is until a key is made —
+// trusts a release on its checksums alone, as every release before this did.
+//
+// Built in rather than fetched, because the point of a signature is that the
+// server the release comes from cannot vouch for it: a machine that already
+// runs llmgw checks the next release against the key the release it runs was
+// built with, so replacing both the binaries and the checksums on the server
+// is no longer enough.
+//
+//go:embed trusted.pub
+var trustedPub string
+
+// TrustedKey is the key releases are verified with: LLMGW_RELEASE_KEY when set
+// (for rotating a key, or testing), the built-in one otherwise; empty is none.
+func TrustedKey() string {
+	if k := strings.TrimSpace(os.Getenv("LLMGW_RELEASE_KEY")); k != "" {
+		return k
+	}
+	return strings.TrimSpace(trustedPub)
+}
+
+// SignatureName is the file a release's checksums are signed in.
+const SignatureName = "checksums.txt.sig"
+
+// GenerateKey makes a signing key pair, both base64: the private key is the
+// 32-byte seed, which is what goes into the CI secret.
+func GenerateKey() (public, private string, err error) {
+	pub, priv, err := ed25519.GenerateKey(rand.Reader)
+	if err != nil {
+		return "", "", err
+	}
+	return base64.StdEncoding.EncodeToString(pub), base64.StdEncoding.EncodeToString(priv.Seed()), nil
+}
+
+// Sign signs data with a base64 seed, returning the signature in base64.
+func Sign(privateSeed string, data []byte) (string, error) {
+	seed, err := base64.StdEncoding.DecodeString(strings.TrimSpace(privateSeed))
+	if err != nil || len(seed) != ed25519.SeedSize {
+		return "", errors.New("the signing key is not a base64 ed25519 seed")
+	}
+	return base64.StdEncoding.EncodeToString(ed25519.Sign(ed25519.NewKeyFromSeed(seed), data)), nil
+}
+
+// Verify checks a base64 signature of data against a base64 public key.
+func Verify(public string, data []byte, signature string) error {
+	pub, err := base64.StdEncoding.DecodeString(strings.TrimSpace(public))
+	if err != nil || len(pub) != ed25519.PublicKeySize {
+		return errors.New("the trusted release key is not a base64 ed25519 public key")
+	}
+	sig, err := base64.StdEncoding.DecodeString(strings.TrimSpace(signature))
+	if err != nil {
+		return fmt.Errorf("the release signature is not base64: %w", err)
+	}
+	if !ed25519.Verify(pub, data, sig) {
+		return errors.New("the release's checksums are not signed by the trusted key")
+	}
+	return nil
+}
@@ -0,0 +1,99 @@
+package release
+
+import (
+	"crypto/sha256"
+	"encoding/hex"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+)
+
+// signedServer publishes a release whose checksums are signed with seed,
+// or not signed at all when seed is empty; tamper rewrites the checksums
+// after signing.
+func signedServer(t *testing.T, seed string, body []byte, tamper bool) (*httptest.Server, string) {
+	t.Helper()
+	asset := AssetName("v2.0.0", "linux", "amd64")
+	sum := sha256.Sum256(body)
+	sums := hex.EncodeToString(sum[:]) + "  " + asset + "\n"
+	sig := ""
+	if seed != "" {
+		var err error
+		if sig, err = Sign(seed, []byte(sums)); err != nil {
+			t.Fatal(err)
+		}
+	}
+	if tamper {
+		sums += strings.Repeat("0", 64) + "  extra-file\n"
+	}
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/v2.0.0/checksums.txt":
+			_, _ = w.Write([]byte(sums))
+		case "/v2.0.0/" + SignatureName:
+			if sig == "" {
+				http.NotFound(w, r)
+				return
+			}
+			_, _ = w.Write([]byte(sig + "\n"))
+		case "/v2.0.0/" + asset:
+			_, _ = w.Write(body)
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	t.Cleanup(srv.Close)
+	return srv, asset
+}
+
+func TestABuildThatTrustsAKeyInstallsOnlyReleasesSignedWithIt(t *testing.T) {
+	pub, priv, err := GenerateKey()
+	if err != nil {
+		t.Fatal(err)
+	}
+	_, otherPriv, _ := GenerateKey()
+	t.Setenv("LLMGW_RELEASE_KEY", pub)
+	install := func(srv *httptest.Server) error {
+		target := filepath.Join(t.TempDir(), "llmgw")
+		_ = os.WriteFile(target, []byte("old"), 0o755)
+		_, err := Install(t.Context(), srv.Client(), srv.URL, "v2.0.0", "linux", "amd64", target)
+		return err
+	}
+	body := []byte("new binary")
+
+	good, _ := signedServer(t, priv, body, false)
+	if err := install(good); err != nil {
+		t.Fatalf("a release signed with the trusted key was refused: %v", err)
+	}
+	for name, srv := range map[string]*httptest.Server{
+		"unsigned":            func() *httptest.Server { s, _ := signedServer(t, "", body, false); return s }(),
+		"signed by another":   func() *httptest.Server { s, _ := signedServer(t, otherPriv, body, false); return s }(),
+		"checksums rewritten": func() *httptest.Server { s, _ := signedServer(t, priv, body, true); return s }(),
+	} {
+		if err := install(srv); err == nil {
+			t.Errorf("%s: installed", name)
+		}
+	}
+
+	// A build that trusts no key installs on checksums, as before.
+	t.Setenv("LLMGW_RELEASE_KEY", "")
+	if TrustedKey() != "" {
+		t.Skip("this build embeds a trusted key; the unsigned path cannot be tested here")
+	}
+	unsigned, _ := signedServer(t, "", body, false)
+	if err := install(unsigned); err != nil {
+		t.Errorf("without a trusted key an unsigned release was refused: %v", err)
+	}
+}
+
+func TestAKeyThatIsNotOneIsSaid(t *testing.T) {
+	if _, err := Sign("not-a-key", []byte("x")); err == nil {
+		t.Error("signed with a malformed key")
+	}
+	if err := Verify("bm9wZQ==", []byte("x"), "c2ln"); err == nil || !strings.Contains(err.Error(), "public key") {
+		t.Errorf("a malformed trusted key: %v", err)
+	}
+}
@@ -0,0 +1,98 @@
+package router
+
+import (
+	"encoding/json"
+	"net/http"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/auth"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/config"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+func adapterModel() catalog.Model {
+	m := catModel("coder", nil)
+	m.Profiles = []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 4}}
+	m.Adapters = []catalog.Adapter{
+		{Name: "coder-sql", Source: "http://mirror/sql.gguf"},
+		{Name: "coder-docs", Source: "http://mirror/docs.gguf", Scale: 0.5},
+	}
+	return m
+}
+
+func TestAnAdapterIsAskedForByNameAndAppliedOnItsModel(t *testing.T) {
+	e := newFakeEngine(t, "coder")
+	cfg := baseConfig(config.Endpoint{ID: "a", URL: e.srv.URL, Models: []string{"coder"}})
+	cfg.Models = []catalog.Model{adapterModel()}
+	rt := newTestRouter(t, cfg)
+
+	sent := func(model string) map[string]any {
+		t.Helper()
+		rec := post(t, rt.Handler(), map[string]any{"model": model, "messages": []any{}}, nil)
+		if rec.Code != http.StatusOK {
+			t.Fatalf("%s: status %d: %s", model, rec.Code, rec.Body.String())
+		}
+		var body map[string]any
+		_ = json.Unmarshal(e.last.Load().([]byte), &body)
+		return body
+	}
+	body := sent("coder-docs")
+	if body["model"] != "coder" {
+		t.Errorf("the engine was asked for %v, not the model the adapter runs on", body["model"])
+	}
+	lora, _ := json.Marshal(body["lora"])
+	if string(lora) != `[{"id":1,"scale":0.5}]` {
+		t.Errorf("lora %s, expected the second adapter at its scale", lora)
+	}
+	if body := sent("coder-sql"); func() string { b, _ := json.Marshal(body["lora"]); return string(b) }() != `[{"id":0,"scale":1}]` {
+		t.Errorf("the first adapter, at the default scale: %v", body["lora"])
+	}
+	if body := sent("coder"); body["lora"] != nil {
+		t.Errorf("the base model was sent an adapter: %v", body["lora"])
+	}
+}
+
+func TestAKeyAllowedAnAdapterReachesThatAdapterOnly(t *testing.T) {
+	cfg := baseConfig()
+	cfg.Models = []catalog.Model{adapterModel()}
+	rt := newTestRouter(t, cfg)
+	key := func(allowed ...string) auth.Principal {
+		return auth.Principal{Limits: store.Effective{AllowedModels: allowed}}
+	}
+	for _, c := range []struct {
+		allowed   []string
+		requested string
+		want      bool
+	}{
+		{[]string{"coder-sql"}, "coder-sql", true},
+		{[]string{"coder-sql"}, "coder", false},
+		{[]string{"coder-sql"}, "coder-docs", false},
+		{[]string{"coder"}, "coder-sql", true}, // the model allows its adapters
+		{[]string{"coder"}, "coder", true},
+	} {
+		if got := rt.permits(key(c.allowed...), c.requested); got != c.want {
+			t.Errorf("allowed %v, asked for %s: %v", c.allowed, c.requested, got)
+		}
+	}
+}
+
+func TestAdaptersShareTheNamespaceOfModels(t *testing.T) {
+	a := adapterModel()
+	b := catModel("coder-sql", nil)
+	if _, err := catalog.New([]catalog.Model{a, b}); err == nil {
+		t.Error("an adapter named like another model was accepted")
+	}
+	bad := adapterModel()
+	bad.Profiles = []catalog.Profile{{Name: "gpu", Engine: "vllm", GPUClass: "h100-80g", TP: 1}}
+	if _, err := catalog.New([]catalog.Model{bad}); err == nil {
+		t.Error("adapters on a vLLM profile were accepted")
+	}
+	c, err := catalog.New([]catalog.Model{adapterModel()})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m, ad, i, ok := c.ResolveAdapter("coder-docs"); !ok || m.Name != "coder" || i != 1 || ad.EffectiveScale() != 0.5 {
+		t.Errorf("resolve: %v %+v %d %v", m.Name, ad, i, ok)
+	}
+}
@@ -34,6 +34,12 @@ type Config struct {
 	// QueueCapacityPerBand bounds each band's queue. Rejecting immediately
 	// beats accumulating silently.
 	QueueCapacityPerBand int `koanf:"queue_capacity_per_band"`
+	// QueueCapacityTotal bounds everything queued, across bands. When it is
+	// reached, a request is not refused while a lower band has requests
+	// queued: the newest of the lowest band is turned away instead
+	// (displacement), since the oldest have waited longest and are nearest
+	// their TTL. Zero takes half of every band's capacity together.
+	QueueCapacityTotal int `koanf:"queue_capacity_total"`
 	// Interval is how often the dispatcher re-checks saturation. Releases also
 	// wake it, so this is only the backstop for saturation falling on its own.
 	Interval time.Duration `koanf:"interval"`
@@ -94,6 +100,9 @@ func (c *Config) normalize() error {
 	if c.QueueCapacityPerBand <= 0 {
 		c.QueueCapacityPerBand = 2000
 	}
+	if c.QueueCapacityTotal <= 0 {
+		c.QueueCapacityTotal = max(c.QueueCapacityPerBand*len(c.Bands)/2, 1)
+	}
 	if c.Interval <= 0 {
 		c.Interval = 10 * time.Millisecond
 	}
@@ -123,6 +132,10 @@ var (
 	ErrQueueTimeout = errors.New("admission queue timeout")
 	// ErrClosed means the controller is shutting down.
 	ErrClosed = errors.New("admission controller closed")
+	// ErrDisplaced means a queued request was turned away to make room for
+	// one of a higher band. Worth retrying: the queue is full now, not for
+	// good.
+	ErrDisplaced = errors.New("displaced from the admission queue by higher-priority requests")
 )

 // Stats is a snapshot of one band's queue, for metrics.
@@ -259,6 +272,12 @@ func (c *Controller) Acquire(ctx context.Context, r Request) (*Lease, error) {
 		c.mu.Unlock()
 		return nil, fmt.Errorf("%w: band %q", ErrQueueFull, band.Name)
 	}
+	if c.queuedLocked() >= c.cfg.QueueCapacityTotal && !c.displaceBelowLocked(idx) {
+		c.rejected[band.Name]++
+		c.mu.Unlock()
+		return nil, fmt.Errorf("%w: %d queued in all, and nothing of a lower band to make room",
+			ErrQueueFull, c.cfg.QueueCapacityTotal)
+	}

 	w := &waiter{
 		tenant:   tenantKey(r),
@@ -279,6 +298,10 @@ func (c *Controller) Acquire(ctx context.Context, r Request) (*Lease, error) {
 			c.mu.Unlock()
 			return nil, ErrClosed
 		}
+		if w.err != nil {
+			c.mu.Unlock()
+			return nil, w.err
+		}
 		c.waitTotal[band.Name] += time.Since(w.enqueued)
 		c.mu.Unlock()
 		return &Lease{c: c, band: band.Name, model: r.Model}, nil
@@ -297,8 +320,9 @@ func (c *Controller) Acquire(ctx context.Context, r Request) (*Lease, error) {
 func (c *Controller) remove(idx int, w *waiter, reason string) {
 	c.mu.Lock()
 	defer c.mu.Unlock()
-	if w.admitted {
-		// It won the race; the caller will see ready closed next time round.
+	if w.admitted || w.err != nil {
+		// It won the race, or was displaced; the caller will see ready
+		// closed next time round.
 		return
 	}
 	c.queues[idx].remove(w)
@@ -410,3 +434,31 @@ func tenantKey(r Request) string {
 	}
 	return "anonymous"
 }
+
+// queuedLocked is how many requests wait in every band. Called with mu held.
+func (c *Controller) queuedLocked() int {
+	n := 0
+	for _, q := range c.queues {
+		n += q.len()
+	}
+	return n
+}
+
+// displaceBelowLocked turns away the newest request of the lowest band below
+// idx that has any queued, and says whether there was one. Called with mu held.
+func (c *Controller) displaceBelowLocked(idx int) bool {
+	// Bands are sorted highest priority first, so lower bands have larger
+	// indices, and the lowest is last.
+	for j := len(c.queues) - 1; j > idx; j-- {
+		w := c.queues[j].newest()
+		if w == nil {
+			continue
+		}
+		c.queues[j].remove(w)
+		w.err = fmt.Errorf("%w (band %q)", ErrDisplaced, c.cfg.Bands[j].Name)
+		close(w.ready)
+		c.rejected[c.cfg.Bands[j].Name+"/displaced"]++
+		return true
+	}
+	return false
+}
@@ -0,0 +1,89 @@
+package admission
+
+import (
+	"context"
+	"errors"
+	"testing"
+	"time"
+)
+
+// queueUntil starts an Acquire in the background and waits until the band
+// holds depth requests, returning where its result will arrive.
+func queueUntil(t *testing.T, c *Controller, band string, bandIdx, depth int) <-chan error {
+	t.Helper()
+	out := make(chan error, 1)
+	go func() {
+		l, err := c.Acquire(context.Background(), Request{Model: "m", Band: band})
+		if err == nil {
+			l.Release()
+		}
+		out <- err
+	}()
+	deadline := time.Now().Add(2 * time.Second)
+	for c.Depths()[bandIdx].Depth < depth && time.Now().Before(deadline) {
+		time.Sleep(time.Millisecond)
+	}
+	if got := c.Depths()[bandIdx].Depth; got != depth {
+		t.Fatalf("band %s holds %d, expected %d", band, got, depth)
+	}
+	return out
+}
+
+func TestAHigherBandDisplacesTheNewestOfTheLowest(t *testing.T) {
+	s := &fixedSat{}
+	s.set(5) // past every ceiling: everything queues
+	cfg := testConfig()
+	cfg.QueueCapacityTotal = 3
+	c := newController(t, cfg, s.fn)
+	// Bands, highest first: platinum 0, gold 1, silver 2, bronze 3.
+	oldest := queueUntil(t, c, "bronze", 3, 1)
+	time.Sleep(2 * time.Millisecond)
+	silver := queueUntil(t, c, "silver", 2, 1)
+	time.Sleep(2 * time.Millisecond)
+	newest := queueUntil(t, c, "bronze", 3, 2)
+
+	// Full at three. Gold arrives: the newest bronze goes, not the oldest,
+	// and not silver, which outranks bronze.
+	gold := queueUntil(t, c, "gold", 1, 1)
+	select {
+	case err := <-newest:
+		if !errors.Is(err, ErrDisplaced) {
+			t.Fatalf("the newest bronze got %v", err)
+		}
+	case <-time.After(time.Second):
+		t.Fatal("nothing was displaced")
+	}
+	if d := c.Depths(); d[3].Depth != 1 || d[2].Depth != 1 || d[1].Depth != 1 {
+		t.Errorf("depths %+v: one bronze, the silver and the gold should remain", d)
+	}
+
+	// Bronze cannot displace anything: nothing is below it.
+	_, err := c.Acquire(t.Context(), Request{Model: "m", Band: "bronze"})
+	if !errors.Is(err, ErrQueueFull) {
+		t.Errorf("a bronze request at a full queue got %v", err)
+	}
+	// Another gold takes the remaining bronze, then silver.
+	queueUntil(t, c, "gold", 1, 2)
+	if err := <-oldest; !errors.Is(err, ErrDisplaced) {
+		t.Errorf("the last bronze got %v", err)
+	}
+	queueUntil(t, c, "gold", 1, 3)
+	if err := <-silver; !errors.Is(err, ErrDisplaced) {
+		t.Errorf("silver got %v", err)
+	}
+	// All gold now: a fourth has nobody lower to displace.
+	if _, err := c.Acquire(t.Context(), Request{Model: "m", Band: "gold"}); !errors.Is(err, ErrQueueFull) {
+		t.Errorf("a gold request with only gold queued got %v", err)
+	}
+
+	// And they are served once there is room.
+	s.set(0)
+	select {
+	case err := <-gold:
+		if err != nil {
+			t.Errorf("the first gold was not served: %v", err)
+		}
+	case <-time.After(2 * time.Second):
+		t.Error("the queue did not drain")
+	}
+}
@@ -12,6 +12,9 @@ type waiter struct {
 	// admitted guards the race between the dispatcher admitting a waiter and
 	// the waiter giving up on its own.
 	admitted bool
+	// err is why the waiter was turned away while queued — displaced by a
+	// higher band — set before ready is closed.
+	err error
 }

 // bandQueue holds one band's waiters, grouped by tenant.
@@ -166,3 +169,17 @@ func (q *bandQueue) dropExpired(now time.Time, onDrop func()) {
 		q.byTenant[tenant] = kept
 	}
 }
+
+// newest is the most recently queued waiter, across every tenant: the one
+// displacement takes, since it has invested the least waiting.
+func (q *bandQueue) newest() *waiter {
+	var out *waiter
+	for _, ws := range q.byTenant {
+		for _, w := range ws {
+			if out == nil || w.enqueued.After(out.enqueued) {
+				out = w
+			}
+		}
+	}
+	return out
+}
@@ -37,8 +37,16 @@ type Info struct {
 	Weight float64
 	// Priority expresses pool preference; higher pools are tried first.
 	Priority int
+	// UpstreamModel is what the endpoint calls the model, when that differs
+	// from the catalog's name: an external provider's. Requests are rewritten
+	// to it. Empty sends the name as the client did.
+	UpstreamModel string
 }

+// EngineExternal marks an endpoint that is a provider outside the cluster.
+// It publishes no metrics, so it is never scraped and never reads as stale.
+const EngineExternal = "external"
+
 // Serves reports whether the replica serves the given model.
 func (i Info) Serves(model string) bool {
 	return slices.Contains(i.Models, model)
@@ -242,7 +250,7 @@ func (r *Registry) Snapshots(model string) []Snapshot {
 			continue
 		}
 		var age time.Duration
-		stale := true
+		stale := l.info.Engine != EngineExternal
 		if !l.scrapedAt.IsZero() {
 			age = now.Sub(l.scrapedAt)
 			stale = age > r.cfg.StaleAfter
@@ -0,0 +1,89 @@
+package router
+
+import (
+	"encoding/json"
+	"fmt"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"sync/atomic"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/config"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
+)
+
+func TestALocalModelWithNoCapacityFallsBackToAnExternalProvider(t *testing.T) {
+	local := newFakeEngine(t, "local")
+	local.status.Store(http.StatusServiceUnavailable)
+
+	var calls atomic.Int64
+	var gotModel, gotAuth atomic.Value
+	provider := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path == "/metrics" {
+			t.Error("the provider was scraped for metrics")
+		}
+		calls.Add(1)
+		gotAuth.Store(r.Header.Get("Authorization"))
+		var body map[string]any
+		b, _ := io.ReadAll(r.Body)
+		_ = json.Unmarshal(b, &body)
+		gotModel.Store(fmt.Sprint(body["model"]))
+		w.Header().Set("Content-Type", "application/json")
+		fmt.Fprint(w, `{"model":"gpt-4o-mini","choices":[{"message":{"content":"from outside"}}],"usage":{"prompt_tokens":3,"completion_tokens":2}}`)
+	}))
+	t.Cleanup(provider.Close)
+
+	cfg := baseConfig(config.Endpoint{ID: "l", URL: local.srv.URL, Models: []string{"local"}})
+	cloud := catalog.Model{Name: "cloud", Source: "external",
+		External: &catalog.External{URL: provider.URL, Model: "gpt-4o-mini"}}
+	cfg.Models = []catalog.Model{catModel("local", nil, "cloud"), cloud}
+	rt := newTestRouter(t, cfg)
+	rt.SetEndpoints(t.Context(), []endpoint.Info{
+		{ID: "l", URL: local.srv.URL, Models: []string{"local"}},
+		{ID: "external:cloud", URL: provider.URL, Models: []string{"cloud"}, APIKey: "provider-key",
+			Engine: endpoint.EngineExternal, Pool: "external", Weight: 1, UpstreamModel: "gpt-4o-mini"},
+	})
+
+	rec := post(t, rt.Handler(), map[string]any{"model": "local", "messages": []any{}}, nil)
+	if rec.Code != http.StatusOK {
+		t.Fatalf("status %d: %s", rec.Code, rec.Body.String())
+	}
+	if calls.Load() != 1 {
+		t.Fatalf("the provider was called %d times", calls.Load())
+	}
+	if gotModel.Load() != "gpt-4o-mini" {
+		t.Errorf("the provider was asked for %v, not its own name for the model", gotModel.Load())
+	}
+	if gotAuth.Load() != "Bearer provider-key" {
+		t.Errorf("the provider got %v, not its key", gotAuth.Load())
+	}
+	if got := rec.Header().Get("x-llmgw-served-model"); got != "cloud" {
+		t.Errorf("the client should be told it was served by cloud, got %q", got)
+	}
+	// Never scraped, and never stale for it.
+	for _, s := range rt.reg.Snapshots("cloud") {
+		if s.Stale {
+			t.Error("an external endpoint reads as stale")
+		}
+	}
+}
+
+func TestAnExternalModelIsNotAModelWithProfiles(t *testing.T) {
+	good := catalog.Model{Name: "cloud", Source: "external",
+		External: &catalog.External{URL: "https://api.example.com", Model: "gpt-4o-mini"}}
+	if _, err := catalog.New([]catalog.Model{good}); err != nil {
+		t.Fatalf("a valid external model was refused: %v", err)
+	}
+	for name, m := range map[string]catalog.Model{
+		"profiles": {Name: "x", Source: "external", External: &catalog.External{URL: "https://a.b", Model: "m"},
+			Profiles: []catalog.Profile{{Name: "p", Engine: "llamacpp"}}},
+		"no url":   {Name: "x", Source: "external", External: &catalog.External{URL: "api.example.com", Model: "m"}},
+		"no model": {Name: "x", Source: "external", External: &catalog.External{URL: "https://a.b"}},
+	} {
+		if _, err := catalog.New([]catalog.Model{m}); err == nil {
+			t.Errorf("%s: accepted", name)
+		}
+	}
+}
@@ -88,6 +88,9 @@ func TestARefusalBeforeDispatchSaysWhetherWaitingHelps(t *testing.T) {
 		{"queued too long", func(w http.ResponseWriter) {
 			rt.rejectAdmission(w, &parsedRequest{Model: "m"}, admission.ErrQueueTimeout, time.Now())
 		}, 504, ""},
+		{"displaced for a higher band", func(w http.ResponseWriter) {
+			rt.rejectAdmission(w, &parsedRequest{Model: "m"}, admission.ErrDisplaced, time.Now())
+		}, 503, "5"},
 		{"shutting down", func(w http.ResponseWriter) {
 			rt.rejectAdmission(w, &parsedRequest{Model: "m"}, admission.ErrClosed, time.Now())
 		}, 503, ""},
@@ -60,9 +60,15 @@ func (r *Router) rejectAdmission(w http.ResponseWriter, req *parsedRequest, err
 		return
 	case errors.Is(err, admission.ErrQueueTimeout):
 		status = http.StatusGatewayTimeout
+	case errors.Is(err, admission.ErrDisplaced):
+		// Turned away for higher-priority work, not for good: said so the
+		// client backs off and comes back rather than giving up.
+		w.Header().Set("Retry-After", "5")
 	}
 	reason := "unavailable"
 	switch {
+	case errors.Is(err, admission.ErrDisplaced):
+		reason = "displaced"
 	case errors.Is(err, admission.ErrQueueFull):
 		reason = "queue_full"
 	case errors.Is(err, admission.ErrQueueTimeout):
@@ -185,6 +191,11 @@ func (r *Router) Handler() http.Handler {
 			pages.RegisterAdminUsers(mux, r.store)
 			// Quotas are written on the same tree the accounts page builds, so
 			// they are mounted together.
+			// Capacity quotas are the control plane's, reached through the
+			// same admin API the CLI uses.
+			if c, ok := r.clusterView.(web.CapacityAdmin); ok {
+				pages.SetCapacity(c)
+			}
 			pages.RegisterAdminQuotas(mux, r.store, r.acct)
 			// And the report of what that tree actually used.
 			pages.RegisterAdminUsage(mux, r.store, r.acct)
@@ -415,6 +426,12 @@ func (r *Router) handleInference(path string) http.HandlerFunc {
 // around the allowlist; matching only the canonical name would silently ignore
 // an allowlist written in the names clients actually use.
 func (r *Router) permits(p auth.Principal, requested string) bool {
+	// An adapter named in the allowlist allows that adapter: not its model,
+	// and not the model's other adapters, which the checks below would let
+	// through if the adapter's own name were not asked about first.
+	if _, ok := r.adapter(requested); ok && p.Allows(requested) {
+		return true
+	}
 	canonical, m := r.resolve(requested)
 	if p.Allows(canonical) {
 		return true
@@ -973,3 +990,44 @@ func (r *Router) rateLimitHeaders(w http.ResponseWriter, p auth.Principal) {
 		w.Header().Set("x-ratelimit-reset-tokens", reset)
 	}
 }
+
+// withModel is body with its model set to name, for an endpoint that knows the
+// model by another name — an external provider. The body unchanged when it is
+// not a JSON object, which the endpoint then refuses as it would have anyway.
+func withModel(body []byte, name string) []byte {
+	var m map[string]json.RawMessage
+	if err := json.Unmarshal(body, &m); err != nil {
+		return body
+	}
+	encoded, err := json.Marshal(name)
+	if err != nil {
+		return body
+	}
+	m["model"] = encoded
+	out, err := json.Marshal(m)
+	if err != nil {
+		return body
+	}
+	return out
+}
+
+// withAdapter is body asking for model with a LoRA adapter applied, in
+// llama.cpp's form: "lora": [{"id": index, "scale": scale}]. Adapters not
+// listed stay at zero, the scale the engine loads each one at.
+func withAdapter(body []byte, model string, index int, scale float64) []byte {
+	var m map[string]json.RawMessage
+	if err := json.Unmarshal(body, &m); err != nil {
+		return body
+	}
+	name, err1 := json.Marshal(model)
+	lora, err2 := json.Marshal([]map[string]any{{"id": index, "scale": scale}})
+	if err1 != nil || err2 != nil {
+		return body
+	}
+	m["model"], m["lora"] = name, lora
+	out, err := json.Marshal(m)
+	if err != nil {
+		return body
+	}
+	return out
+}
@@ -61,6 +61,8 @@ type Router struct {
 	// router's half of not having to. A request takes the map once and works
 	// from it, so a swap mid-request is invisible rather than a race.
 	models atomic.Pointer[map[string]catalog.Model]
+	// adapters are the LoRA adapters by the name clients ask for them by.
+	adapters atomic.Pointer[map[string]adapterRef]

 	// extraMetrics are collectors other components publish on /metrics.
 	extraMetrics []prometheus.Collector
@@ -315,14 +317,7 @@ func New(cfg config.Server, log *slog.Logger) (*Router, error) {
 	// all. A control plane pushes over the top of this at startup and on every
 	// change after.
 	if len(cfg.Models) > 0 {
-		seed := map[string]catalog.Model{}
-		for _, m := range cfg.Models {
-			seed[m.Name] = m
-			for _, a := range m.Aliases {
-				seed[a] = m
-			}
-		}
-		r.models.Store(&seed)
+		r.storeModels(cfg.Models)
 	}
 	return r, nil
 }
@@ -347,6 +342,7 @@ func admissionConfig(rc config.Router) admission.Config {
 		},
 		QueueTTL:             rc.Admission.QueueTTL,
 		QueueCapacityPerBand: rc.Admission.QueueCapacityPerBand,
+		QueueCapacityTotal:   rc.Admission.QueueCapacityTotal,
 		Interval:             rc.Admission.Interval,
 	}
 }
@@ -504,15 +500,44 @@ func (r *Router) modelMap() map[string]catalog.Model {
 // The catalog's counterpart to SetEndpoints, and pushed for the same reason:
 // the control plane owns what should exist, the router needs to know, and
 // neither has to be in the other's process for that to work.
-func (r *Router) SetCatalog(c *catalog.Catalog) {
+func (r *Router) SetCatalog(c *catalog.Catalog) { r.storeModels(c.Models()) }
+
+// storeModels makes models what names resolve to: each model by its name, its
+// aliases and its adapters. One place for it, whether the models came from
+// configuration at start or from the control plane since.
+func (r *Router) storeModels(models []catalog.Model) {
 	m := map[string]catalog.Model{}
-	for _, entry := range c.Models() {
+	ads := map[string]adapterRef{}
+	for _, entry := range models {
 		m[entry.Name] = entry
 		for _, a := range entry.Aliases {
 			m[a] = entry
 		}
+		// An adapter resolves to the model it runs on, as an alias does, and
+		// is applied to the request on the way there.
+		for i, a := range entry.Adapters {
+			m[a.Name] = entry
+			ads[a.Name] = adapterRef{base: entry.Name, index: i, scale: a.EffectiveScale()}
+		}
 	}
 	r.models.Store(&m)
+	r.adapters.Store(&ads)
+}
+
+// adapterRef is where an adapter runs and how it is applied.
+type adapterRef struct {
+	base  string
+	index int
+	scale float64
+}
+
+// adapter finds one by the name a client asked for.
+func (r *Router) adapter(name string) (adapterRef, bool) {
+	if p := r.adapters.Load(); p != nil {
+		a, ok := (*p)[name]
+		return a, ok
+	}
+	return adapterRef{}, false
 }

 // LastUsed reports when each endpoint was last sent a request, satisfying
@@ -714,10 +739,20 @@ func (r *Router) dispatchOne(ctx context.Context, w http.ResponseWriter, req *pa
 		uctx, uspan := tracer().Start(ctx, "upstream", trace.WithSpanKind(trace.SpanKindClient),
 			trace.WithAttributes(attribute.String("llmgw.endpoint", dec.Endpoint.ID),
 				attribute.String("url.full", dec.Endpoint.URL+req.Path)))
+		body := req.UpstreamBody
+		if dec.Endpoint.UpstreamModel != "" {
+			body = withModel(body, dec.Endpoint.UpstreamModel)
+		}
+		// An adapter asked for by name, served by its model's replicas: the
+		// engine is asked for the model, with the adapter applied. Not on a
+		// fallback to another model, which does not have it.
+		if a, ok := r.adapter(req.Model); ok && a.base == model {
+			body = withAdapter(body, model, a.index, a.scale)
+		}
 		out, perr := r.px.Do(uctx, w, proxy.Request{
 			URL:                dec.Endpoint.URL + req.Path,
 			APIKey:             dec.Endpoint.APIKey,
-			Body:               req.UpstreamBody,
+			Body:               body,
 			Header:             withTraceparent(uctx, req.Header),
 			Stream:             req.Stream,
 			SuppressUsageChunk: req.SuppressUsageChunk,
@@ -1,6 +1,9 @@
 package scheduler

 import (
+	"sort"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/router/admission"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/router/breaker"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
 )
@@ -51,34 +54,41 @@ func KVSaturated(ceiling float64) Filter {
 	}
 }

-// PoolPriority keeps only the endpoints of the highest-priority pool
-// (plan §7.3, the two-stage decision: pool first, then replica).
+// PoolPriority keeps the endpoints of one pool: the highest-priority one,
+// unless it is saturated and a lower one is not (plan §7.3, the two-stage
+// decision — pool first, then replica).
 //
-// Spillover arrives in phase 5, at which point this filter will fall through
-// to the next pool once the preferred one is saturated.
-func PoolPriority() Filter {
-	return poolPriorityFilter{}
+// Saturated is what admission means by it, the same formula over the same
+// thresholds, so the two layers agree on when a pool is full: the H100 pool is
+// used until it is, and only then the A100 one — never the A100 because it
+// happens to be emptier. When every pool is saturated the preferred one is
+// kept: moving to a pool that is just as full buys nothing and loses whatever
+// made the first preferred.
+func PoolPriority(targetWaiting, kvCeiling float64) Filter {
+	return poolPriorityFilter{t: admission.Thresholds{TargetWaitingPerReplica: targetWaiting, KVCeiling: kvCeiling}}
 }

-type poolPriorityFilter struct{}
+type poolPriorityFilter struct{ t admission.Thresholds }

 func (poolPriorityFilter) Name() string { return "pool-priority" }

-func (poolPriorityFilter) Filter(_ *Ctx, eps []endpoint.Snapshot) []endpoint.Snapshot {
+func (f poolPriorityFilter) Filter(_ *Ctx, eps []endpoint.Snapshot) []endpoint.Snapshot {
 	if len(eps) == 0 {
 		return eps
 	}
-	best := eps[0].Priority
-	for _, e := range eps[1:] {
-		if e.Priority > best {
-			best = e.Priority
-		}
-	}
-	out := make([]endpoint.Snapshot, 0, len(eps))
+	byPriority := map[int][]endpoint.Snapshot{}
+	var order []int
 	for _, e := range eps {
-		if e.Priority == best {
-			out = append(out, e)
+		if _, ok := byPriority[e.Priority]; !ok {
+			order = append(order, e.Priority)
+		}
+		byPriority[e.Priority] = append(byPriority[e.Priority], e)
+	}
+	sort.Sort(sort.Reverse(sort.IntSlice(order)))
+	for _, p := range order {
+		if admission.Saturation(byPriority[p], 0, f.t) < 1 {
+			return byPriority[p]
 		}
 	}
-	return out
+	return byPriority[order[0]]
 }
@@ -50,7 +50,7 @@ func Chat(c ProfileConfig) *Profile {
 	c.normalize()
 	return &Profile{
 		Name:    "chat",
-		Filters: []Filter{BreakerOpen(), PoolPriority()},
+		Filters: []Filter{BreakerOpen(), PoolPriority(c.TargetWaiting, c.KVCeiling)},
 		Scorers: c.withPrefix([]WeightedScorer{
 			{Session(c.SessionLoadFactor), 3.0},
 			{Queue(c.TargetWaiting), 1.0},
@@ -81,7 +81,7 @@ func Batch(c ProfileConfig) *Profile {
 	c.normalize()
 	return &Profile{
 		Name:    "batch",
-		Filters: []Filter{BreakerOpen(), PoolPriority()},
+		Filters: []Filter{BreakerOpen(), PoolPriority(c.TargetWaiting, c.KVCeiling)},
 		Scorers: []WeightedScorer{
 			{Queue(c.TargetWaiting), 3.0},
 			{KV(c.KVCeiling), 1.0},
@@ -99,7 +99,7 @@ func LongContext(c ProfileConfig) *Profile {
 	c.normalize()
 	return &Profile{
 		Name:    "long-context",
-		Filters: []Filter{BreakerOpen(), PoolPriority()},
+		Filters: []Filter{BreakerOpen(), PoolPriority(c.TargetWaiting, c.KVCeiling)},
 		// Prefix matching earns more here than anywhere else: a long prompt is
 		// mostly prefill, and prefill is what a cache hit skips.
 		Scorers: c.withPrefix([]WeightedScorer{
@@ -177,7 +177,7 @@ func TestStaleEndpointIsPenalizedNotDropped(t *testing.T) {
 func TestPoolPriority(t *testing.T) {
 	p := &Profile{
 		Name:    "t",
-		Filters: []Filter{PoolPriority()},
+		Filters: []Filter{PoolPriority(4, 0.9)},
 		Scorers: []WeightedScorer{{Queue(4), 1}},
 		Picker:  MaxScore(),
 	}
@@ -189,7 +189,8 @@ func TestPoolPriority(t *testing.T) {
 	if err != nil {
 		t.Fatal(err)
 	}
-	// The lower-priority pool must not win despite being emptier (spillover is phase 5).
+	// The lower-priority pool must not win for being emptier: the preferred
+	// one is not saturated (3 waiting of a healthy 4).
 	if d.Endpoint.ID != "h100" {
 		t.Errorf("pool priority was ignored: %s", d.Endpoint.ID)
 	}
@@ -383,3 +384,36 @@ func TestBuiltinProfilesSchedule(t *testing.T) {
 		}
 	}
 }
+
+func TestASaturatedPoolSpillsToTheNext(t *testing.T) {
+	p := &Profile{Name: "t", Filters: []Filter{PoolPriority(4, 0.9)},
+		Scorers: []WeightedScorer{{Queue(4), 1}}, Picker: MaxScore()}
+	h100 := func(w int) endpoint.Snapshot {
+		return ep("h100", waiting(w), func(s *endpoint.Snapshot) { s.Priority = 100; s.Pool = "ib-a/h100" })
+	}
+	a100 := func(w int) endpoint.Snapshot {
+		return ep("a100", waiting(w), func(s *endpoint.Snapshot) { s.Priority = 50; s.Pool = "ib-b/a100" })
+	}
+	pick := func(eps ...endpoint.Snapshot) string {
+		d, err := p.Schedule(newCtx(Request{Model: "m"}), eps)
+		if err != nil {
+			t.Fatal(err)
+		}
+		return d.Endpoint.ID
+	}
+	if got := pick(h100(8), a100(0)); got != "a100" {
+		t.Errorf("the preferred pool at twice its healthy queue did not spill: %s", got)
+	}
+	if got := pick(h100(8), a100(6)); got != "h100" {
+		t.Errorf("both pools saturated: the preferred one should be kept, got %s", got)
+	}
+	if got := pick(h100(4), a100(0)); got != "a100" {
+		t.Errorf("at exactly its healthy limit a pool is full: %s", got)
+	}
+	// KV pressure saturates a pool as queue depth does.
+	full := h100(0)
+	full.Stats.KVCacheUsage = 0.95
+	if got := pick(full, a100(0)); got != "a100" {
+		t.Errorf("a pool out of KV cache did not spill: %s", got)
+	}
+}
@@ -96,6 +96,9 @@ func (p *Poller) Sync(ctx context.Context) {

 	want := make(map[string]endpoint.Info, len(infos))
 	for _, in := range infos {
+		if in.Engine == endpoint.EngineExternal {
+			continue // a provider elsewhere publishes no /metrics
+		}
 		want[in.ID] = in
 	}

@@ -29,6 +29,9 @@ var copyOrder = []string{
 	"key_claims",
 	"sessions",
 	"limits",
+	// Beside limits, and like them keyed by scope with no foreign key: a
+	// quota on a scope that is later removed is inert, not an error.
+	"capacity_quotas",
 	"usage",
 	"models",
 	// No foreign key to models, deliberately: a measurement outlives the
@@ -0,0 +1,19 @@
+-- Capacity a part of the tenancy tree is guaranteed, and the most it may hold.
+--
+-- Beside limits rather than in it: limits are counted on the request path, per
+-- key, in requests and tokens; these are counted by the reconciler, per model,
+-- in the GPUs, cores and memory its replicas hold. A NULL is a resource this
+-- row does not govern: guaranteed nothing, or limited by nothing.
+CREATE TABLE capacity_quotas (
+  scope_type      TEXT NOT NULL,
+  scope_id        TEXT NOT NULL,
+  gpus            INTEGER,
+  cpu_cores       INTEGER,
+  memory_gb       INTEGER,
+  limit_gpus      INTEGER,
+  limit_cpu_cores INTEGER,
+  limit_memory_gb INTEGER,
+  updated_at      BIGINT NOT NULL,
+  updated_by      TEXT NOT NULL,
+  PRIMARY KEY (scope_type, scope_id)
+);
@@ -0,0 +1,19 @@
+-- Capacity a part of the tenancy tree is guaranteed, and the most it may hold.
+--
+-- Beside limits rather than in it: limits are counted on the request path, per
+-- key, in requests and tokens; these are counted by the reconciler, per model,
+-- in the GPUs, cores and memory its replicas hold. A NULL is a resource this
+-- row does not govern: guaranteed nothing, or limited by nothing.
+CREATE TABLE capacity_quotas (
+  scope_type      TEXT NOT NULL,
+  scope_id        TEXT NOT NULL,
+  gpus            INTEGER,
+  cpu_cores       INTEGER,
+  memory_gb       INTEGER,
+  limit_gpus      INTEGER,
+  limit_cpu_cores INTEGER,
+  limit_memory_gb INTEGER,
+  updated_at      BIGINT NOT NULL,
+  updated_by      TEXT NOT NULL,
+  PRIMARY KEY (scope_type, scope_id)
+);
@@ -0,0 +1,127 @@
+package store
+
+import (
+	"context"
+	"database/sql"
+	"fmt"
+	"time"
+)
+
+// Resources are amounts of the three things a replica holds of a cluster.
+// Nil is not governed: nothing guaranteed, or nothing limited.
+type Resources struct {
+	GPUs     *int `json:"gpus,omitempty"`
+	CPUCores *int `json:"cpu_cores,omitempty"`
+	MemoryGB *int `json:"memory_gb,omitempty"`
+}
+
+// Any reports whether any resource is governed.
+func (r Resources) Any() bool { return r.GPUs != nil || r.CPUCores != nil || r.MemoryGB != nil }
+
+// CapacityQuota is what one part of the tenancy tree is guaranteed of the
+// cluster, and the most it may hold.
+type CapacityQuota struct {
+	Scope   Scope  `json:"scope"`
+	ScopeID string `json:"scope_id"`
+	// Guarantee is held for the scope: while it holds less, what others
+	// borrowed beyond their own guarantees is taken back for it.
+	Guarantee Resources `json:"guarantee"`
+	// Limit is the most it may hold, borrowing included.
+	Limit     Resources `json:"limit"`
+	UpdatedAt time.Time `json:"updated_at"`
+	UpdatedBy string    `json:"updated_by"`
+}
+
+// PutCapacityQuota sets a scope's quota, replacing what was there.
+func (s *DB) PutCapacityQuota(ctx context.Context, q CapacityQuota) error {
+	if !validMembershipScope(q.Scope) || q.ScopeID == "" {
+		return fmt.Errorf("a capacity quota belongs to an organization, team or project")
+	}
+	for _, v := range []*int{q.Guarantee.GPUs, q.Guarantee.CPUCores, q.Guarantee.MemoryGB,
+		q.Limit.GPUs, q.Limit.CPUCores, q.Limit.MemoryGB} {
+		if v != nil && *v < 0 {
+			return fmt.Errorf("a quota cannot be negative")
+		}
+	}
+	if q.UpdatedAt.IsZero() {
+		q.UpdatedAt = time.Now()
+	}
+	_, err := s.db.ExecContext(ctx,
+		`INSERT INTO capacity_quotas (scope_type, scope_id, gpus, cpu_cores, memory_gb,
+		   limit_gpus, limit_cpu_cores, limit_memory_gb, updated_at, updated_by)
+		 VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+		 ON CONFLICT (scope_type, scope_id) DO UPDATE SET
+		   gpus = excluded.gpus, cpu_cores = excluded.cpu_cores, memory_gb = excluded.memory_gb,
+		   limit_gpus = excluded.limit_gpus, limit_cpu_cores = excluded.limit_cpu_cores,
+		   limit_memory_gb = excluded.limit_memory_gb,
+		   updated_at = excluded.updated_at, updated_by = excluded.updated_by`,
+		string(q.Scope), q.ScopeID,
+		nullInt(q.Guarantee.GPUs), nullInt(q.Guarantee.CPUCores), nullInt(q.Guarantee.MemoryGB),
+		nullInt(q.Limit.GPUs), nullInt(q.Limit.CPUCores), nullInt(q.Limit.MemoryGB),
+		q.UpdatedAt.UnixMilli(), q.UpdatedBy)
+	return err
+}
+
+// CapacityQuotas lists every quota.
+func (s *DB) CapacityQuotas(ctx context.Context) ([]CapacityQuota, error) {
+	return queryAll(ctx, s.db,
+		`SELECT scope_type, scope_id, gpus, cpu_cores, memory_gb,
+		   limit_gpus, limit_cpu_cores, limit_memory_gb, updated_at, updated_by
+		 FROM capacity_quotas ORDER BY scope_type, scope_id`, nil,
+		func(rows *sql.Rows) (CapacityQuota, error) {
+			var q CapacityQuota
+			var scope string
+			var g [6]sql.NullInt64
+			var at int64
+			err := rows.Scan(&scope, &q.ScopeID, &g[0], &g[1], &g[2], &g[3], &g[4], &g[5], &at, &q.UpdatedBy)
+			q.Scope = Scope(scope)
+			q.Guarantee = Resources{GPUs: intOf(g[0]), CPUCores: intOf(g[1]), MemoryGB: intOf(g[2])}
+			q.Limit = Resources{GPUs: intOf(g[3]), CPUCores: intOf(g[4]), MemoryGB: intOf(g[5])}
+			q.UpdatedAt = time.UnixMilli(at)
+			return q, err
+		})
+}
+
+// DeleteCapacityQuota removes a scope's quota.
+func (s *DB) DeleteCapacityQuota(ctx context.Context, scope Scope, id string) error {
+	_, err := s.db.ExecContext(ctx,
+		`DELETE FROM capacity_quotas WHERE scope_type = ? AND scope_id = ?`, string(scope), id)
+	return err
+}
+
+// ScopeParents maps every team and project to the scopes above it, nearest
+// first: a project to its team and organization, a team to its organization.
+// Keys and values are written scope:id.
+func (s *DB) ScopeParents(ctx context.Context) (map[string][]string, error) {
+	out := map[string][]string{}
+	teams, err := s.AllTeams(ctx)
+	if err != nil {
+		return nil, err
+	}
+	orgOf := map[string]string{}
+	for _, t := range teams {
+		orgOf[t.ID] = t.OrgID
+		out["team:"+t.ID] = []string{"org:" + t.OrgID}
+	}
+	type pt struct{ project, team string }
+	rows, err := queryAll(ctx, s.db, `SELECT id, team_id FROM projects`, nil,
+		func(r *sql.Rows) (pt, error) {
+			var p pt
+			return p, r.Scan(&p.project, &p.team)
+		})
+	if err != nil {
+		return nil, err
+	}
+	for _, p := range rows {
+		out["project:"+p.project] = []string{"team:" + p.team, "org:" + orgOf[p.team]}
+	}
+	return out, nil
+}
+
+func intOf(n sql.NullInt64) *int {
+	if !n.Valid {
+		return nil
+	}
+	v := int(n.Int64)
+	return &v
+}
@@ -0,0 +1,63 @@
+package store
+
+import "testing"
+
+func ip(v int) *int { return &v }
+
+func TestAQuotaKeepsWhatItGovernsAndNothingElse(t *testing.T) {
+	s := tenancyStore(t)
+	org, _ := s.CreateOrg(t.Context(), "Acme")
+	q := CapacityQuota{Scope: ScopeOrg, ScopeID: org, UpdatedBy: "ops",
+		Guarantee: Resources{GPUs: ip(8)}, Limit: Resources{GPUs: ip(16), CPUCores: ip(0)}}
+	if err := s.PutCapacityQuota(t.Context(), q); err != nil {
+		t.Fatal(err)
+	}
+	all, err := s.CapacityQuotas(t.Context())
+	if err != nil || len(all) != 1 {
+		t.Fatalf("%v %+v", err, all)
+	}
+	got := all[0]
+	if *got.Guarantee.GPUs != 8 || got.Guarantee.CPUCores != nil || got.Guarantee.MemoryGB != nil {
+		t.Errorf("guarantee %+v: only GPUs were set", got.Guarantee)
+	}
+	// Zero is a limit of nothing, not an absent one.
+	if *got.Limit.GPUs != 16 || got.Limit.CPUCores == nil || *got.Limit.CPUCores != 0 {
+		t.Errorf("limit %+v", got.Limit)
+	}
+	q.Guarantee = Resources{MemoryGB: ip(64)}
+	if err := s.PutCapacityQuota(t.Context(), q); err != nil {
+		t.Fatal(err)
+	}
+	all, _ = s.CapacityQuotas(t.Context())
+	if len(all) != 1 || all[0].Guarantee.GPUs != nil || *all[0].Guarantee.MemoryGB != 64 {
+		t.Errorf("a second put should replace the first: %+v", all)
+	}
+	if err := s.DeleteCapacityQuota(t.Context(), ScopeOrg, org); err != nil {
+		t.Fatal(err)
+	}
+	if all, _ = s.CapacityQuotas(t.Context()); len(all) != 0 {
+		t.Error("the quota survived its deletion")
+	}
+	if err := s.PutCapacityQuota(t.Context(), CapacityQuota{Scope: ScopeOrg, ScopeID: org,
+		Guarantee: Resources{GPUs: ip(-1)}}); err == nil {
+		t.Error("a negative quota was stored")
+	}
+}
+
+func TestEveryScopeKnowsWhatIsAboveIt(t *testing.T) {
+	s := tenancyStore(t)
+	org, _ := s.CreateOrg(t.Context(), "Acme")
+	team, _ := s.CreateTeam(t.Context(), org, "ml")
+	proj, _ := s.CreateProject(t.Context(), team, "training")
+	p, err := s.ScopeParents(t.Context())
+	if err != nil {
+		t.Fatal(err)
+	}
+	want := []string{"team:" + team, "org:" + org}
+	if got := p["project:"+proj]; len(got) != 2 || got[0] != want[0] || got[1] != want[1] {
+		t.Errorf("project's parents %v, expected %v", got, want)
+	}
+	if got := p["team:"+team]; len(got) != 1 || got[0] != "org:"+org {
+		t.Errorf("team's parents %v", got)
+	}
+}
@@ -9,6 +9,7 @@ import (
 	"net/http"
 	"strconv"
 	"strings"
+	"sync"
 	"time"

 	"gitea.mynodes.xyz/ilker/llmgw/internal/account"
@@ -61,6 +62,8 @@ type AccountPages struct {
 	hasMirror bool
 	// hasQuotas is set when /admin/quotas is registered.
 	hasQuotas bool
+	// capacity sets capacity quotas, from the quotas page; nil leaves them off.
+	capacity CapacityAdmin
 	// personalOff refuses personal keys; see SetPersonalOff.
 	personalOff bool
 	// personalDefaults is what personal use falls under by default.
@@ -74,6 +77,9 @@ type AccountPages struct {
 	priced Priced
 	// registry browses a model registry, or is nil where there is no way out.
 	registry Registry
+	// kinds remembers what each held GGUF file is, by its address: a held
+	// version never changes, so it is read once.
+	kinds sync.Map
 	// catalog puts a mirrored model into service; fleet says whether anything
 	// could run it. Both nil where the mirror page is read-only.
 	catalog CatalogAdmin
@@ -0,0 +1,138 @@
+package web
+
+import (
+	"context"
+	"fmt"
+	"net/http"
+	"strconv"
+	"strings"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// CapacityAdmin sets what parts of the tenancy tree are guaranteed of the
+// cluster and may hold. The admin API implements it; the page and the CLI go
+// through the same checks.
+type CapacityAdmin interface {
+	QuotaViews(ctx context.Context) ([]adminapi.QuotaView, error)
+	SetQuota(ctx context.Context, req adminapi.QuotaRequest, by string) (string, error)
+	ClearQuota(ctx context.Context, scope string) (string, error)
+	Scopes(ctx context.Context) ([]adminapi.ScopeName, error)
+}
+
+// SetCapacity gives the quotas page capacity quotas to show and set. Without
+// it the section is left off.
+func (p *AccountPages) SetCapacity(c CapacityAdmin) { p.capacity = c }
+
+// capacityRow is one scope's capacity quota as the page shows it.
+type capacityRow struct {
+	adminapi.QuotaView
+	GPUs, Cores, Memory string
+}
+
+// capacityForm is what the form starts with: a scope's quota to change, or
+// nothing for a new one.
+type capacityForm struct {
+	Scope               string
+	GPUs, Cores, Memory string
+	LGPUs, LCores, LMem string
+}
+
+func (p *AccountPages) capacitySection(ctx context.Context, d *adminQuotasData, editing string) {
+	if p.capacity == nil {
+		return
+	}
+	d.HasCapacity = true
+	views, err := p.capacity.QuotaViews(ctx)
+	if err != nil {
+		d.Error = err.Error()
+		return
+	}
+	for _, v := range views {
+		d.Capacity = append(d.Capacity, capacityRow{QuotaView: v,
+			GPUs:   capCell(v.Holds.GPUs, v.Guarantee.GPUs, v.Limit.GPUs),
+			Cores:  capCell(v.Holds.CPUCores, v.Guarantee.CPUCores, v.Limit.CPUCores),
+			Memory: capCell(v.Holds.MemoryGB, v.Guarantee.MemoryGB, v.Limit.MemoryGB)})
+		if v.Scope == editing {
+			d.CapForm = capacityForm{Scope: v.Scope,
+				GPUs: intStr(v.Guarantee.GPUs), Cores: intStr(v.Guarantee.CPUCores), Memory: intStr(v.Guarantee.MemoryGB),
+				LGPUs: intStr(v.Limit.GPUs), LCores: intStr(v.Limit.CPUCores), LMem: intStr(v.Limit.MemoryGB)}
+		}
+	}
+	if d.Scopes, err = p.capacity.Scopes(ctx); err != nil {
+		d.Error = err.Error()
+	}
+}
+
+// capCell is "6 of 8 (at most 16)": held, guaranteed, limit.
+func capCell(holds int, g, l *int) string {
+	if g == nil && l == nil {
+		return ""
+	}
+	out := strconv.Itoa(holds)
+	if g != nil {
+		out += " of " + strconv.Itoa(*g)
+	}
+	if l != nil {
+		out += fmt.Sprintf(" (at most %d)", *l)
+	}
+	return out
+}
+
+func intStr(v *int) string {
+	if v == nil {
+		return ""
+	}
+	return strconv.Itoa(*v)
+}
+
+// applyCapacity sets or clears one scope's capacity quota from the form.
+func (p *AccountPages) applyCapacity(r *http.Request, admin store.User) (string, error) {
+	scope := strings.TrimSpace(r.FormValue("scope"))
+	if scope == "" {
+		return "", fmt.Errorf("choose where the quota applies")
+	}
+	if r.FormValue("clear") != "" {
+		key, err := p.capacity.ClearQuota(r.Context(), scope)
+		if err != nil {
+			return "", err
+		}
+		return "capacity quota removed from " + key, nil
+	}
+	field := func(name, what string) (*int, error) {
+		v := strings.TrimSpace(r.FormValue(name))
+		if v == "" {
+			return nil, nil
+		}
+		n, err := strconv.Atoi(v)
+		if err != nil || n < 0 {
+			return nil, fmt.Errorf("%s must be a whole number, or blank for none", what)
+		}
+		return &n, nil
+	}
+	var req adminapi.QuotaRequest
+	req.Scope = scope
+	var err error
+	for _, f := range []struct {
+		dst  **int
+		name string
+		what string
+	}{
+		{&req.Guarantee.GPUs, "gpus", "guaranteed GPUs"},
+		{&req.Guarantee.CPUCores, "cores", "guaranteed cores"},
+		{&req.Guarantee.MemoryGB, "memory", "guaranteed memory"},
+		{&req.Limit.GPUs, "limit_gpus", "the GPU limit"},
+		{&req.Limit.CPUCores, "limit_cores", "the core limit"},
+		{&req.Limit.MemoryGB, "limit_memory", "the memory limit"},
+	} {
+		if *f.dst, err = field(f.name, f.what); err != nil {
+			return "", err
+		}
+	}
+	key, err := p.capacity.SetQuota(r.Context(), req, admin.Email)
+	if err != nil {
+		return "", err
+	}
+	return "capacity quota set on " + key + "; it takes effect within half a minute", nil
+}
@@ -0,0 +1,88 @@
+package web
+
+import (
+	"context"
+	"errors"
+	"log/slog"
+	"net/http"
+	"net/url"
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/quota"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+type fakeCapacity struct {
+	views   []adminapi.QuotaView
+	set     []adminapi.QuotaRequest
+	cleared []string
+}
+
+func (f *fakeCapacity) QuotaViews(context.Context) ([]adminapi.QuotaView, error) { return f.views, nil }
+func (f *fakeCapacity) SetQuota(_ context.Context, r adminapi.QuotaRequest, _ string) (string, error) {
+	if r.Scope == "team:nope" {
+		return "", errors.New(`there is no team "nope"`)
+	}
+	f.set = append(f.set, r)
+	return r.Scope, nil
+}
+func (f *fakeCapacity) ClearQuota(_ context.Context, s string) (string, error) {
+	f.cleared = append(f.cleared, s)
+	return s, nil
+}
+func (f *fakeCapacity) Scopes(context.Context) ([]adminapi.ScopeName, error) {
+	return []adminapi.ScopeName{{Key: "org:o1", Name: "Acme"}, {Key: "team:t1", Name: "Acme / ml"}}, nil
+}
+
+func TestCapacityQuotasAreShownAndSetFromTheQuotasPage(t *testing.T) {
+	eight, sixteen := 8, 16
+	fc := &fakeCapacity{views: []adminapi.QuotaView{{Scope: "team:t1", Name: "Acme / ml",
+		Guarantee: store.Resources{GPUs: &eight}, Limit: store.Resources{GPUs: &sixteen},
+		Holds: quota.Amount{GPUs: 6}, UpdatedBy: "ops@acme"}}}
+	a := fakeAccounts{user: store.User{ID: "u1", Email: "a@example.com", IsAdmin: true}}
+	p := NewAccountPages(a, nil, nil, "v0", slog.New(slog.DiscardHandler))
+	p.SetCapacity(fc)
+	mux := http.NewServeMux()
+	p.RegisterAdminQuotas(mux, &fakeQuotas{}, pricedYes{})
+
+	body := mirrorGet(t, mux, "/admin/quotas").Body.String()
+	for _, want := range []string{`id="capacity"`, "Acme / ml", "6 of 8 (at most 16)", "ops@acme",
+		`<option value="team:t1">Acme / ml</option>`} {
+		if !strings.Contains(body, want) {
+			t.Errorf("the page lacks %q", want)
+		}
+	}
+	// "change" fills the form with the scope's figures.
+	edit := mirrorGet(t, mux, "/admin/quotas?cap=team:t1").Body.String()
+	if !strings.Contains(edit, `<option value="team:t1" selected>`) || !strings.Contains(edit, `name="limit_gpus" value="16"`) {
+		t.Error("change did not open the form on the scope's quota")
+	}
+
+	res := mirrorPost(t, mux, "/admin/quotas/capacity", url.Values{"csrf": {"tok"}, "scope": {"team:t1"},
+		"gpus": {"8"}, "cores": {""}, "limit_memory": {"64"}}).Body.String()
+	if len(fc.set) != 1 || *fc.set[0].Guarantee.GPUs != 8 || fc.set[0].Guarantee.CPUCores != nil || *fc.set[0].Limit.MemoryGB != 64 {
+		t.Errorf("sent %+v", fc.set)
+	}
+	if !strings.Contains(res, "capacity quota set on team:t1") {
+		t.Errorf("no word of it: %s", res)
+	}
+	for _, bad := range []url.Values{{"scope": {"team:t1"}, "gpus": {"eight"}}, {"scope": {"team:nope"}, "gpus": {"1"}}, {"gpus": {"1"}}} {
+		bad.Set("csrf", "tok")
+		n := len(fc.set)
+		out := mirrorPost(t, mux, "/admin/quotas/capacity", bad).Body.String()
+		if len(fc.set) != n || !strings.Contains(out, `class="error"`) {
+			t.Errorf("%v was not refused with a reason", bad)
+		}
+	}
+	mirrorPost(t, mux, "/admin/quotas/capacity", url.Values{"csrf": {"tok"}, "scope": {"team:t1"}, "clear": {"1"}})
+	if len(fc.cleared) != 1 || fc.cleared[0] != "team:t1" {
+		t.Errorf("cleared %v", fc.cleared)
+	}
+	// Without a CSRF token nothing happens.
+	mirrorPost(t, mux, "/admin/quotas/capacity", url.Values{"scope": {"team:t1"}, "clear": {"1"}})
+	if len(fc.cleared) != 1 {
+		t.Error("a change was made without the CSRF token")
+	}
+}
@@ -11,6 +11,7 @@ import (
 	"sort"
 	"strconv"
 	"strings"
+	"sync"
 	"time"

 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
@@ -46,6 +47,10 @@ type Registry interface {

 // repoSet is one thing worth fetching: a file, or every shard of one.
 type repoSet struct {
+	// Kind is what the file says it is, read from its first bytes: "adapter"
+	// for a LoRA adapter, "model", or empty when it was not read. Arch is the
+	// architecture it names, which an adapter must share with its model.
+	Kind, Arch string
 	// Verdict is whether the nodes' llama.cpp can load it, from the
 	// architecture the registry reports; a set that cannot be served is
 	// shown with the reason and no fetch button.
@@ -71,6 +76,13 @@ type modelUse struct {
 	Replicas int
 	// Min and Max are its scaling as the catalog holds it.
 	Min, Max int
+	// Class is what it set, empty for its default; EffClass what that comes
+	// to. Owner is org:ID or empty for shared. MemoryGB and CPUCores are what
+	// a replica of its first profile reserves.
+	Class, EffClass, Owner string
+	MemoryGB, CPUCores     int
+	// Adapters are the LoRA adapters it serves.
+	Adapters []catalog.Adapter
 	// Ready, Wanted and Nodes are how it runs now, for the row itself:
 	// replicas ready of those it should have, and the nodes they are on.
 	// Fleet is false where nothing could be asked, so the row says nothing
@@ -100,6 +112,9 @@ type mirrorRow struct {
 	// Detail is the card opened beneath the row: what runs it, on which
 	// nodes, and how many replicas it runs.
 	Detail *heldDetail
+	// Kind is what a single GGUF file is — "adapter" or "model" — and Arch
+	// its architecture; empty when not read.
+	Kind, Arch string
 	// Open is the row's ?open= value.
 	Open string

@@ -293,7 +308,8 @@ func (p *AccountPages) RegisterAdminMirror(mux *http.ServeMux, d MirrorDeps) {
 				return name + " now runs all the time", nil
 			}))

-		mux.HandleFunc("POST /admin/mirror/replicas", p.mirrorAction(m, p.setReplicas))
+		mux.HandleFunc("POST /admin/mirror/replicas", p.mirrorAction(m, p.setModel))
+		mux.HandleFunc("POST /admin/mirror/adapters", p.mirrorAction(m, p.setAdapters))

 		mux.HandleFunc("POST /admin/mirror/serve", p.mirrorAction(m,
 			func(ctx context.Context, admin store.User, r *http.Request) (string, error) {
@@ -463,8 +479,13 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
 			u := modelUse{
 				Name: m.Name, Mode: modeOf(m.Scaling), Replicas: max(m.Scaling.Max, 1),
 				Min: m.Scaling.Min, Max: m.Scaling.Max,
+				Class: m.Class, EffClass: m.EffectiveClass(), Owner: m.Owner,
+				Adapters: m.Adapters,
 				Declared: p.catalog.Declared(m.Name),
 			}
+			if len(m.Profiles) > 0 {
+				u.MemoryGB, u.CPUCores = m.Profiles[0].MinMemoryGB, m.Profiles[0].CPUCores
+			}
 			if v, ok := views[m.Name]; ok {
 				u.Fleet, u.Ready, u.Wanted = true, v.Ready, v.Desired
 				u.Parked, u.Unplaceable = v.Parked, v.Unplaceable
@@ -478,13 +499,16 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
 		}
 	}

+	p.heldKinds(r.Context(), d.Rows)
+
 	// The one row whose card is open. From the form too, so a change made in
 	// the card comes back with the card still open to show it.
 	open := r.FormValue("open")
 	for i := range d.Rows {
 		d.Rows[i].Open = openKey(d.Rows[i].Model, d.Rows[i].Version)
 		if open != "" && d.Rows[i].Open == open && d.Rows[i].State == mirror.Present {
-			d.Rows[i].Detail = p.heldDetailFor(d.Rows[i])
+			d.Rows[i].Detail = p.heldDetailFor(r.Context(), d.Rows[i])
+			d.Rows[i].Detail.addAdapterChoices(d.Rows[i], d.Rows)
 		}
 	}

@@ -746,6 +770,46 @@ func (p *AccountPages) openRepo(r *http.Request, d *adminMirrorData, repo string
 		}
 		d.RepoSets = append(d.RepoSets, rs)
 	}
+	p.probeKinds(r.Context(), d.RepoSets)
+}
+
+// kindOf reads what a GGUF file is; a variable so tests need no registry.
+var kindOf = func(ctx context.Context, url string) (gguf.Header, error) {
+	return gguf.KindFromURL(ctx, nil, url)
+}
+
+// probeKinds reads the start of each single-file GGUF set, to tell LoRA
+// adapters from models: a repository tagged "lora" holds a model with the
+// adapter merged in as often as an adapter, and only the file says which.
+// A few at once and a few in all, since each is a request to the registry.
+func (p *AccountPages) probeKinds(ctx context.Context, sets []repoSet) {
+	const most, together = 12, 6
+	sem := make(chan struct{}, together)
+	var wg sync.WaitGroup
+	n := 0
+	for i := range sets {
+		s := &sets[i]
+		if len(s.Sources) != 1 || !gguf.IsGGUF(s.Sources[0].Path) || n == most {
+			continue
+		}
+		n++
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			sem <- struct{}{}
+			defer func() { <-sem }()
+			cctx, cancel := context.WithTimeout(ctx, 10*time.Second)
+			defer cancel()
+			if h, err := kindOf(cctx, s.Sources[0].URL); err == nil {
+				s.Arch = h.Architecture
+				s.Kind = "model"
+				if h.IsAdapter() {
+					s.Kind = "adapter"
+				}
+			}
+		}()
+	}
+	wg.Wait()
 }

 // defaultModelName is the catalog name a repository suggests.
@@ -9,6 +9,7 @@ import (
 	"strings"
 	"time"

+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
 )

@@ -129,12 +130,17 @@ type adminQuotasData struct {
 	// Editing is the "scope:id" whose form is open, so the template can keep
 	// the link on every other row.
 	Editing string
-	Since   string
-	Message string
-	Error   string
-	CSRF    string
-	Version string
-	Now     string
+	// The capacity section: quotas on GPUs, cores and memory.
+	HasCapacity bool
+	Capacity    []capacityRow
+	Scopes      []adminapi.ScopeName
+	CapForm     capacityForm
+	Since       string
+	Message     string
+	Error       string
+	CSRF        string
+	Version     string
+	Now         string
 }

 // RegisterAdminQuotas mounts quota administration under /admin/quotas.
@@ -176,6 +182,24 @@ func (p *AccountPages) RegisterAdminQuotas(mux *http.ServeMux, s QuotaAdmin, pri
 			}
 			p.renderQuotas(w, r, s, msg, "")
 		}))
+
+	mux.HandleFunc("POST /admin/quotas/capacity", p.adminOnly(
+		func(w http.ResponseWriter, r *http.Request, admin store.User) {
+			sess, _, _ := p.session(r)
+			if p.capacity == nil || !p.accounts.CheckCSRF(sess, r.FormValue("csrf")) {
+				http.Error(w, "bad request", http.StatusBadRequest)
+				return
+			}
+			msg, err := p.applyCapacity(r, admin)
+			if err != nil {
+				p.log.Warn("a capacity quota change was refused",
+					"admin", admin.Email, "scope", r.FormValue("scope"), "err", err)
+				p.renderQuotas(w, r, s, "", err.Error())
+				return
+			}
+			p.log.Info("capacity quota changed", "admin", admin.Email, "what", msg)
+			p.renderQuotas(w, r, s, msg, "")
+		}))
 }

 // applyQuota writes one scope's limits, and says what it did.
@@ -259,6 +283,7 @@ func (p *AccountPages) renderQuotas(w http.ResponseWriter, r *http.Request,

 	d.BudgetsBind = p.priced != nil && p.priced.AnythingPriced()
 	d.Editing = r.URL.Query().Get("edit")
+	p.capacitySection(r.Context(), &d, r.FormValue("cap"))

 	limits, err := s.AllLimits(r.Context())
 	if err != nil {
@@ -22,6 +22,12 @@ type CatalogAdmin interface {
 	// Scale changes how a model added at runtime runs: always, on demand, or
 	// not at all. One declared in configuration is refused.
 	Scale(ctx context.Context, name string, sc catalog.Scaling, by string) (catalog.Model, error)
+	// SetClass, SetOwner and Resources change what a model's capacity is
+	// worth, whose quota it counts against, and what a replica reserves.
+	SetClass(ctx context.Context, name, class, by string) (catalog.Model, error)
+	SetOwner(ctx context.Context, name, owner, by string) (catalog.Model, error)
+	Resources(ctx context.Context, name, profile string, memoryGB, cpuCores int, by string) (catalog.Model, error)
+	SetAdapters(ctx context.Context, name string, adapters []catalog.Adapter, by string) (catalog.Model, error)
 }

 // serveForm is the offer to put one mirrored version into service.
@@ -49,6 +55,9 @@ type serveForm struct {
 	// 3.5 GB for a KV pool. Nothing was wrong with placement — it was never
 	// told what the model needed.
 	MinMemoryGB int
+	// CPUCores is the cores a replica computes on: half the smallest node it
+	// fits on, or zero when no node says how many it has.
+	CPUCores int
 	// MemoryBasis says how MinMemoryGB was arrived at, for the form to show.
 	MemoryBasis string
 	// CtxPool is the engine context pool the form starts with, and the one
@@ -152,6 +161,15 @@ func planServe(st mirror.Status, files []mirror.Status, base string,
 			defaultCtxPool, header.BlockCount, kvHeads,
 			float64(int64(1<<30)+weights/20)/(1<<30))
 	}
+	if fleet != nil {
+		var cpus []int
+		for _, n := range fleet.Nodes() {
+			if n.State == "ready" && (n.MemoryGB == 0 || n.MemoryGB >= f.MinMemoryGB) {
+				cpus = append(cpus, n.CPUCount)
+			}
+		}
+		f.CPUCores = catalog.SuggestCores(cpus)
+	}
 	if biggest := biggestNode(fleet); biggest > 0 && f.MinMemoryGB > biggest {
 		f.Warning = strings.TrimSuffix(f.Warning, ".") +
 			fmt.Sprintf("%sthe largest node has %d GB and this needs about %d",
@@ -277,6 +295,11 @@ func buildModel(get func(string) string) (catalog.Model, error) {
 			return m, err
 		}
 	}
+	if v := strings.TrimSpace(get("cpu_cores")); v != "" {
+		if p.CPUCores, err = positive(v, "the cores"); err != nil {
+			return m, err
+		}
+	}
 	if k, v := strings.TrimSpace(get("selector_key")), strings.TrimSpace(get("selector_value")); k != "" && v != "" {
 		// What keeps a large model off the small nodes. A CPU profile has no
 		// card to be short of, so without this every ready node looks like a
@@ -3,6 +3,7 @@ package web
 import (
 	"context"
 	"errors"
+	"fmt"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
@@ -15,12 +16,16 @@ import (
 )

 type fakeCatalog struct {
-	added     []catalog.Model
-	declared  string
-	sources   map[string][]catalog.Model
-	scaled    []string
-	lastScale catalog.Scaling
-	err       error
+	added       []catalog.Model
+	declared    string
+	sources     map[string][]catalog.Model
+	scaled      []string
+	lastScale   catalog.Scaling
+	err         error
+	classes     []string
+	owners      []string
+	resources   []string
+	adapterSets [][]catalog.Adapter
 }

 func (f *fakeCatalog) Add(_ context.Context, m catalog.Model, _ string) error {
@@ -35,6 +40,26 @@ func (f *fakeCatalog) Sources(context.Context) (map[string][]catalog.Model, erro
 	return f.sources, nil
 }

+func (f *fakeCatalog) SetAdapters(_ context.Context, name string, as []catalog.Adapter, _ string) (catalog.Model, error) {
+	f.adapterSets = append(f.adapterSets, as)
+	return catalog.Model{Name: name, Adapters: as}, f.err
+}
+
+func (f *fakeCatalog) SetClass(_ context.Context, name, class, _ string) (catalog.Model, error) {
+	f.classes = append(f.classes, name+"="+class)
+	return catalog.Model{Name: name, Class: class}, f.err
+}
+
+func (f *fakeCatalog) SetOwner(_ context.Context, name, owner, _ string) (catalog.Model, error) {
+	f.owners = append(f.owners, name+"="+owner)
+	return catalog.Model{Name: name, Owner: owner}, f.err
+}
+
+func (f *fakeCatalog) Resources(_ context.Context, name, _ string, mem, cores int, _ string) (catalog.Model, error) {
+	f.resources = append(f.resources, fmt.Sprintf("%s=%d/%d", name, mem, cores))
+	return catalog.Model{Name: name}, f.err
+}
+
 func (f *fakeCatalog) Scale(_ context.Context, name string, sc catalog.Scaling, _ string) (catalog.Model, error) {
 	f.scaled = append(f.scaled, name)
 	f.lastScale = sc
@@ -45,6 +45,13 @@ type DashboardData struct {

 	HasPeople bool

+	// NodePrefix is what every node's name starts with, left off in tables
+	// so a row fits on a line; the full name is on hover.
+	NodePrefix string
+	// Columns shown only when some row has something in them: a CPU-only
+	// cluster's nodes table was three columns of dashes wide.
+	AnyGPUs, AnyClasses, AnyFabrics, AnyFallbacks bool
+
 	// Charts are pre-rendered rather than assembled in the template: building
 	// SVG out of template fragments is how a chart ends up unescaped.
 	RequestChart template.HTML
@@ -177,6 +184,17 @@ func Dashboard(src Source, usage UsageSource, version string, refresh time.Durat
 				}
 			}
 			d.Unplaced = unplaced(src, d.Models)
+			ids := make([]string, 0, len(d.Nodes))
+			for _, n := range d.Nodes {
+				ids = append(ids, n.ID)
+				d.AnyGPUs = d.AnyGPUs || n.GPUTotal > 0
+				d.AnyClasses = d.AnyClasses || len(n.Classes) > 0
+				d.AnyFabrics = d.AnyFabrics || len(n.Fabrics) > 0
+			}
+			d.NodePrefix = sharedPrefix(ids)
+			for _, m := range d.Models {
+				d.AnyFallbacks = d.AnyFallbacks || len(m.Fallbacks) > 0
+			}
 			d.Summary = src.Summary()
 			if fs, ok := src.(FabricSource); ok {
 				r := fs.Fabrics()
@@ -4,13 +4,19 @@ import (
 	"context"
 	"errors"
 	"fmt"
+	"html/template"
 	"net/http"
+	"net/url"
+	"slices"
 	"strconv"
 	"strings"
+	"sync"
 	"time"

 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
 )

@@ -29,14 +35,23 @@ const maxReplicas = 64
 // and room to change it.
 type heldDetail struct {
 	Models []heldModel
+	// Scopes are the parts of the tenancy tree a model can belong to; empty
+	// when capacity quotas are not set up, and the owner choice is left off.
+	Scopes []adminapi.ScopeName
 }

 // heldModel is one catalog model running a held version.
 type heldModel struct {
 	modelUse
-	View     adminapi.ModelView
-	Replicas []heldReplica
-	Headroom *adminapi.HeadroomView
+	// Adapters are the model's own; AdapterChoices the held adapter files
+	// of its architecture not attached yet; FindAdapters the explorer's
+	// list of adapters made for it.
+	Adapters       []catalog.Adapter
+	AdapterChoices []adapterChoice
+	FindAdapters   template.URL
+	View           adminapi.ModelView
+	Replicas       []heldReplica
+	Headroom       *adminapi.HeadroomView
 	// MostNow is how many replicas could always run, Current plus the room
 	// found; zero when the room was not measured or had no end in sight.
 	MostNow int
@@ -64,11 +79,14 @@ func openKey(model, version string) string { return model + ":" + version }

 // heldDetailFor reads, for each model using a row, how it runs and how much
 // more room it has.
-func (p *AccountPages) heldDetailFor(row mirrorRow) *heldDetail {
+func (p *AccountPages) heldDetailFor(ctx context.Context, row mirrorRow) *heldDetail {
 	d := &heldDetail{}
+	if p.capacity != nil {
+		d.Scopes, _ = p.capacity.Scopes(ctx)
+	}
 	if p.fleet == nil {
 		for _, u := range row.UsedBy {
-			d.Models = append(d.Models, heldModel{modelUse: u})
+			d.Models = append(d.Models, heldModel{modelUse: u, Adapters: u.Adapters})
 		}
 		return d
 	}
@@ -82,7 +100,7 @@ func (p *AccountPages) heldDetailFor(row mirrorRow) *heldDetail {
 	}
 	hs, _ := p.fleet.(HeadroomSource)
 	for _, u := range row.UsedBy {
-		hm := heldModel{modelUse: u, View: views[u.Name]}
+		hm := heldModel{modelUse: u, View: views[u.Name], Adapters: u.Adapters}
 		for _, rep := range p.fleet.Replicas(u.Name) {
 			// Built field by field: a ReplicaView carries the engine's key,
 			// which is for the admin listener and never for a page.
@@ -98,7 +116,7 @@ func (p *AccountPages) heldDetailFor(row mirrorRow) *heldDetail {
 			if h, ok := hs.Headroom(u.Name, 16); ok {
 				hm.Headroom = &h
 				if !h.Limited {
-					hm.MostNow = h.Current + h.More
+					hm.MostNow = h.Current + h.More + h.ByPreempting
 				}
 			}
 		}
@@ -144,7 +162,7 @@ func (p *AccountPages) setReplicas(ctx context.Context, admin store.User, r *htt
 		if !ok {
 			return "", fmt.Errorf("no model %q", name)
 		}
-		fits := h.Current + h.More
+		fits := h.Current + h.More + h.ByPreempting
 		if lo > fits && !h.Limited {
 			why := "nothing more fits"
 			if len(h.Blocked) > 0 {
@@ -156,6 +174,11 @@ func (p *AccountPages) setReplicas(ctx context.Context, admin store.User, r *htt
 		if hi > fits && !h.Limited {
 			note = fmt.Sprintf("; there is room for %d now, so it grows past that only if room is made", fits)
 		}
+		// Room its class would make by stopping others: said, since it is
+		// somebody else's replica that goes.
+		if lo > h.Current+h.More && len(h.Preempts) > 0 {
+			note += "; to make room, replicas of " + strings.Join(h.Preempts, ", ") + " will be stopped"
+		}
 	}

 	sc := catalog.Scaling{Min: lo, Max: hi, IdleTimeout: current.Scaling.IdleTimeout}
@@ -171,7 +194,7 @@ func (p *AccountPages) setReplicas(ctx context.Context, admin store.User, r *htt
 	case lo == 0:
 		return fmt.Sprintf("%s loads when asked for, up to %d under load%s", name, hi, note), nil
 	case lo == hi:
-		return fmt.Sprintf("%s runs %s", name, plural(lo, "replica", "replicas")), nil
+		return fmt.Sprintf("%s runs %s%s", name, plural(lo, "replica", "replicas"), note), nil
 	}
 	return fmt.Sprintf("%s runs %d, and up to %d under load%s", name, lo, hi, note), nil
 }
@@ -205,3 +228,206 @@ func age(sec int64) string {
 	}
 	return fmt.Sprintf("%dd", int(d.Hours()/24))
 }
+
+// setModel applies the card's settings form: what a replica reserves, the
+// model's class and owner, then how many replicas it runs — in that order, so
+// the room checked for the replicas is the room its new class and reservation
+// would get. Only what changed is applied, and each change is said.
+func (p *AccountPages) setModel(ctx context.Context, admin store.User, r *http.Request) (string, error) {
+	name := r.FormValue("model")
+	current, err := p.currentModel(ctx, name)
+	if err != nil {
+		return "", err
+	}
+	if p.catalog.Declared(name) {
+		return "", fmt.Errorf("%s is set in server.yaml; change it there", name)
+	}
+	var done []string
+	if r.Form.Has("memory") && len(current.Profiles) > 0 {
+		mem, err1 := strconv.Atoi(strings.TrimSpace(r.FormValue("memory")))
+		cores, err2 := strconv.Atoi(strings.TrimSpace(r.FormValue("cores")))
+		if err1 != nil || err2 != nil || mem < 0 || cores < 0 {
+			return "", errors.New("memory and cores are whole numbers")
+		}
+		if cur := current.Profiles[0]; mem != cur.MinMemoryGB || cores != cur.CPUCores {
+			if _, err := p.catalog.Resources(ctx, name, "", mem, cores, admin.Email); err != nil {
+				return "", err
+			}
+			done = append(done, fmt.Sprintf("a replica reserves %d GB and %d cores (running ones keep theirs until restarted)", mem, cores))
+		}
+	}
+	if r.Form.Has("class") {
+		if c := r.FormValue("class"); c != current.Class {
+			m, err := p.catalog.SetClass(ctx, name, c, admin.Email)
+			if err != nil {
+				return "", err
+			}
+			done = append(done, "is "+m.EffectiveClass())
+		}
+	}
+	if r.Form.Has("owner") && p.capacity != nil {
+		if o := r.FormValue("owner"); o != current.Owner {
+			if _, err := p.catalog.SetOwner(ctx, name, o, admin.Email); err != nil {
+				return "", err
+			}
+			if o == "" {
+				done = append(done, "is shared")
+			} else {
+				done = append(done, "counts against "+o)
+			}
+		}
+	}
+	lo, _ := strconv.Atoi(strings.TrimSpace(r.FormValue("min")))
+	hi, _ := strconv.Atoi(strings.TrimSpace(r.FormValue("max")))
+	if r.Form.Has("min") && (lo != current.Scaling.Min || hi != current.Scaling.Max) {
+		msg, err := p.setReplicas(ctx, admin, r)
+		if err != nil {
+			if len(done) > 0 {
+				return "", fmt.Errorf("%s %s, but not the replicas: %w", name, strings.Join(done, ", "), err)
+			}
+			return "", err
+		}
+		done = append(done, strings.TrimPrefix(msg, name+" "))
+	}
+	if len(done) == 0 {
+		return name + ": nothing changed", nil
+	}
+	return name + " " + strings.Join(done, "; "), nil
+}
+
+// heldKinds reads what each held single-file GGUF is — an adapter or a model —
+// once per file, remembered after.
+func (p *AccountPages) heldKinds(ctx context.Context, rows []mirrorRow) {
+	var wg sync.WaitGroup
+	sem := make(chan struct{}, 6)
+	for i := range rows {
+		row := &rows[i]
+		if row.State != mirror.Present || row.Files != 1 || !gguf.IsGGUF(row.File) {
+			continue
+		}
+		if h, ok := p.kinds.Load(row.URL); ok {
+			row.Kind, row.Arch = kindName(h.(gguf.Header)), h.(gguf.Header).Architecture
+			continue
+		}
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			sem <- struct{}{}
+			defer func() { <-sem }()
+			cctx, cancel := context.WithTimeout(ctx, 5*time.Second)
+			defer cancel()
+			if h, err := kindOf(cctx, row.URL); err == nil {
+				p.kinds.Store(row.URL, h)
+				row.Kind, row.Arch = kindName(h), h.Architecture
+			}
+		}()
+	}
+	wg.Wait()
+}
+
+func kindName(h gguf.Header) string {
+	if h.IsAdapter() {
+		return "adapter"
+	}
+	return "model"
+}
+
+// adapterChoice is a held adapter file that could be attached to a model.
+type adapterChoice struct {
+	URL, Label, Suggest string
+}
+
+// addAdapterChoices offers, on each model of a held model's card, the held
+// adapter files of the same architecture, and a way to find more.
+func (d *heldDetail) addAdapterChoices(row mirrorRow, rows []mirrorRow) {
+	for i := range d.Models {
+		hm := &d.Models[i]
+		hm.FindAdapters = findAdaptersHref(row.Source)
+		if row.Kind == "adapter" {
+			continue
+		}
+		for _, r := range rows {
+			if r.Kind != "adapter" || (row.Arch != "" && r.Arch != row.Arch) {
+				continue
+			}
+			attached := false
+			for _, a := range hm.Adapters {
+				attached = attached || a.Source == r.URL
+			}
+			if !attached {
+				hm.AdapterChoices = append(hm.AdapterChoices, adapterChoice{URL: r.URL,
+					Label: r.Model + " " + r.Version + " · " + r.File, Suggest: hm.Name + "-" + r.Model})
+			}
+		}
+	}
+}
+
+// findAdaptersHref is the explorer's list of adapters for a held model: those
+// made for the model its repository was made from, when it came from the
+// registry, and every adapter otherwise.
+func findAdaptersHref(source string) template.URL {
+	const base = "/admin/mirror?tab=explore&view=adapters"
+	u, err := url.Parse(source)
+	if err != nil || u.Host != "huggingface.co" {
+		return base
+	}
+	parts := strings.Split(strings.Trim(u.Path, "/"), "/")
+	if len(parts) < 2 {
+		return base
+	}
+	return template.URL(base + "&from=" + url.QueryEscape(parts[0]+"/"+parts[1]))
+}
+
+// setAdapters attaches or detaches one adapter on a model, from its card.
+// An adapter is checked before it is attached: a file that is not an adapter,
+// or one made for another architecture, is a replica that fails to start.
+func (p *AccountPages) setAdapters(ctx context.Context, admin store.User, r *http.Request) (string, error) {
+	name := r.FormValue("model")
+	current, err := p.currentModel(ctx, name)
+	if err != nil {
+		return "", err
+	}
+	adapters := slices.Clone(current.Adapters)
+	switch r.FormValue("op") {
+	case "remove":
+		gone := r.FormValue("name")
+		adapters = slices.DeleteFunc(adapters, func(a catalog.Adapter) bool { return a.Name == gone })
+		if len(adapters) == len(current.Adapters) {
+			return "", fmt.Errorf("%s has no adapter %q", name, gone)
+		}
+		if _, err := p.catalog.SetAdapters(ctx, name, adapters, admin.Email); err != nil {
+			return "", err
+		}
+		return fmt.Sprintf("%s: adapter %s removed; replicas drop it when restarted (llmgw ctl restart --model %s)",
+			name, gone, name), nil
+	case "add":
+		an := strings.TrimSpace(r.FormValue("name"))
+		src := strings.TrimSpace(r.FormValue("source"))
+		scale := 1.0
+		if v := strings.TrimSpace(r.FormValue("scale")); v != "" {
+			if scale, err = strconv.ParseFloat(v, 64); err != nil || scale <= 0 {
+				return "", errors.New("the scale is a positive number")
+			}
+		}
+		if an == "" || src == "" {
+			return "", errors.New("an adapter needs a name and a file")
+		}
+		h, err := kindOf(ctx, src)
+		if err != nil {
+			return "", fmt.Errorf("the adapter file could not be read: %w", err)
+		}
+		if !h.IsAdapter() {
+			return "", errors.New("that file is a model, not a LoRA adapter")
+		}
+		if base, err := kindOf(ctx, current.Source); err == nil && base.Architecture != h.Architecture {
+			return "", fmt.Errorf("that adapter is for %s models and %s is %s", h.Architecture, name, base.Architecture)
+		}
+		adapters = append(adapters, catalog.Adapter{Name: an, Source: src, Scale: scale})
+		if _, err := p.catalog.SetAdapters(ctx, name, adapters, admin.Email); err != nil {
+			return "", err
+		}
+		return fmt.Sprintf("%s: adapter %s added; ask for it by that name once the replicas are restarted "+
+			"(llmgw ctl restart --model %s)", name, an, name), nil
+	}
+	return "", errors.New("add or remove an adapter")
+}
@@ -1,21 +1,28 @@
 package web

 import (
+	"context"
+	"log/slog"
+	"net/http"
 	"net/url"
 	"strings"
 	"testing"

 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
 )

 // roomyFleet runs one replica of "coder" on n2 and has room for `more`.
 type roomyFleet struct {
 	fakeFleet
-	more    int
-	blocked []string
-	asked   int
+	more     int
+	blocked  []string
+	asked    int
+	preempt  int
+	preempts []string
 }

 func (f *roomyFleet) Nodes() []adminapi.NodeView {
@@ -41,6 +48,7 @@ func (f *roomyFleet) Headroom(model string, limit int) (adminapi.HeadroomView, b
 	} else {
 		h.Limited = true
 	}
+	h.ByPreempting, h.Preempts = f.preempt, f.preempts
 	return h, true
 }

@@ -181,3 +189,147 @@ func TestNodeNamesLoseOnlyWhatEveryNodeShares(t *testing.T) {
 		}
 	}
 }
+
+func settingsPage(t *testing.T, cat *fakeCatalog, withQuotas bool) *http.ServeMux {
+	t.Helper()
+	a := fakeAccounts{user: store.User{ID: "u1", Email: "a@example.com", IsAdmin: true}}
+	p := NewAccountPages(a, nil, nil, "v0", slog.New(slog.DiscardHandler))
+	if withQuotas {
+		p.SetCapacity(&fakeCapacity{})
+	}
+	mux := http.NewServeMux()
+	p.RegisterAdminMirror(mux, MirrorDeps{Mirror: &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 1},
+	}}, Catalog: cat, Fleet: &roomyFleet{more: 3}})
+	return mux
+}
+
+func TestACardSetsClassOwnerAndReservationAndOnlyWhatChanged(t *testing.T) {
+	cat := coderCatalog()
+	cat.sources["http://mirror:8081/coder/v1/c.gguf"][0].Profiles = []catalog.Profile{{Name: "cpu", MinMemoryGB: 12, CPUCores: 4}}
+	mux := settingsPage(t, cat, true)
+
+	card := mirrorGet(t, mux, "/admin/mirror?open=coder:v1").Body.String()
+	for _, want := range []string{`name="class"`, `<option value="" selected>default (standard)</option>`,
+		`name="owner"`, `<option value="team:t1">Acme / ml</option>`, `name="memory" value="12"`, `name="cores" value="4"`} {
+		if !strings.Contains(card, want) {
+			t.Errorf("the card lacks %q", want)
+		}
+	}
+	post := func(v url.Values) string {
+		v.Set("csrf", "tok")
+		v.Set("model", "coder")
+		v.Set("open", "coder:v1")
+		return mirrorPost(t, mux, "/admin/mirror/replicas", v).Body.String()
+	}
+	same := url.Values{"min": {"1"}, "max": {"1"}, "class": {""}, "owner": {""}, "memory": {"12"}, "cores": {"4"}}
+	if body := post(same); !strings.Contains(body, "coder: nothing changed") ||
+		len(cat.classes)+len(cat.owners)+len(cat.resources)+len(cat.scaled) != 0 {
+		t.Errorf("an unchanged form changed something: %v %v %v %v", cat.classes, cat.owners, cat.resources, cat.scaled)
+	}
+	changed := url.Values{"min": {"1"}, "max": {"1"}, "class": {"critical"}, "owner": {"team:t1"}, "memory": {"14"}, "cores": {"4"}}
+	body := post(changed)
+	if strings.Join(cat.resources, ",") != "coder=14/4" || strings.Join(cat.classes, ",") != "coder=critical" ||
+		strings.Join(cat.owners, ",") != "coder=team:t1" || len(cat.scaled) != 0 {
+		t.Errorf("applied %v %v %v %v", cat.resources, cat.classes, cat.owners, cat.scaled)
+	}
+	if !strings.Contains(body, "a replica reserves 14 GB and 4 cores") || !strings.Contains(body, "counts against team:t1") {
+		t.Errorf("the change was not said: %s", body)
+	}
+
+	// Without capacity quotas there is no owner to choose, and none is sent.
+	plain := mirrorGet(t, settingsPage(t, coderCatalog(), false), "/admin/mirror?open=coder:v1").Body.String()
+	if strings.Contains(plain, `name="owner"`) {
+		t.Error("an owner choice with no quotas to count against")
+	}
+}
+
+func TestRoomAHigherClassWouldMakeIsOfferedAndSaid(t *testing.T) {
+	fleet := &roomyFleet{more: 0, preempt: 1, preempts: []string{"chat"}, blocked: []string{"n1: full"}}
+	cat := coderCatalog()
+	mux := mirrorPageDeps(t, MirrorDeps{Mirror: &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 1},
+	}}, Catalog: cat, Fleet: fleet}, true)
+
+	card := mirrorGet(t, mux, "/admin/mirror?open=coder:v1").Body.String()
+	if !strings.Contains(card, "No free room for another replica now.") ||
+		!strings.Contains(card, "1 by stopping replicas its class outranks: chat.") ||
+		!strings.Contains(card, `name="min" value="1" min="0" max="2"`) {
+		t.Errorf("the card should offer the room preemption makes, and say whose")
+	}
+	body := mirrorPost(t, mux, "/admin/mirror/replicas", url.Values{"csrf": {"tok"},
+		"model": {"coder"}, "min": {"2"}, "max": {"2"}, "open": {"coder:v1"}}).Body.String()
+	if len(cat.scaled) != 1 || cat.lastScale.Min != 2 {
+		t.Fatalf("two were refused though preemption makes room: %s", body)
+	}
+	if !strings.Contains(body, "replicas of chat will be stopped") {
+		t.Errorf("stopping somebody else's replica should be said: %s", body)
+	}
+	body = mirrorPost(t, mux, "/admin/mirror/replicas", url.Values{"csrf": {"tok"},
+		"model": {"coder"}, "min": {"3"}, "max": {"3"}, "open": {"coder:v1"}}).Body.String()
+	if len(cat.scaled) != 1 || !strings.Contains(body, "at most 2") {
+		t.Errorf("three is past even what preemption makes: %s", body)
+	}
+}
+
+func TestAnAdapterHeldInTheMirrorIsAttachedFromItsModelsCard(t *testing.T) {
+	prev := kindOf
+	t.Cleanup(func() { kindOf = prev })
+	kindOf = func(_ context.Context, u string) (gguf.Header, error) {
+		switch {
+		case strings.Contains(u, "/sql/"):
+			return gguf.Header{Architecture: "qwen2", Type: "adapter"}, nil
+		case strings.Contains(u, "/llama-lora/"):
+			return gguf.Header{Architecture: "llama", Type: "adapter"}, nil
+		}
+		return gguf.Header{Architecture: "qwen2"}, nil
+	}
+	cat := coderCatalog()
+	coder := &cat.sources["http://mirror:8081/coder/v1/c.gguf"][0]
+	coder.Source = "http://mirror:8081/coder/v1/c.gguf"
+	coder.Profiles = []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 12}}
+	m := &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 1,
+			Source: "https://huggingface.co/bartowski/Qwen2.5-Coder-7B-Instruct-GGUF/resolve/abc/c.gguf"},
+		{Model: "sql", Version: "v1", File: "sql.gguf", State: mirror.Present, BytesDone: 1},
+		{Model: "llama-lora", Version: "v1", File: "l.gguf", State: mirror.Present, BytesDone: 1},
+	}}
+	mux := mirrorPageDeps(t, MirrorDeps{Mirror: m, Catalog: cat, Fleet: &roomyFleet{more: 1}}, true)
+
+	list := mirrorGet(t, mux, "/admin/mirror").Body.String()
+	if strings.Count(list, ">LoRA adapter</span>") != 2 {
+		t.Error("the two adapter files are not marked as adapters in the list")
+	}
+	card := mirrorGet(t, mux, "/admin/mirror?open=coder:v1").Body.String()
+	if !strings.Contains(card, `<option value="http://mirror:8081/sql/v1/sql.gguf" data-suggest="coder-sql">`) {
+		t.Error("the qwen2 adapter is not offered for the qwen2 model")
+	}
+	if strings.Contains(card, "llama-lora") && strings.Contains(card, `value="http://mirror:8081/llama-lora/v1/l.gguf"`) {
+		t.Error("an adapter for another architecture is offered")
+	}
+	if !strings.Contains(card, "view=adapters&amp;from=bartowski%2FQwen2.5-Coder-7B-Instruct-GGUF") {
+		t.Error("finding adapters should start from the model's own repository")
+	}
+
+	post := func(v url.Values) string {
+		v.Set("csrf", "tok")
+		v.Set("model", "coder")
+		v.Set("open", "coder:v1")
+		return mirrorPost(t, mux, "/admin/mirror/adapters", v).Body.String()
+	}
+	body := post(url.Values{"op": {"add"}, "name": {"coder-sql"}, "source": {"http://mirror:8081/sql/v1/sql.gguf"}, "scale": {"0.8"}})
+	if len(cat.adapterSets) != 1 || len(cat.adapterSets[0]) != 1 || cat.adapterSets[0][0].Scale != 0.8 {
+		t.Fatalf("attached %+v: %s", cat.adapterSets, body)
+	}
+	if !strings.Contains(body, "adapter coder-sql added") {
+		t.Errorf("the attach was not said: %s", body)
+	}
+	for name, src := range map[string]string{"another architecture": "http://mirror:8081/llama-lora/v1/l.gguf",
+		"a model": "http://mirror:8081/coder/v1/c.gguf"} {
+		n := len(cat.adapterSets)
+		out := post(url.Values{"op": {"add"}, "name": {"x"}, "source": {src}})
+		if len(cat.adapterSets) != n || !strings.Contains(out, `class="error"`) {
+			t.Errorf("%s was attached", name)
+		}
+	}
+}
@@ -18,6 +18,9 @@ const (
 	viewTrending  = "trending"
 	viewNew       = "new"
 	viewDownloads = "downloads"
+	// viewAdapters lists LoRA adapters, all of them or those made for one
+	// model.
+	viewAdapters = "adapters"
 )

 // newWithin is how recent a repository has to be to count as new.
@@ -31,6 +34,7 @@ const exploreShown = 25

 var exploreViews = []struct{ key, label string }{
 	{viewTrending, "Trending"}, {viewNew, "New this month"}, {viewDownloads, "Most downloaded"},
+	{viewAdapters, "LoRA adapters"},
 }

 var exploreTasks = []struct{ key, label, tag string }{
@@ -48,6 +52,9 @@ var exploreSizes = []struct {
 // open: a listing, its filters, and links that keep them.
 type explorer struct {
 	View, Task, Size string
+	// Of is the model whose adapters are listed, in the adapters view; empty
+	// lists adapters for any model.
+	Of string
 	// Here keeps to what the nodes' llama.cpp can load. On by default: a
 	// trending list is mostly things this cluster cannot run, and finding
 	// that out one repository at a time is what the explorer is for avoiding.
@@ -123,11 +130,12 @@ func trimZero(s string) string { return strings.TrimSuffix(s, ".0") }
 // — the page opened from the menu — it is trending chat models this cluster
 // can run.
 func exploreState(q url.Values) explorer {
-	e := explorer{View: q.Get("view"), Task: q.Get("task"), Size: q.Get("size")}
+	e := explorer{View: q.Get("view"), Task: q.Get("task"), Size: q.Get("size"),
+		Of: strings.Trim(strings.TrimSpace(q.Get("of")), "/")}
 	// "x" marks a submitted form, so an unticked box reads as off rather than
 	// as not yet chosen.
 	e.Here = q.Get("x") != "1" || q.Get("here") == "1"
-	if !oneOf(e.View, viewTrending, viewNew, viewDownloads) {
+	if !oneOf(e.View, viewTrending, viewNew, viewDownloads, viewAdapters) {
 		e.View = viewTrending
 		// A search with no view asks for what a name usually means, which is
 		// the one everybody downloads.
@@ -165,6 +173,10 @@ func (e explorer) href(query, key, value string) template.URL {
 	if key != "" {
 		v.Set(key, value)
 	}
+	// Whose adapters, kept while it is adapters being looked at.
+	if e.Of != "" && v.Get("view") == viewAdapters {
+		v.Set("of", e.Of)
+	}
 	return template.URL("/admin/mirror?" + v.Encode())
 }

@@ -181,6 +193,16 @@ func (e explorer) hfQuery(search string) hf.Query {
 		// Trending, then kept to the recent: sorted by creation date alone,
 		// the newest repositories are mostly empty uploads nobody has used.
 		q.Limit = 100
+	case viewAdapters:
+		// Adapters rarely carry a task or a parameter count, so neither is
+		// asked for; what makes one useful is the model it was made for.
+		q.Sort = hf.SortDownloads
+		if e.Of != "" {
+			q.Filters = []string{"base_model:adapter:" + e.Of}
+		} else {
+			q.Filters = []string{"lora"}
+		}
+		return q
 	}
 	for _, t := range exploreTasks {
 		if t.key == e.Task {
@@ -198,6 +220,15 @@ func (e explorer) hfQuery(search string) hf.Query {
 // exploreList fills in the listing.
 func (p *AccountPages) exploreList(r *http.Request, d *adminMirrorData) {
 	e := exploreState(r.URL.Query())
+	// From a held model's card: its repository, whose adapters are those of
+	// the model it was made from — a GGUF repository is a quantisation, and
+	// adapters are made for the original.
+	if from := strings.TrimSpace(r.URL.Query().Get("from")); e.View == viewAdapters && e.Of == "" && from != "" {
+		e.Of = from
+		if repo, err := p.registry.Repo(r.Context(), from); err == nil && len(repo.BaseModel) > 0 {
+			e.Of = repo.BaseModel[0]
+		}
+	}
 	for _, v := range exploreViews {
 		e.Views = append(e.Views, exploreLink{v.label, e.href(d.Query, "view", v.key), v.key == e.View})
 	}
@@ -225,6 +256,14 @@ func (p *AccountPages) exploreList(r *http.Request, d *adminMirrorData) {
 		if a := m.Architecture(); a != "" && p.llamaVersion != "" {
 			row.Verdict = gguf.LlamaCpp(p.llamaVersion, a)
 		}
+		// Adapters are small and say little about their architecture in a
+		// listing; whether one fits is read from its file when it is opened.
+		if e.View == viewAdapters {
+			if len(e.Rows) < exploreShown {
+				e.Rows = append(e.Rows, row)
+			}
+			continue
+		}
 		// An adapter or a fragment, not a model: tens of megabytes of LoRA
 		// weights listed among the models they need a model to run on.
 		if e.Here && m.Params() > 0 && m.Params() < minParams {
Author	SHA1	Message	Date
Ilker ManapandClaude Opus 5.5	0c7934ba96	v0.10.6 Release / build (push) Successful in 1m11s Details Release / notes (push) Successful in 4s Details CI / test (push) Successful in 3m51s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 08:52:01 +02:00
Ilker ManapandClaude Opus 5.5	e40b9ac202	Load LoRA adapters at scale zero Release / build (push) Successful in 1m17s Details CI / test (push) Successful in 4m0s Details Release / notes (push) Successful in 5s Details --lora with --lora-init-without-apply leaves an adapter's scale at 1, and llama-server applies it at that scale to any request without a "lora" list of its own: every plain request for the model was served with its adapters on. Found live on qwen2.5-coder-7b-instruct. Each adapter is now loaded with --lora-scaled FILE:0. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 07:39:12 +02:00
Ilker ManapandClaude Opus 5.5	323d172538	Attach LoRA adapters from the command line Release / build (push) Successful in 1m14s Details Release / notes (push) Successful in 4s Details CI / test (push) Successful in 3m52s Details llmgw ctl model adapters lists a model's adapters and adds or removes them through POST /api/v1/models/{name}/adapters, which reads each new file first and refuses one that is not an adapter for the model's architecture, as the mirror page's card does. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 07:30:13 +02:00
Ilker ManapandClaude Opus 5.5	6b1fa65f59	Format kind_live_test.go Release / build (push) Successful in 1m12s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 4m3s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 07:20:35 +02:00
Ilker ManapandClaude Opus 5.5	663770226e	Find LoRA adapters on Hugging Face and attach them from the mirror page CI / test (push) Failing after 33s Details Release / build (push) Successful in 1m3s Details Release / notes (push) Successful in 3s Details An adapters view in the explorer, filtered by the model a held model was made from; GGUF files told apart as adapter or model by their first bytes; held adapter files marked; and a model's card attaches and detaches held adapters of its architecture, refusing models and mismatched architectures. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 07:19:08 +02:00
Ilker ManapandClaude Opus 5.5	cf53904cc5	Serve LoRA adapters on llama.cpp models, asked for by name Release / notes (push) Successful in 3s Details Release / build (push) Successful in 1m11s Details CI / test (push) Successful in 3m47s Details A model's adapters are fetched by each node with its weights and loaded unapplied at start; a request naming an adapter reaches the model's replicas with llama.cpp's per-request lora field. Adapter names share the model namespace, are listed, and follow allowlists on their own. The router's config seed and pushed catalog now build the name map in one place. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 23:40:47 +02:00
Ilker ManapandClaude Opus 5.5	94eec557ad	Serve a model from an external OpenAI-compatible provider CI / test (push) Successful in 3m47s Details Release / build (push) Successful in 1m11s Details Release / notes (push) Successful in 3s Details A catalog model with external {url, model, api_key_file} and no profiles is published as an endpoint by the control plane, the key read from its file each time; the router rewrites the request's model to the provider's name and sends its key. It is never scraped nor stale, and nothing is placed for it. Named in fallbacks, it serves when the cluster has no room. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 23:29:01 +02:00
Ilker ManapandClaude Opus 5.5	d5f02c0a4b	Spill a saturated pool's requests to the next pool CI / test (push) Successful in 3m40s Details The pool-priority filter kept the highest-priority pool however full it was. It now takes the first pool, by priority, that is not saturated by admission's own measure, and the preferred one when all are. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 23:23:32 +02:00
Ilker ManapandClaude Opus 5.5	3d3c96e422	Displace the newest lowest-band request when the gateway queue is full Release / notes (push) Successful in 3s Details Release / build (push) Successful in 1m9s Details CI / test (push) Successful in 3m50s Details A total bound across bands (admission.queue_capacity_total, default half of all bands' capacity together); reached, a request is not refused while a lower band has requests queued, and the newest of the lowest band gets a 503 with Retry-After instead. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 23:07:53 +02:00
Ilker ManapandClaude Opus 5.5	623cdb5ba3	Sign releases, and install only signed ones once a key is trusted CI / test (push) Successful in 4m6s Details ed25519 over checksums.txt: llmgw release keygen makes the pair, CI signs with the RELEASE_SIGNING_KEY secret when it is set, and release.Install refuses an unsigned or wrongly signed release when the build embeds a trusted key (internal/release/trusted.pub, empty for now) or LLMGW_RELEASE_KEY names one. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 23:03:48 +02:00
Ilker ManapandClaude Opus 5.5	561667c9be	Name only the replicas that would make room in headroom CI / test (push) Successful in 3m58s Details Candidates tried on the way to finding no room were listed as whose replicas a placement would stop. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 23:01:40 +02:00
Ilker ManapandClaude Opus 5.5	cf0178bf05	Test that a restart borrows a lower class's room and returns it CI / test (push) Successful in 3m41s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 22:47:12 +02:00
Ilker ManapandClaude Opus 5.5	f288f3c423	Set capacity quotas, class, owner and reservation from the admin pages Release / build (push) Successful in 1m9s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m49s Details The quotas page gains a capacity section over the same API methods the CLI uses; a model's card sets its class, owner, memory and cores, applying only what changed. Headroom counts the replicas a model could get by preempting what its class outranks, and the card and ctl model target use it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 22:40:45 +02:00
Ilker ManapandClaude Opus 5.5	5cf0003e74	Release v0.10.5 Release / notes (push) Successful in 3s Details Release / build (push) Successful in 1m12s Details CI / test (push) Successful in 3m53s Details Collects rc.1 to rc.14: the mirror page in tabs with a card per model, one line per record in every admin table, reserved CPU cores and required memory, capacity classes and preemption, tenant capacity quotas, sizing by measured throughput, and node sensor readings. Adds `llmgw upgrade` for the mirror host, which `ctl upgrade` does not reach, and says so when it ends. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 22:26:19 +02:00
Ilker ManapandClaude Opus 5.5	d10fad064e	Mark facility steps 1-5 deferred; node sensors collected CI / test (push) Successful in 3m45s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 20:25:50 +02:00
Ilker ManapandClaude Opus 5.5	cd10e086c8	Read nodes' heat and power: GPUs, CPUs, board inlet, RAPL Release / build (push) Successful in 1m10s Details CI / test (push) Successful in 3m51s Details Release / notes (push) Successful in 3s Details Every agent samples, at most every ten seconds: nvidia-smi's temperature, HBM temperature, power draw, enforced limit and throttle reasons per GPU; hwmon's CPU package and board inlet temperatures; and CPU package power from RAPL's energy counters. Sent with heartbeats, published as metrics and shown on the node page; unmeasured sensors are absent, never zero. Observe-only, step 0 of docs/facility-aware-placement.md, which now describes the agent source and what these readings can and cannot say. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 19:49:08 +02:00
Ilker ManapandClaude Opus 5.5	8724dc027a	Plan facility-aware placement: rack power, temperature and cooling CI / test (push) Successful in 3m41s Details A design document for step 5 of resource allocation, deferred: location labels and rack spreading, power budgets with estimated then measured draw, thermal and cooling conditions with hysteresis, opt-in shedding through preemption, pluggable facility sources, and a rollout that is useful at each step and safe when data is missing. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 19:25:31 +02:00
Ilker ManapandClaude Opus 5.5	e7a8105aa4	Log a preempted replica's stop with the grace it had Release / build (push) Successful in 1m7s Details Release / notes (push) Successful in 2s Details CI / test (push) Successful in 3m47s Details It said "stopping a drained replica without a replacement after=5m0s" when it had been given the 30-second preemption grace. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 18:41:43 +02:00
Ilker ManapandClaude Opus 5.5	d0e9813ce9	Of equally idle victims, try the smallest first Release / build (push) Successful in 1m11s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m55s Details Idle times are unknown after a control plane restart, which left the choice between a 1-core and a 4-core replica to map order. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 18:23:08 +02:00
Ilker ManapandClaude Opus 5.5	0e73f37128	Stop only the replicas that make the room Release / build (push) Successful in 1m4s Details CI / test (push) Successful in 4m32s Details Release / notes (push) Successful in 3s Details Candidates were added in order until the model fitted, and every one added was stopped, including those tried first that freed room on the wrong node. A set that works is now pruned to the replicas it needs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 18:21:58 +02:00
Ilker ManapandClaude Opus 5.5	78e41460f9	Scale and place by what a replica was measured to serve Release / build (push) Successful in 1m8s Details CI / test (push) Successful in 4m24s Details Release / notes (push) Successful in 3s Details Stored ctl bench results reach the reconciler: requests per replica past the measured SLO concurrency scale a model up, a replica is taken back only if the rest stay under 70% of theirs, and among profiles of equal priority the most measured tokens per GPU or core is tried first. ctl model target sizes a model for a throughput against the room there is. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 18:21:08 +02:00
Ilker ManapandClaude Opus 5.5	9dc1b62fdc	Guarantee and limit capacity per part of the tenancy tree Release / build (push) Successful in 1m8s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m51s Details A model has an owner (org, team or project). Quotas guarantee a scope GPUs, cores and memory and may limit it; what a scope holds is summed from its models' replicas and those beneath it. A replica within its owner's guarantee is not taken for another scope; what is held beyond a guarantee is borrowed and taken back for a scope within its own. A limit stops a model before it starts, and headroom stops there too. ctl quota and ctl model owner set them; metrics and a CapacityAtLimit alert report them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 18:15:38 +02:00
Ilker ManapandClaude Opus 5.5	ac24e204a6	Add capacity classes and preempt lower ones to place a model Release / build (push) Successful in 1m7s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m46s Details A model's class (critical, standard, opportunistic; unset follows its scaling) decides what may be stopped to make room for it: a lower class for a higher, and within a class only what another grew into under load, for a model below its minimum. Victims are tried on a copy of the inventory and stopped only if they make the room. Preempted replicas are drained with a short grace and their model is held back until the room is used. ctl model resources and class move under ctl model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 18:02:59 +02:00
Ilker ManapandClaude Opus 5.5	990d82d801	Reserve CPU cores and require memory for CPU models Release / build (push) Successful in 1m5s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m47s Details A profile's cpu_cores is reserved out of its node's CPUs by placement, as memory is, and becomes the engine's thread count. Agents report their CPU count in registration and heartbeats. New CPU models must declare memory; model views show reserved against measured memory with warnings, and ctl model resources sets both for a stored model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 17:43:13 +02:00
Ilker ManapandClaude Opus 5.5	26151ec25a	Format web.go Release / build (push) Successful in 1m4s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m50s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 11:02:32 +02:00
Ilker ManapandClaude Opus 5.5	235bf33dcd	Keep every admin table to one line per record CI / test (push) Failing after 31s Details Release / notes (push) Successful in 4s Details Release / build (push) Successful in 1m4s Details Headers never wrap, free text is clipped with the whole on hover, full-width rows are only notes and forms. The cluster page hides columns empty on every row and shortens node names; People shows two memberships and a count. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 11:02:18 +02:00