Add capacity classes and preempt lower ones to place a model

A model's class (critical, standard, opportunistic; unset follows its scaling) decides what may be stopped to make room for it: a lower class for a higher, and within a class only what another grew into under load, for a model below its minimum. Victims are tried on a copy of the inventory and stopped only if they make the room. Preempted replicas are drained with a short grace and their model is held back until the room is used. ctl model resources and class move under ctl model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Reserve CPU cores and require memory for CPU models
2026-09-30 18:02:59 +02:00 · 2026-09-30 17:43:13 +02:00 · 2026-09-30 11:02:32 +02:00 · 2026-09-30 11:02:18 +02:00 · 2026-09-30 10:50:02 +02:00 · 2026-09-30 10:38:49 +02:00
52 changed files with 2489 additions and 157 deletions
@@ -20,6 +20,110 @@ is only released once its **exit criterion** is met, not when its code lands.
 Until v1.0.0 the minor version is the breaking one: configuration formats and Go
 APIs change between minors without a deprecation period.

+## Unreleased
+
+Changes since v0.10.4, published as release candidates (v0.10.5-rc.N) until
+there are enough for v0.10.5.
+
+### CPU cores are reserved, and memory has to be declared
+
+The first step of resource allocation for larger clusters: what a replica
+holds of a node is reserved, not assumed.
+
+- **Cores.** A profile's `cpu_cores` is reserved out of its node's CPUs, as
+  memory is, and is what its engine computes on (llama.cpp's `--threads`). A
+  node holds as many CPU replicas as it has cores for, and two on one node no
+  longer spin against each other. Agents report their CPU count; a node that
+  does not, and a profile that declares no cores, constrain nothing.
+- **Memory is required of new CPU models.** A model with no GPU is all host
+  memory, and one declaring none looked to placement as if it fitted
+  everywhere, any number of times. Adding one without `min_memory_gb` is now
+  refused. Models stored before this still load, and say what is missing.
+- **Declared against measured.** Each model shows what one replica reserves
+  and the most one of its replicas uses — in `ctl models` (RESERVES) and on the
+  cluster page — with a warning when nothing is declared or when it uses more
+  than a tenth over what it reserves.
+- **Setting them.** `llmgw ctl model resources MODEL --memory GB --cores N`
+  (and `POST /api/v1/models/{name}/resources`) for a model added at runtime.
+  `ctl model serve` and the mirror's serve form fill in cores — half the
+  smallest node the model fits on — beside the memory they already estimate.
+  Running replicas keep their old figures until restarted.
+
+The commands are `llmgw ctl model resources` and `llmgw ctl model class`; in
+rc.7 the first was briefly `llmgw ctl resources`.
+
+### Capacity classes, and preemption
+
+A model has a class — `critical`, `standard` or `opportunistic` — saying what
+its capacity is worth against another's when there is not enough for both.
+Unset, an on-demand model is opportunistic and every other model standard,
+which is how the catalog behaved before. `llmgw ctl model class MODEL CLASS`
+sets it.
+
+When a model cannot be placed, replicas are stopped to make room for it, by
+these rules:
+
+- A lower class gives way to a higher one.
+- Within a class, only replicas above a model's minimum — room it grew into
+  under load — and only for a model still below its own minimum.
+- A critical model's minimum is never taken, nor a replica that has only just
+  loaded, nor one spanning several nodes.
+- Idle on-demand replicas go first, then what lower classes grew into, then
+  the same within the class, then lower classes' minimums; the longest unused
+  first within each.
+- The set is tried on a copy of the inventory first, at most three replicas,
+  and nothing is stopped unless it makes the room.
+
+A replica stopped this way is drained and given 30 seconds to finish what it
+is serving. Its model is held back for up to five minutes, or until the model
+it gave way to has its replicas, so it does not take the room straight back.
+`ctl models` shows each model's class and "gave room to X" while held; the
+cluster page shows the class and how many replicas were lost; the metric is
+`llmgw_model_preempted_total{model,class}`, and every preemption is logged.
+
+### The mirror page in tabs
+
+/admin/mirror was one long page: what the mirror holds, the Hugging Face
+explorer and the fetch form, one under another, with every visit waiting on
+Hugging Face for a list it might not show. It is now three tabs — Held (with
+how many are held, downloading and failed), Explore Hugging Face, and Fetch by
+URL — each with its own address. Only the tab shown does its work, and only
+Held reloads while something downloads. A download started from the explorer
+links to Held, where its progress is; a fetch the mirror refused comes back on
+the form it was sent from.
+
+### A held model opens to where it runs, and how many
+
+Clicking a model under Held opens a card beneath it: each replica with its
+node, state, age and the CPU and memory its engine uses, and how many more
+replicas the cluster has room for now, on which nodes, and why not more. The
+replicas it always runs and the most it grows to under load are changed there.
+Room is found by trying the scheduler itself on a copy of the inventory — the
+same decisions the next passes would make, not an estimate beside them — and a
+number of always-running replicas it cannot place is refused with the reason,
+rather than accepted and left short. A ceiling above the room is taken, and
+said. The same answer is at `GET /api/v1/models/{name}/headroom` on the admin
+API.
+
+Each row also says how its model runs without being opened: replicas ready of
+those it should have, how many are still starting, and the nodes they are on —
+in amber when short, in red when parked or when it does not fit. It is one line
+per model beside its mode, and nodes drop the part of the name every node
+shares (node05 for llmgw-node05, the full name on hover).
+
+### One line per record, in every table
+
+The same rule across the admin pages. Headers do not wrap; a record is one
+line; free text is cut to its column with the whole of it on hover rather than
+given a row of its own; a row across the whole table is only a note or an open
+form. On the cluster page that means a node is one line (its address and logs
+beside its name, not under it), a model's description is a column rather than a
+second row, node names drop the prefix they all share, and the GPUs, classes,
+fabrics and fallbacks columns appear only when some row has something in them —
+on a CPU-only cluster they were three columns of dashes pushing the table past
+the page. On People, a person in many teams shows two and a count, the rest on
+hover.
+
 ## v0.10.4 — 2026-09-30

 Published first as release candidates v0.10.4-rc.1 to rc.9.
@@ -534,6 +534,10 @@ type Register struct {
 	// meant "an agent too old to say" and constrained nothing — so a freshly
 	// built node with no engine installed yet was a candidate for every model.
 	EnginesProbed bool `protobuf:"varint,14,opt,name=engines_probed,json=enginesProbed,proto3" json:"engines_probed,omitempty"`
+	// cpu_count is the CPUs the agent may run on. Placement reserves cores
+	// for CPU replicas out of it, the way it reserves memory out of
+	// memory_total_mb; zero, from an older agent, reserves nothing.
+	CpuCount int32 `protobuf:"varint,18,opt,name=cpu_count,json=cpuCount,proto3" json:"cpu_count,omitempty"`
 	// imex is the service that lets a multi-node NVLink domain share memory.
 	Imex *IMEX `protobuf:"bytes,15,opt,name=imex,proto3" json:"imex,omitempty"`
 	// installable_engines are the engines this agent knows how to install
@@ -679,6 +683,13 @@ func (x *Register) GetEnginesProbed() bool {
 	return false
 }

+func (x *Register) GetCpuCount() int32 {
+	if x != nil {
+		return x.CpuCount
+	}
+	return 0
+}
+
 func (x *Register) GetImex() *IMEX {
 	if x != nil {
 		return x.Imex
@@ -2017,8 +2028,10 @@ type Heartbeat struct {
 	// gpu_fabric is each GPU's NVLink domain membership as it stands now, sent
 	// once a minute: a fabric manager restarting drops every card out of its
 	// domain, and the node has to stop being offered work that spans it.
-	GpuFabric     []*GPUFabricState `protobuf:"bytes,16,rep,name=gpu_fabric,json=gpuFabric,proto3" json:"gpu_fabric,omitempty"`
-	Imex          *IMEX             `protobuf:"bytes,17,opt,name=imex,proto3" json:"imex,omitempty"`
+	GpuFabric []*GPUFabricState `protobuf:"bytes,16,rep,name=gpu_fabric,json=gpuFabric,proto3" json:"gpu_fabric,omitempty"`
+	Imex      *IMEX             `protobuf:"bytes,17,opt,name=imex,proto3" json:"imex,omitempty"`
+	// cpu_count as in Register, for a VM given more CPUs while it runs.
+	CpuCount      int32 `protobuf:"varint,18,opt,name=cpu_count,json=cpuCount,proto3" json:"cpu_count,omitempty"`
 	unknownFields protoimpl.UnknownFields
 	sizeCache     protoimpl.SizeCache
 }
@@ -2172,6 +2185,13 @@ func (x *Heartbeat) GetImex() *IMEX {
 	return nil
 }

+func (x *Heartbeat) GetCpuCount() int32 {
+	if x != nil {
+		return x.CpuCount
+	}
+	return 0
+}
+
 // Provision is the software a node should have: which engines, at which
 // versions. Nothing more.
 //
@@ -2561,7 +2581,10 @@ type StartShard struct {
 	// api_key is what the engine demands of every request, derived by the
 	// control plane from the replica id. The router sends it; nothing else
 	// that can reach the node's ports has it.
-	ApiKey        string `protobuf:"bytes,16,opt,name=api_key,json=apiKey,proto3" json:"api_key,omitempty"`
+	ApiKey string `protobuf:"bytes,16,opt,name=api_key,json=apiKey,proto3" json:"api_key,omitempty"`
+	// cpu_threads is how many CPU threads a shard with no GPU computes on:
+	// the cores placement reserved for it. Zero leaves it to the agent.
+	CpuThreads    int32 `protobuf:"varint,17,opt,name=cpu_threads,json=cpuThreads,proto3" json:"cpu_threads,omitempty"`
 	unknownFields protoimpl.UnknownFields
 	sizeCache     protoimpl.SizeCache
 }
@@ -2708,6 +2731,13 @@ func (x *StartShard) GetApiKey() string {
 	return ""
 }

+func (x *StartShard) GetCpuThreads() int32 {
+	if x != nil {
+		return x.CpuThreads
+	}
+	return 0
+}
+
 type EngineProfile struct {
 	state           protoimpl.MessageState `protogen:"open.v1"`
 	Name            string                 `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
@@ -3655,7 +3685,7 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"fabricTest\x122\n" +
 	"\aupgrade\x18\n" +
 	" \x01(\v2\x16.llmgw.v1.UpgradeAgentH\x00R\aupgradeB\x05\n" +
-	"\x03msg\"\xe3\x06\n" +
+	"\x03msg\"\x80\a\n" +
 	"\bRegister\x12\x17\n" +
 	"\anode_id\x18\x01 \x01(\tR\x06nodeId\x12\x1a\n" +
 	"\bhostname\x18\x02 \x01(\tR\bhostname\x12\x1b\n" +
@@ -3671,7 +3701,8 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"\aweights\x18\v \x03(\v2\x16.llmgw.v1.LocalWeightsR\aweights\x12+\n" +
 	"\astorage\x18\f \x03(\v2\x11.llmgw.v1.StorageR\astorage\x12\"\n" +
 	"\x04host\x18\r \x01(\v2\x0e.llmgw.v1.HostR\x04host\x12%\n" +
-	"\x0eengines_probed\x18\x0e \x01(\bR\renginesProbed\x12\"\n" +
+	"\x0eengines_probed\x18\x0e \x01(\bR\renginesProbed\x12\x1b\n" +
+	"\tcpu_count\x18\x12 \x01(\x05R\bcpuCount\x12\"\n" +
 	"\x04imex\x18\x0f \x01(\v2\x0e.llmgw.v1.IMEXR\x04imex\x12/\n" +
 	"\x13installable_engines\x18\x10 \x03(\tR\x12installableEngines\x12\x1a\n" +
 	"\bfeatures\x18\x11 \x03(\tR\bfeatures\x1a9\n" +
@@ -3811,7 +3842,7 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"\x06status\x18\x03 \x01(\tR\x06status\x12!\n" +
 	"\fcluster_uuid\x18\x04 \x01(\tR\vclusterUuid\x12\x1b\n" +
 	"\tclique_id\x18\x05 \x01(\tR\bcliqueId\x12\x1c\n" +
-	"\tbandwidth\x18\x06 \x01(\tR\tbandwidth\"\xd9\x06\n" +
+	"\tbandwidth\x18\x06 \x01(\tR\tbandwidth\"\xf6\x06\n" +
 	"\tHeartbeat\x12\x17\n" +
 	"\anode_id\x18\x01 \x01(\tR\x06nodeId\x12+\n" +
 	"\x12gpu_memory_used_mb\x18\x02 \x03(\x05R\x0fgpuMemoryUsedMb\x12!\n" +
@@ -3831,7 +3862,8 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"\afabrics\x18\x0f \x03(\v2\x1a.llmgw.v1.FabricAttachmentR\afabrics\x127\n" +
 	"\n" +
 	"gpu_fabric\x18\x10 \x03(\v2\x18.llmgw.v1.GPUFabricStateR\tgpuFabric\x12\"\n" +
-	"\x04imex\x18\x11 \x01(\v2\x0e.llmgw.v1.IMEXR\x04imex\x1aA\n" +
+	"\x04imex\x18\x11 \x01(\v2\x0e.llmgw.v1.IMEXR\x04imex\x12\x1b\n" +
+	"\tcpu_count\x18\x12 \x01(\x05R\bcpuCount\x1aA\n" +
 	"\x13EngineVersionsEntry\x12\x10\n" +
 	"\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
 	"\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\";\n" +
@@ -3872,7 +3904,7 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"\n" +
 	"bytes_done\x18\x02 \x01(\x03R\tbytesDone\x12\x1f\n" +
 	"\vbytes_total\x18\x03 \x01(\x03R\n" +
-	"bytesTotal\"\xf2\x03\n" +
+	"bytesTotal\"\x93\x04\n" +
 	"\n" +
 	"StartShard\x12\x1d\n" +
 	"\n" +
@@ -3896,7 +3928,9 @@ const file_llmgw_v1_agent_proto_rawDesc = "" +
 	"\x06fabric\x18\x0e \x01(\tR\x06fabric\x12\x1d\n" +
 	"\n" +
 	"profile_fp\x18\x0f \x01(\tR\tprofileFp\x12\x17\n" +
-	"\aapi_key\x18\x10 \x01(\tR\x06apiKey\"\xfd\x02\n" +
+	"\aapi_key\x18\x10 \x01(\tR\x06apiKey\x12\x1f\n" +
+	"\vcpu_threads\x18\x11 \x01(\x05R\n" +
+	"cpuThreads\"\xfd\x02\n" +
 	"\rEngineProfile\x12\x12\n" +
 	"\x04name\x18\x01 \x01(\tR\x04name\x12\x16\n" +
 	"\x06engine\x18\x02 \x01(\tR\x06engine\x12\x1b\n" +
@@ -120,6 +120,10 @@ message Register {
  // meant "an agent too old to say" and constrained nothing — so a freshly
  // built node with no engine installed yet was a candidate for every model.
  bool engines_probed = 14;
+  // cpu_count is the CPUs the agent may run on. Placement reserves cores
+  // for CPU replicas out of it, the way it reserves memory out of
+  // memory_total_mb; zero, from an older agent, reserves nothing.
+  int32 cpu_count = 18;
  // imex is the service that lets a multi-node NVLink domain share memory.
  IMEX imex = 15;

@@ -420,6 +424,8 @@ message Heartbeat {
  // domain, and the node has to stop being offered work that spans it.
  repeated GPUFabricState gpu_fabric = 16;
  IMEX imex = 17;
+  // cpu_count as in Register, for a VM given more CPUs while it runs.
+  int32 cpu_count = 18;
 }

 // Provision is the software a node should have: which engines, at which
@@ -530,6 +536,9 @@ message StartShard {
  // control plane from the replica id. The router sends it; nothing else
  // that can reach the node's ports has it.
  string api_key = 16;
+  // cpu_threads is how many CPU threads a shard with no GPU computes on:
+  // the cores placement reserved for it. Zero leaves it to the agent.
+  int32 cpu_threads = 17;
 }

 message EngineProfile {
@@ -84,6 +84,17 @@ groups:
            <model>` says what each node lacks. For an on-demand model, requests
            for it are told this rather than asked to retry.

+      - alert: ModelPreempted
+        expr: increase(llmgw_model_preempted_total[15m]) > 0
+        labels: { severity: info }
+        annotations:
+          summary: "{{ $labels.model }} ({{ $labels.class }}) gave up replicas to a higher class"
+          description: >-
+            Capacity ran short and a model of a higher class took this one's
+            room. Expected now and then; if it keeps firing for the same model,
+            the cluster is too small for what is asked of it at these classes.
+            `llmgw ctl models` shows who it gave way to.
+
      - alert: EngineInstallFailed
        expr: llmgw_node_engine_install{state="failed"} == 1
        for: 15m
@@ -92,6 +92,7 @@ rather than repeated.
 | `source` | `string` | [Model](#model) |
 | `profiles` | `[]Profile` | [Model](#model) |
 | `scaling` | `Scaling` | [Model](#model) |
+| `class` | `string` | [Model](#model) |
 | `fallbacks` | `[]string` | [Model](#model) |
 | `routing_profile` | `string` | [Model](#model) |
 | `slo` | `SLO` | [Model](#model) |
@@ -129,6 +130,7 @@ rather than repeated.
 | `args` | `[]string` | [Profile](#profile) |
 | `env` | `map[string]string` | [Profile](#profile) |
 | `min_memory_gb` | `int` | [Profile](#profile) |
+| `cpu_cores` | `int` | [Profile](#profile) |
 | `min_disk_gb` | `int` | [Profile](#profile) |
 | `node_selector` | `map[string]string` | [Profile](#profile) |
 | `weight` | `float64` | [Profile](#profile) |
@@ -735,6 +737,15 @@ Source is the weights path or a HuggingFace id.

 Takes a [Scaling](#scaling) block.

+### `class`
+
+`string`
+
+Class is how much this model's capacity is worth against another's,
+when there is not enough for both: critical, standard or opportunistic.
+Empty is standard, or opportunistic for a model loaded on demand. See
+EffectiveClass.
+
 ### `fallbacks`

 `[]string`
@@ -966,6 +977,20 @@ this. A node that does not report its memory is not excluded either —
 an older agent must not have its nodes quietly emptied by an upgrade of
 the control plane.

+### `cpu_cores`
+
+`int`
+
+CPUCores is the cores a replica of a profile with no GPU is given:
+reserved out of its node's CPUs by placement, as memory is, and what
+its engine computes on (llama.cpp's --threads).
+
+Reserved because an engine takes every core it is told about, and two
+on one node that each think they have all of them spin against each
+other: on an 8-vCPU node, two 7B replicas fell from 10 tokens a second
+to under one each. Zero reserves nothing and leaves the thread count to
+the node, which is every profile from before this.
+
 ### `min_disk_gb`

 `int`
@@ -16,6 +16,7 @@ import (
 	"net/http"
 	"os"
 	"path/filepath"
+	"runtime"
 	"slices"
 	"sync"
 	"time"
@@ -549,6 +550,7 @@ func (a *Agent) heartbeat() *llmgwv1.AgentMessage {
 	// and a node that grew from 7 GB to 32 GB has to be seen to have grown
 	// without anybody restarting anything.
 	hb.MemoryTotalMb = int64(discovery.MemoryTotalMB())
+	hb.CpuCount = int32(runtime.NumCPU())

 	a.reprobeEngines()
 	a.mu.Lock()
@@ -76,6 +76,10 @@ type ShardSpec struct {
 	// and accounting. Given to the engine in its environment rather than on
 	// its command line, which every user on the node can read.
 	APIKey string `json:"api_key,omitempty"`
+	// Threads is the CPU cores placement reserved for a shard with no GPU,
+	// and so what its engine computes on. Zero leaves it to the engine's own
+	// configuration.
+	Threads int `json:"threads,omitempty"`

 	// Pool and Fabric are control plane concepts the agent cannot derive. They
 	// are carried so the agent can report them back after a restart; Fabric
@@ -133,7 +133,12 @@ func (l *LlamaCpp) Commands(s ShardSpec) ([]Command, error) {
 		args = append(args, "--n-gpu-layers", "999")
 	}
 	if len(s.GPUIndices) == 0 && !hasFlag(p.Args, "--threads", "-t") {
-		n := l.cfg.Threads
+		// What placement reserved first: it is what the node's other
+		// replicas were placed around.
+		n := s.Threads
+		if n <= 0 {
+			n = l.cfg.Threads
+		}
 		if n <= 0 {
 			n = max(numCPU()/2, 1)
 		}
@@ -294,3 +294,15 @@ func TestACPUShardLeavesHalfTheMachineForAnother(t *testing.T) {
 		t.Errorf("--threads %q on one CPU", v)
 	}
 }
+
+func TestTheCoresPlacementReservedAreTheThreads(t *testing.T) {
+	s := llamaSpec(nil)
+	s.Threads = 3
+	cmds, err := (&LlamaCpp{cfg: LlamaCppConfig{Threads: 8}}).Commands(s)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if v, _ := flagValue(cmds[0].Args, "--threads"); v != "3" {
+		t.Errorf("--threads %q; what placement reserved should win over the node's setting", v)
+	}
+}
@@ -253,6 +253,8 @@ func modelStatus(m adminapi.ModelView) string {
 	switch {
 	case m.Parked:
 		return "parked"
+	case m.HeldFor != "":
+		return "gave room to " + m.HeldFor
 	case m.Unplaceable:
 		return "cannot place"
 	case m.Min == 0 && m.Total == 0:
@@ -323,17 +325,27 @@ func newModelsCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
 			}

 			w := tab()
-			fmt.Fprintln(w, "MODEL\tSERVING\tREPLICAS\tSTATUS\tALIASES\tPROFILES\tFALLBACKS")
+			fmt.Fprintln(w, "MODEL\tSERVING\tREPLICAS\tSTATUS\tCLASS\tRESERVES\tALIASES\tPROFILES\tFALLBACKS")
+			var warnings []string
 			for _, m := range models {
 				serving := "no"
 				if m.Ready > 0 {
 					serving = "yes"
 				}
-				fmt.Fprintf(w, "%s\t%s\t%d/%d\t%s\t%s\t%s\t%s\n",
-					m.Name, serving, m.Ready, m.Min, modelStatus(m),
+				fmt.Fprintf(w, "%s\t%s\t%d/%d\t%s\t%s\t%s\t%s\t%s\t%s\n",
+					m.Name, serving, m.Ready, m.Min, modelStatus(m), m.Class, reserves(m),
 					join(m.Aliases), join(m.Profiles), join(m.Fallbacks))
+				for _, x := range m.Warnings {
+					warnings = append(warnings, m.Name+": "+x)
+				}
 			}
-			return w.Flush()
+			if err := w.Flush(); err != nil {
+				return err
+			}
+			for _, x := range warnings {
+				fmt.Println("warning:", x)
+			}
+			return nil
 		},
 	}
 }
@@ -511,3 +523,22 @@ func dash(s string) string {
 	}
 	return s
 }
+
+// reserves is what one replica of a model holds of a node, with what it was
+// measured using beside it: "6G 2c (5.1G used)".
+func reserves(m adminapi.ModelView) string {
+	if m.MemoryGB == 0 && m.CPUCores == 0 {
+		return "-"
+	}
+	out := fmt.Sprintf("%dG", m.MemoryGB)
+	if m.CPUCores > 0 {
+		out += fmt.Sprintf(" %dc", m.CPUCores)
+	}
+	if m.MeasuredMemoryGB > 0 {
+		out += fmt.Sprintf(" (%.1fG used)", m.MeasuredMemoryGB)
+	}
+	if len(m.Warnings) > 0 {
+		out += " !"
+	}
+	return out
+}
@@ -30,6 +30,7 @@ func newModelCmd(client func() *ctlClient) *cobra.Command {
 		Short: "Add or remove models while the cluster is running",
 	}
 	c.AddCommand(newModelAddCmd(client), newModelRemoveCmd(client), newModelPlanCmd(client),
+		newResourcesCmd(client), newClassCmd(client),
 		newModelServeCmd(client))
 	return c
 }
@@ -192,3 +193,73 @@ func newScaleCmd(client func() *ctlClient) *cobra.Command {
 		"how long an on-demand model stays up unasked-for; zero takes the cluster default")
 	return c
 }
+
+func newResourcesCmd(client func() *ctlClient) *cobra.Command {
+	var memory, cores int
+	var profile string
+	c := &cobra.Command{
+		Use:   "resources <model> --memory GB [--cores N]",
+		Short: "Set what a model reserves of a node: memory, and CPU cores",
+		Long: "--memory is the host RAM a replica needs: for a model with no GPU, its\n" +
+			"weights, KV cache and engine together. Placement reserves it on the node,\n" +
+			"so a node holds only as many replicas as it has memory for.\n\n" +
+			"--cores is the CPU cores a replica with no GPU computes on. They are\n" +
+			"reserved the same way, and become the engine's thread count, so two\n" +
+			"replicas on one node stop competing for the same cores. Zero reserves\n" +
+			"none and leaves the thread count to the node.\n\n" +
+			"Running replicas keep what they were started with; `llmgw ctl restart\n" +
+			"--model M` moves them to the new figures.",
+		Args: cobra.ExactArgs(1),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			if !cmd.Flags().Changed("memory") {
+				return fmt.Errorf("--memory is required")
+			}
+			body, err := json.Marshal(adminapi.ResourcesRequest{Profile: profile, MemoryGB: memory, CPUCores: cores})
+			if err != nil {
+				return err
+			}
+			var m catalog.Model
+			if err := client().post(cmd.Context(), "/api/v1/models/"+args[0]+"/resources", body, &m); err != nil {
+				return err
+			}
+			for _, p := range m.Profiles {
+				fmt.Fprintf(cmd.OutOrStdout(), "%s/%s: %d GB memory, %d cores\n", m.Name, p.Name, p.MinMemoryGB, p.CPUCores)
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "Running replicas keep their old figures until restarted: llmgw ctl restart --model %s\n", m.Name)
+			return nil
+		},
+	}
+	c.Flags().IntVar(&memory, "memory", 0, "host RAM a replica needs, in GB")
+	c.Flags().IntVar(&cores, "cores", 0, "CPU cores a replica with no GPU computes on; 0 reserves none")
+	c.Flags().StringVar(&profile, "profile", "", "one profile; every profile when left out")
+	return c
+}
+
+func newClassCmd(client func() *ctlClient) *cobra.Command {
+	return &cobra.Command{
+		Use:   "class <model> critical|standard|opportunistic|default",
+		Short: "Set how much a model's capacity is worth against another's",
+		Long: "When a model cannot be placed, replicas of a lower class are stopped to\n" +
+			"make room for it:\n\n" +
+			"  critical       never gives up the replicas of its minimum\n" +
+			"  standard       gives way to critical\n" +
+			"  opportunistic  gives way to both; the default for an on-demand model\n\n" +
+			"Within a class, a model below its minimum may take what another grew into\n" +
+			"under load, but not its minimum. \"default\" goes back to what the model's\n" +
+			"scaling implies: opportunistic when on demand, standard otherwise.",
+		Args: cobra.ExactArgs(2),
+		RunE: func(cmd *cobra.Command, args []string) error {
+			class := args[1]
+			if class == "default" {
+				class = ""
+			}
+			body, _ := json.Marshal(map[string]string{"class": class})
+			var res map[string]string
+			if err := client().post(cmd.Context(), "/api/v1/models/"+args[0]+"/class", body, &res); err != nil {
+				return err
+			}
+			fmt.Fprintf(cmd.OutOrStdout(), "%s is %s\n", res["model"], res["class"])
+			return nil
+		},
+	}
+}
@@ -75,6 +75,7 @@ func newModelServeCmd(client func() *ctlClient) *cobra.Command {
 	f.IntVar(&req.Replicas, "replicas", 1, "how many replicas to run")
 	f.BoolVar(&onDemand, "on-demand", false, "run none until first asked for")
 	f.IntVar(&req.CtxPool, "ctx-pool", 16384, "llama.cpp's --ctx-size, shared by its parallel slots")
+	f.IntVar(&req.CPUCores, "cores", 0, "CPU cores a replica computes on; 0 takes half the smallest node it fits")
 	f.IntVar(&req.ContextLength, "context-length", 4096, "what one request may use")
 	return c
 }
@@ -4,6 +4,8 @@ import (
 	"net/http/httptest"
 	"strings"
 	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
 )

 func TestAClaimedActorIsOneShortName(t *testing.T) {
@@ -22,3 +24,22 @@ func TestAClaimedActorIsOneShortName(t *testing.T) {
 		}
 	}
 }
+
+func TestAReservationThatPlacementCannotTrustIsSaid(t *testing.T) {
+	cpu := func(mem int) catalog.Model {
+		return catalog.Model{Name: "m", Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: mem}}}
+	}
+	if w := reservationWarnings(cpu(0), 0); len(w) != 1 || !strings.Contains(w[0], "declares no memory") {
+		t.Errorf("no memory declared: %v", w)
+	}
+	if w := reservationWarnings(cpu(6), 8.2); len(w) != 1 || !strings.Contains(w[0], "uses 8.2 GB and reserves 6") {
+		t.Errorf("under-declared: %v", w)
+	}
+	if w := reservationWarnings(cpu(6), 6.4); len(w) != 0 {
+		t.Errorf("within a tenth is mapped weights, not a wrong figure: %v", w)
+	}
+	gpu := catalog.Model{Name: "g", Profiles: []catalog.Profile{{Name: "tp1", Engine: "vllm", GPUClass: "h100-80g"}}}
+	if w := reservationWarnings(gpu, 3); len(w) != 0 {
+		t.Errorf("a GPU model needs no host memory figure: %v", w)
+	}
+}
@@ -15,6 +15,7 @@ import (
 	"io"
 	"net/http"
 	"slices"
+	"strconv"
 	"strings"
 	"time"
 	"unicode"
@@ -56,6 +57,26 @@ func (a *API) Register(mux *http.ServeMux) {
 	mux.HandleFunc("POST /api/v1/models", a.addModel)
 	mux.HandleFunc("DELETE /api/v1/models/{name}", a.removeModel)
 	mux.HandleFunc("POST /api/v1/models/{name}/scale", a.scaleModel)
+	mux.HandleFunc("POST /api/v1/models/{name}/resources", a.setResources)
+	mux.HandleFunc("POST /api/v1/models/{name}/class", func(w http.ResponseWriter, r *http.Request) {
+		if a.models == nil {
+			writeError(w, http.StatusNotImplemented, "this control plane has no store")
+			return
+		}
+		var req struct {
+			Class string `json:"class"`
+		}
+		if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
+			writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+			return
+		}
+		m, err := a.models.SetClass(r.Context(), r.PathValue("name"), req.Class, actorOf(r))
+		if err != nil {
+			writeError(w, http.StatusBadRequest, err.Error())
+			return
+		}
+		writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "class": m.EffectiveClass()})
+	})
 	mux.HandleFunc("PUT /api/v1/models/{name}/capacity", a.putCapacity)
 	mux.HandleFunc("GET /api/v1/capacity", a.listCapacity)
 	mux.HandleFunc("GET /api/v1/usage", a.usageByScope)
@@ -91,6 +112,15 @@ func (a *API) Register(mux *http.ServeMux) {
 	mux.HandleFunc("GET /api/v1/jobs/{id}", a.job)
 	mux.HandleFunc("GET /api/v1/requests/{id}", a.request)
 	mux.HandleFunc("POST /api/v1/models/{name}/retry", a.retryModel)
+	mux.HandleFunc("GET /api/v1/models/{name}/headroom", func(w http.ResponseWriter, r *http.Request) {
+		limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
+		v, ok := a.Headroom(r.PathValue("name"), limit)
+		if !ok {
+			http.Error(w, "no such model", http.StatusNotFound)
+			return
+		}
+		writeJSON(w, http.StatusOK, v)
+	})
 }

 // --- responses ---
@@ -109,6 +139,8 @@ type NodeView struct {
 	// what decides whether a CPU profile can be placed here, so it belongs
 	// where someone looks when a model will not schedule.
 	MemoryGB int `json:"memory_gb,omitempty"`
+	// CPUCount is the machine's CPUs, 0 when the agent did not report them.
+	CPUCount int `json:"cpu_count,omitempty"`
 	// Engines are the inference engines installed here, as the agent found
 	// them. Absent means the agent is older than the probe and said nothing,
 	// which is not the same as having none — placement treats it that way too,
@@ -260,6 +292,20 @@ type ModelView struct {
 	Unplaceable bool     `json:"unplaceable,omitempty"`
 	Fallbacks   []string `json:"fallbacks,omitempty"`
 	Profiles    []string `json:"profiles"`
+	// MemoryGB and CPUCores are what one replica reserves, from its first
+	// profile; MeasuredMemoryGB is the most one of its replicas uses now, 0
+	// when none is measured.
+	MemoryGB         int     `json:"memory_gb,omitempty"`
+	CPUCores         int     `json:"cpu_cores,omitempty"`
+	MeasuredMemoryGB float64 `json:"measured_memory_gb,omitempty"`
+	// Warnings are what makes its reservation wrong: nothing declared where
+	// placement needs a figure, or less declared than it uses.
+	Warnings []string `json:"warnings,omitempty"`
+	// Class is its capacity class; Preempted the replicas it lost to a higher
+	// class; HeldFor the model it is giving room to now, if any.
+	Class     string `json:"class"`
+	Preempted int    `json:"preempted,omitempty"`
+	HeldFor   string `json:"held_for,omitempty"`
 }

 // ReplicaView is a replica as `ctl replicas` shows it.
@@ -336,6 +382,22 @@ type ExplainView struct {
 	Rejections []RejectionView `json:"rejections,omitempty"`
 }

+// HeadroomView is how many more replicas of a model fit now, and where.
+type HeadroomView struct {
+	Model string `json:"model"`
+	// Current is the replicas holding capacity: starting or ready.
+	Current int `json:"current"`
+	// More is how many more placement would find room for now.
+	More int `json:"more"`
+	// Nodes is where each of those would go.
+	Nodes [][]string `json:"nodes,omitempty"`
+	// Limited is set when More stopped at the limit asked rather than at the
+	// cluster's edge: there may be room for more.
+	Limited bool `json:"limited,omitempty"`
+	// Blocked says why one more does not fit, a line per node considered.
+	Blocked []string `json:"blocked,omitempty"`
+}
+
 // RejectionView is one rejected candidate.
 type RejectionView struct {
 	Profile string `json:"profile"`
@@ -370,8 +432,8 @@ func (a *API) Nodes() []NodeView {
 			ID: n.ID, Hostname: n.Hostname, MgmtAddr: n.MgmtAddr,
 			State: n.State.String(), Labels: n.Labels,
 			GPUTotal: len(n.GPUs), GPUFree: len(inv.FreeGPUs(n.ID)),
-			MemoryGB: n.MemoryTotalMB / 1024,
-			Fabrics:  n.FabricNames(), AgentVersion: n.AgentVersion, Features: n.Features,
+			MemoryGB: n.MemoryTotalMB / 1024, CPUCount: n.CPUCount,
+			Fabrics: n.FabricNames(), AgentVersion: n.AgentVersion, Features: n.Features,
 			Replicas: replicasOn[n.ID],
 		}
 		if free, total := n.LocalDisk(); total > 0 {
@@ -471,6 +533,15 @@ func (a *API) Models() []ModelView {
 	for _, h := range a.ctl.Reconciler().ModelHealth() {
 		health[h.Model] = h
 	}
+	// The most one replica of each model uses, summed over its shards.
+	measured := map[string]int64{}
+	for _, rep := range a.Replicas("") {
+		var b int64
+		for _, sh := range rep.Shards {
+			b += sh.MemoryBytes
+		}
+		measured[rep.Model] = max(measured[rep.Model], b)
+	}
 	out := make([]ModelView, 0)
 	for _, m := range a.ctl.Catalog().Models() {
 		h := health[m.Name]
@@ -481,10 +552,16 @@ func (a *API) Models() []ModelView {
 			Fallbacks: m.Fallbacks,
 			Desired:   h.Desired, Failures: h.Failures,
 			Parked: h.Parked, Reason: h.Reason, Unplaceable: h.Unplaceable,
+			Class: m.EffectiveClass(), Preempted: h.Preempted, HeldFor: h.HeldFor,
 		}
 		for _, p := range m.Profiles {
 			v.Profiles = append(v.Profiles, p.Name)
 		}
+		if len(m.Profiles) > 0 {
+			v.MemoryGB, v.CPUCores = m.Profiles[0].MinMemoryGB, m.Profiles[0].CPUCores
+		}
+		v.MeasuredMemoryGB = float64(measured[m.Name]) / (1 << 30)
+		v.Warnings = reservationWarnings(m, v.MeasuredMemoryGB)
 		out = append(out, v)
 	}
 	return out
@@ -555,6 +632,16 @@ type Models interface {
 	Add(ctx context.Context, m catalog.Model, by string) error
 	Remove(ctx context.Context, name string) error
 	Scale(ctx context.Context, name string, s catalog.Scaling, by string) (catalog.Model, error)
+	Resources(ctx context.Context, name, profile string, memoryGB, cpuCores int, by string) (catalog.Model, error)
+	SetClass(ctx context.Context, name, class, by string) (catalog.Model, error)
+}
+
+// ResourcesRequest sets what a model's profiles reserve of a node.
+type ResourcesRequest struct {
+	// Profile is one profile's name; empty is every profile.
+	Profile  string `json:"profile,omitempty"`
+	MemoryGB int    `json:"memory_gb"`
+	CPUCores int    `json:"cpu_cores"`
 }

 // Usage reports what each group of the tenancy tree spent. The store
@@ -800,6 +887,26 @@ func (a *API) scaleModel(w http.ResponseWriter, r *http.Request) {
 	})
 }

+// setResources sets a model's reserved memory and cores. Placed replicas keep
+// what they were started with until they are restarted.
+func (a *API) setResources(w http.ResponseWriter, r *http.Request) {
+	if a.models == nil {
+		writeError(w, http.StatusNotImplemented, "this control plane has no store")
+		return
+	}
+	var req ResourcesRequest
+	if err := json.NewDecoder(io.LimitReader(r.Body, 1<<16)).Decode(&req); err != nil {
+		writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
+		return
+	}
+	m, err := a.models.Resources(r.Context(), r.PathValue("name"), req.Profile, req.MemoryGB, req.CPUCores, actorOf(r))
+	if err != nil {
+		writeError(w, http.StatusBadRequest, err.Error())
+		return
+	}
+	writeJSON(w, http.StatusOK, m)
+}
+
 func (a *API) removeModel(w http.ResponseWriter, r *http.Request) {
 	if a.models == nil {
 		writeError(w, http.StatusNotImplemented, "this control plane has no store")
@@ -836,6 +943,20 @@ func (a *API) explainPlacement(w http.ResponseWriter, r *http.Request) {
 // It does not check that the model exists; the handler above does that so it
 // can answer 404, while the dashboard only ever asks about models it just read
 // out of the catalog.
+// Headroom tries placing up to limit more replicas of a model, on a copy of
+// the inventory, and says how many fit. A limit of zero or less takes 16.
+func (a *API) Headroom(model string, limit int) (HeadroomView, bool) {
+	if limit <= 0 || limit > 64 {
+		limit = 16
+	}
+	h, ok := a.ctl.Reconciler().Headroom(model, limit)
+	if !ok {
+		return HeadroomView{}, false
+	}
+	return HeadroomView{Model: model, Current: h.Current, More: h.More, Nodes: h.Nodes,
+		Limited: h.Limited, Blocked: h.Blocked}, true
+}
+
 func (a *API) Explain(model string) ExplainView {
 	v := ExplainView{Model: model}
 	for _, rep := range a.ctl.Reconciler().Replicas() {
@@ -1267,3 +1388,40 @@ func actorOf(r *http.Request) string {
 	}
 	return b.String()
 }
+
+// NodeCPUs is the CPU counts of ready nodes with at least this much memory:
+// where a replica needing it could go.
+func (a *API) NodeCPUs(minMemoryGB int) []int {
+	var out []int
+	for _, n := range a.ctl.Inventory().Nodes() {
+		if n.State.String() != "ready" || n.CPUCount == 0 {
+			continue
+		}
+		if n.MemoryTotalMB > 0 && n.MemoryTotalMB/1024 < minMemoryGB {
+			continue
+		}
+		out = append(out, n.CPUCount)
+	}
+	return out
+}
+
+// reservationWarnings says what is wrong with what a model reserves.
+//
+// Resident memory includes the weights mapped from disk, which the kernel can
+// take back, so it overstates what a model needs by the part of the weights
+// not in use. A tenth over is let pass for that; more is a figure too low for
+// placement to be trusted with.
+func reservationWarnings(m catalog.Model, measuredGB float64) []string {
+	var out []string
+	for _, p := range m.Profiles {
+		if p.CPUOnly() && p.MinMemoryGB <= 0 {
+			out = append(out, fmt.Sprintf("profile %s declares no memory: placement cannot tell when a node is full "+
+				"(llmgw ctl model resources %s --memory GB)", p.Name, m.Name))
+		}
+	}
+	if len(m.Profiles) == 1 && m.Profiles[0].MinMemoryGB > 0 && measuredGB > float64(m.Profiles[0].MinMemoryGB)*1.1 {
+		out = append(out, fmt.Sprintf("uses %.1f GB and reserves %d: raise it, or nodes will be filled past their memory",
+			measuredGB, m.Profiles[0].MinMemoryGB))
+	}
+	return out
+}
@@ -258,6 +258,17 @@ type fakeModels struct {
 	err     error
 }

+func (f *fakeModels) SetClass(_ context.Context, name, class, _ string) (catalog.Model, error) {
+	return catalog.Model{Name: name, Class: class}, f.err
+}
+
+func (f *fakeModels) Resources(_ context.Context, name, profile string, mem, cores int, _ string) (catalog.Model, error) {
+	if f.err != nil {
+		return catalog.Model{}, f.err
+	}
+	return catalog.Model{Name: name, Profiles: []catalog.Profile{{Name: "p", MinMemoryGB: mem, CPUCores: cores}}}, nil
+}
+
 func (f *fakeModels) Scale(_ context.Context, name string, s catalog.Scaling, by string) (catalog.Model, error) {
 	if f.err != nil {
 		return catalog.Model{}, f.err
@@ -4,6 +4,7 @@ package agentsvc
 import (
 	"fmt"
 	"maps"
+	"runtime"
 	"time"

 	llmgwv1 "gitea.mynodes.xyz/ilker/llmgw/api/gen/llmgw/v1"
@@ -27,6 +28,7 @@ func NodeFromRegister(r *llmgwv1.Register) inventory.Node {
 		EngineVersions: r.GetEngineVersions(),
 		EnginesProbed:  r.GetEnginesProbed(),
 		MemoryTotalMB:  int(r.GetMemoryTotalMb()),
+		CPUCount:       int(r.GetCpuCount()),
 	}
 	if h := r.GetHost(); h != nil {
 		n.Host = discovery.Host{
@@ -90,6 +92,7 @@ func RegisterFromInventory(nodeID, hostname, mgmtAddr, agentVersion string,
 		NodeId: nodeID, Hostname: hostname, MgmtAddr: mgmtAddr,
 		Labels: labels, AgentVersion: agentVersion,
 		MemoryTotalMb: int64(inv.MemoryTotalMB),
+		CpuCount:      int32(runtime.NumCPU()),
 	}
 	for _, w := range inv.Weights {
 		r.Weights = append(r.Weights, &llmgwv1.LocalWeights{
@@ -192,6 +195,7 @@ func StartShardFromCommand(cmd reconciler.ShardCommand) *llmgwv1.StartShard {
 		Fabric:           cmd.Fabric,
 		ProfileFp:        cmd.ProfileFP,
 		ApiKey:           cmd.Spec.APIKey,
+		CpuThreads:       int32(cmd.Spec.Threads),
 	}
 }

@@ -213,6 +217,7 @@ func ShardSpecFromStart(m *llmgwv1.StartShard) (engines.ShardSpec, time.Duration
 		Fabric:        m.GetFabric(),
 		ProfileFP:     m.GetProfileFp(),
 		APIKey:        m.GetApiKey(),
+		Threads:       int(m.GetCpuThreads()),
 		Profile: engines.Profile{
 			Name: p.GetName(), Engine: p.GetEngine(), GPUClass: p.GetGpuClass(),
 			TP: int(p.GetTp()), PP: int(p.GetPp()),
@@ -345,8 +345,9 @@ func (s *Service) applyHeartbeat(nodeID string, hb *llmgwv1.Heartbeat) {
 	problems := hb.GetNewProblems()
 	storage := hb.GetStorage()
 	memMB := int(hb.GetMemoryTotalMb())
+	cpus := int(hb.GetCpuCount())
 	if len(used) == 0 && len(problems) == 0 && len(storage) == 0 &&
-		!hb.GetWeightsChanged() && memMB == 0 && !hb.GetEnginesChanged() &&
+		!hb.GetWeightsChanged() && memMB == 0 && cpus == 0 && !hb.GetEnginesChanged() &&
 		!hb.GetProvisionChanged() && hb.GetLoad() == nil && len(hb.GetIbPorts()) == 0 &&
 		!hb.GetFabricsChanged() && len(hb.GetGpuFabric()) == 0 && hb.GetImex() == nil {
 		return
@@ -406,6 +407,12 @@ func (s *Service) applyHeartbeat(nodeID string, hb *llmgwv1.Heartbeat) {
 			"from_mb", n.MemoryTotalMB, "to_mb", memMB)
 		n.MemoryTotalMB = memMB
 	}
+	if cpus > 0 && cpus != n.CPUCount {
+		if n.CPUCount > 0 {
+			s.log.Info("node CPUs changed", "node", nodeID, "from", n.CPUCount, "to", cpus)
+		}
+		n.CPUCount = cpus
+	}
 	if hb.GetEnginesChanged() {
 		vers := maps.Clone(hb.GetEngineVersions())
 		if !maps.Equal(vers, n.EngineVersions) {
@@ -74,6 +74,17 @@ type Profile struct {
 	// the control plane.
 	MinMemoryGB int `json:"min_memory_gb,omitempty" koanf:"min_memory_gb"`

+	// CPUCores is the cores a replica of a profile with no GPU is given:
+	// reserved out of its node's CPUs by placement, as memory is, and what
+	// its engine computes on (llama.cpp's --threads).
+	//
+	// Reserved because an engine takes every core it is told about, and two
+	// on one node that each think they have all of them spin against each
+	// other: on an 8-vCPU node, two 7B replicas fell from 10 tokens a second
+	// to under one each. Zero reserves nothing and leaves the thread count to
+	// the node, which is every profile from before this.
+	CPUCores int `json:"cpu_cores,omitempty" koanf:"cpu_cores"`
+
 	// MinDiskGB is the free space a node needs before it can hold these
 	// weights.
 	//
@@ -138,10 +149,15 @@ type Model struct {
 	// choice by guesswork.
 	Description string `json:"description,omitempty" koanf:"description"`
 	// Source is the weights path or a HuggingFace id.
-	Source    string    `json:"source,omitempty" koanf:"source"`
-	Profiles  []Profile `json:"profiles,omitempty" koanf:"profiles"`
-	Scaling   Scaling   `json:"scaling,omitempty" koanf:"scaling"`
-	Fallbacks []string  `json:"fallbacks,omitempty" koanf:"fallbacks"`
+	Source   string    `json:"source,omitempty" koanf:"source"`
+	Profiles []Profile `json:"profiles,omitempty" koanf:"profiles"`
+	Scaling  Scaling   `json:"scaling,omitempty" koanf:"scaling"`
+	// Class is how much this model's capacity is worth against another's,
+	// when there is not enough for both: critical, standard or opportunistic.
+	// Empty is standard, or opportunistic for a model loaded on demand. See
+	// EffectiveClass.
+	Class     string   `json:"class,omitempty" koanf:"class"`
+	Fallbacks []string `json:"fallbacks,omitempty" koanf:"fallbacks"`
 	// RoutingProfile selects how the router scores endpoints for this model.
 	RoutingProfile string `json:"routing_profile,omitempty" koanf:"routing_profile"`
 	SLO            SLO    `json:"slo,omitempty" koanf:"slo"`
@@ -288,6 +304,9 @@ func (m Model) validate() error {
 	if len(m.Profiles) == 0 {
 		return fmt.Errorf("model %q: at least one profile is required", m.Name)
 	}
+	if !validClass(m.Class) {
+		return fmt.Errorf("model %q: class %q is not one of critical, standard, opportunistic", m.Name, m.Class)
+	}
 	if m.Scaling.Max < m.Scaling.Min {
 		return fmt.Errorf("model %q: scaling max %d is below min %d", m.Name, m.Scaling.Max, m.Scaling.Min)
 	}
@@ -380,6 +399,7 @@ func (m Model) PlacementProfiles() []placement.Profile {
 			NodeSelector: p.NodeSelector,
 			MinMemoryGB:  p.MinMemoryGB,
 			MinDiskGB:    p.MinDiskGB,
+			CPUCores:     p.CPUCores,
 			// The model's source, so placement can prefer nodes that already
 			// hold the weights rather than one that must first move them.
 			Source: m.Source,
@@ -475,3 +495,56 @@ func (p Profile) Fingerprint() string {
 	sum := sha256.Sum256(b)
 	return hex.EncodeToString(sum[:8])
 }
+
+// SuggestCores is the cores to give a replica with no GPU on nodes of these
+// CPU counts: half of the smallest, so two replicas share any of them without
+// competing. Zero when no node says, which reserves nothing.
+func SuggestCores(cpuCounts []int) int {
+	smallest := 0
+	for _, c := range cpuCounts {
+		if c > 0 && (smallest == 0 || c < smallest) {
+			smallest = c
+		}
+	}
+	if smallest == 0 {
+		return 0
+	}
+	return max(smallest/2, 1)
+}
+
+// The capacity classes, lowest first.
+const (
+	ClassOpportunistic = "opportunistic"
+	ClassStandard      = "standard"
+	ClassCritical      = "critical"
+)
+
+func validClass(c string) bool {
+	return c == "" || c == ClassOpportunistic || c == ClassStandard || c == ClassCritical
+}
+
+// EffectiveClass is the class a model is treated as: what it says, or, when
+// it says nothing, opportunistic for one loaded on demand — it holds capacity
+// only while asked for, and has always been the first to give it up — and
+// standard for everything else.
+func (m Model) EffectiveClass() string {
+	if m.Class != "" {
+		return m.Class
+	}
+	if m.Scaling.OnDemand() {
+		return ClassOpportunistic
+	}
+	return ClassStandard
+}
+
+// ClassRank orders classes: a higher rank may take capacity from a lower one.
+func ClassRank(c string) int {
+	switch c {
+	case ClassCritical:
+		return 3
+	case ClassStandard, "":
+		return 2
+	default:
+		return 1
+	}
+}
@@ -55,6 +55,7 @@ type Collector struct {
 	modelFailures    *prometheus.Desc
 	modelParked      *prometheus.Desc
 	modelUnplaceable *prometheus.Desc
+	modelPreempted   *prometheus.Desc
 	engineInstall    *prometheus.Desc
 	leader           *prometheus.Desc

@@ -168,6 +169,9 @@ func New(src Source) *Collector {
 		modelUnplaceable: prometheus.NewDesc("llmgw_model_unplaceable",
 			"1 when a model is short of replicas and the last attempt found no node to place one on.",
 			[]string{"model"}, nil),
+		modelPreempted: prometheus.NewDesc("llmgw_model_preempted_total",
+			"Replicas of a model stopped to make room for a model of a higher capacity class.",
+			[]string{"model", "class"}, nil),
 		engineInstall: prometheus.NewDesc("llmgw_node_engine_install",
 			"Where installing an engine stands on a node: 1 for the current state.",
 			[]string{"node", "engine", "version", "state"}, nil),
@@ -236,7 +240,7 @@ func (c *Collector) Describe(ch chan<- *prometheus.Desc) {
 		c.gpuMemoryTotal, c.gpuMemoryUsed, c.gpuUtil, c.gpuTemp, c.gpuHealthy,
 		c.gpuNVLink, c.gpuECC,
 		c.replicaState, c.replicaReady, c.replicaIdle,
-		c.modelDesired, c.modelFailures, c.modelParked, c.modelUnplaceable,
+		c.modelDesired, c.modelFailures, c.modelParked, c.modelUnplaceable, c.modelPreempted,
 		c.engineInstall, c.leader,
 	} {
 		ch <- d
@@ -455,6 +459,7 @@ func (c *Collector) Collect(ch chan<- prometheus.Metric) {
 		ch <- prometheus.MustNewConstMetric(c.modelFailures, prometheus.GaugeValue, float64(h.Failures), h.Model)
 		ch <- prometheus.MustNewConstMetric(c.modelParked, prometheus.GaugeValue, b2f(h.Parked), h.Model)
 		ch <- prometheus.MustNewConstMetric(c.modelUnplaceable, prometheus.GaugeValue, b2f(h.Unplaceable), h.Model)
+		ch <- prometheus.MustNewConstMetric(c.modelPreempted, prometheus.CounterValue, float64(h.Preempted), h.Model, h.Class)
 	}
 }

@@ -79,6 +79,9 @@ type Node struct {
 	// it, which is treated as no constraint — the same as every node before
 	// this field existed.
 	MemoryTotalMB int `json:"memory_total_mb,omitempty"`
+	// CPUCount is the CPUs the agent may run on. Zero is an agent too old to
+	// say, which, like memory, constrains nothing.
+	CPUCount int `json:"cpu_count,omitempty"`

 	// Weights are the model files this node can see, however they got there.
 	// Empty means the node was not asked to look, which is not the same as
@@ -373,6 +376,9 @@ type Allocation struct {
 	// Zero for a profile that declared nothing, which constrains nothing and
 	// is counted as nothing.
 	MemoryGB int
+	// CPUCores is the cores this shard was given to compute on, for a shard
+	// with no GPU; zero for one that shares whatever is free.
+	CPUCores int
 }

 // Store holds the cluster inventory. It is safe for concurrent use.
@@ -392,6 +398,23 @@ func NewStore() *Store {
 	return &Store{nodes: map[string]*Node{}, allocs: map[string][]Allocation{}}
 }

+// Clone is a copy to try decisions on without making them: allocations made
+// on it are its own. Nodes are copied by value; the slices inside them are
+// shared, and a clone must not write through them.
+func (s *Store) Clone() *Store {
+	s.mu.RLock()
+	defer s.mu.RUnlock()
+	c := NewStore()
+	for id, n := range s.nodes {
+		cp := *n
+		c.nodes[id] = &cp
+	}
+	for id, list := range s.allocs {
+		c.allocs[id] = slices.Clone(list)
+	}
+	return c
+}
+
 // Upsert records a node's registration, preserving operator-set state such as
 // a cordon: an agent reconnecting must not silently undo an operator's decision.
 // RecordSMChange keeps a port's change of master subnet manager, the last
@@ -91,6 +91,9 @@ type Request struct {
 	// use. They default to 16384 and 4096, what four parallel slots divide.
 	CtxPool       int `json:"ctx_pool,omitempty"`
 	ContextLength int `json:"context_length,omitempty"`
+	// CPUCores is what a replica computes on; zero takes half the smallest
+	// node it fits on.
+	CPUCores int `json:"cpu_cores,omitempty"`
 	// By is who asked, for the catalog's record.
 	By string `json:"-"`
 }
@@ -278,13 +281,20 @@ func (s *Server) run(j *Job) {
 	}
 	s.step(j, "check", Done, fmt.Sprintf("%s, needs about %d GB (%s)", orUnknown(header.Architecture), memGB, basis))

+	cores := req.CPUCores
+	if cores <= 0 {
+		if nc, ok := s.deps.Fleet.(interface{ NodeCPUs(int) []int }); ok {
+			cores = catalog.SuggestCores(nc.NodeCPUs(memGB))
+		}
+	}
+
 	// 4. The catalog.
 	m := catalog.Model{
 		Name: name, Aliases: req.Aliases, Description: req.Description, Source: source,
 		RoutingProfile: "chat", ContextLength: req.ContextLength, MaxOutput: 1024,
 		Profiles: []catalog.Profile{{
 			Name: "cpu", Engine: "llamacpp", Weight: 1, Priority: 100,
-			Args: []string{"--ctx-size", strconv.Itoa(req.CtxPool)}, MinMemoryGB: memGB,
+			Args: []string{"--ctx-size", strconv.Itoa(req.CtxPool)}, MinMemoryGB: memGB, CPUCores: cores,
 		}},
 		Scaling: catalog.Scaling{Min: req.Replicas, Max: max(req.Replicas, 1)},
 	}
@@ -18,6 +18,7 @@ package modelsvc
 import (
 	"context"
 	"encoding/json"
+	"errors"
 	"fmt"
 	"log/slog"
 	"sync"
@@ -104,6 +105,16 @@ func (s *Service) Add(ctx context.Context, m catalog.Model, by string) error {
 	if s.Declared(m.Name) {
 		return fmt.Errorf("model %q is declared in configuration; edit that rather than adding a second definition", m.Name)
 	}
+	// A model with no GPU is all host memory, and one that declares none
+	// fits everywhere as far as placement can tell — however many of it
+	// there already are. Required of what is added now; models stored before
+	// this still load, and say what they are missing.
+	for _, p := range m.Profiles {
+		if p.CPUOnly() && p.MinMemoryGB <= 0 {
+			return fmt.Errorf("model %q profile %q runs without a GPU and declares no memory (min_memory_gb): "+
+				"placement could not tell when a node is full", m.Name, p.Name)
+		}
+	}
 	if s.check != nil {
 		if err := s.check(ctx, m); err != nil {
 			return err
@@ -189,36 +200,85 @@ func (s *Service) Remove(ctx context.Context, name string) error {
 	return nil
 }

-// Scale changes how many replicas a model runs, without a restart.
-//
-// Only the scaling block moves. Everything else about the model — its source,
-// its profiles, the engine arguments somebody tuned — is read back from what
-// was stored and written out again untouched, so changing a number cannot
-// quietly become a redefinition.
-//
-// A model declared in configuration is refused, for the same reason Remove
-// refuses one: the YAML is read on every start, so the new number would hold
-// until the next restart and then silently revert. A cluster that disagrees
-// with its own config file is worse than an error message saying where to edit.
+// Scale changes how many replicas a model added at runtime runs.
 func (s *Service) Scale(ctx context.Context, name string, sc catalog.Scaling, by string) (catalog.Model, error) {
+	final, err := s.update(ctx, name, by, "change its scaling", func(m *catalog.Model) error {
+		m.Scaling = sc
+		return nil
+	})
+	if err == nil {
+		s.log.Info("model scaled", "model", name,
+			"min", final.Scaling.Min, "max", final.Scaling.Max, "by", by)
+	}
+	return final, err
+}
+
+// Resources sets what a model's profile reserves of a node: host memory, and
+// the CPU cores a replica without a GPU computes on. An empty profile name
+// sets every profile.
+func (s *Service) Resources(ctx context.Context, name, profile string, memoryGB, cpuCores int, by string) (catalog.Model, error) {
+	if memoryGB < 0 || cpuCores < 0 {
+		return catalog.Model{}, errors.New("memory and cores cannot be negative")
+	}
+	final, err := s.update(ctx, name, by, "change its resources", func(m *catalog.Model) error {
+		found := false
+		for i := range m.Profiles {
+			if profile != "" && m.Profiles[i].Name != profile {
+				continue
+			}
+			found = true
+			m.Profiles[i].MinMemoryGB = memoryGB
+			m.Profiles[i].CPUCores = cpuCores
+		}
+		if !found {
+			return fmt.Errorf("model %q has no profile %q", name, profile)
+		}
+		return nil
+	})
+	if err == nil {
+		s.log.Info("model resources set", "model", name, "profile", profile,
+			"memory_gb", memoryGB, "cpu_cores", cpuCores, "by", by)
+	}
+	return final, err
+}
+
+// SetClass sets a model's capacity class: critical, standard or
+// opportunistic, or empty to go back to what its scaling implies.
+func (s *Service) SetClass(ctx context.Context, name, class, by string) (catalog.Model, error) {
+	final, err := s.update(ctx, name, by, "change its class", func(m *catalog.Model) error {
+		m.Class = class
+		return nil
+	})
+	if err == nil {
+		s.log.Info("model class set", "model", name, "class", final.EffectiveClass(), "by", by)
+	}
+	return final, err
+}
+
+// update changes one model added at runtime and keeps the change.
+//
+// Built before anything is written, like Add: catalog.New is what applies the
+// defaults and refuses what does not validate, so the model that comes back
+// is the one that will run rather than the one that was asked for.
+func (s *Service) update(ctx context.Context, name, by, what string, change func(*catalog.Model) error) (catalog.Model, error) {
 	s.mu.Lock()
 	defer s.mu.Unlock()

 	if s.Declared(name) {
 		return catalog.Model{}, fmt.Errorf(
-			"model %q is declared in configuration; change its scaling there, or it reverts on the next restart", name)
+			"model %q is declared in configuration; %s there, or it reverts on the next restart", name, what)
 	}
-
 	stored, err := s.storedModels(ctx)
 	if err != nil {
 		return catalog.Model{}, err
 	}
-
 	next := append([]catalog.Model{}, s.declared...)
 	found := false
 	for _, e := range stored {
 		if e.Name == name {
-			e.Scaling = sc
+			if err := change(&e); err != nil {
+				return catalog.Model{}, err
+			}
 			found = true
 		}
 		next = append(next, e)
@@ -226,20 +286,14 @@ func (s *Service) Scale(ctx context.Context, name string, sc catalog.Scaling, by
 	if !found {
 		return catalog.Model{}, fmt.Errorf("model %q was not added at runtime", name)
 	}
-
-	// Built before anything is written, like Add: catalog.New is what applies
-	// the defaults (max follows min when it is left out) and what refuses a
-	// max below a min, so the model that comes back is the one that will run
-	// rather than the one that was asked for.
 	built, err := catalog.New(next)
 	if err != nil {
-		return catalog.Model{}, fmt.Errorf("scaling %q that way would make the catalog invalid: %w", name, err)
+		return catalog.Model{}, fmt.Errorf("that would make the catalog invalid: %w", err)
 	}
 	final, ok := built.Resolve(name)
 	if !ok {
 		return catalog.Model{}, fmt.Errorf("model %q vanished from the catalog it was just built into", name)
 	}
-
 	def, err := json.Marshal(final)
 	if err != nil {
 		return catalog.Model{}, err
@@ -249,10 +303,7 @@ func (s *Service) Scale(ctx context.Context, name string, sc catalog.Scaling, by
 	}); err != nil {
 		return catalog.Model{}, err
 	}
-
 	s.apply.ReplaceCatalog(ctx, built)
-	s.log.Info("model scaled", "model", name,
-		"min", final.Scaling.Min, "max", final.Scaling.Max, "by", by)
 	return final, nil
 }

@@ -48,7 +48,7 @@ func model(name string) catalog.Model {
 	return catalog.Model{
 		Name: name, Source: "/models/" + name,
 		Scaling:  catalog.Scaling{Min: 1, Max: 1},
-		Profiles: []catalog.Profile{{Name: "p", Engine: "llamacpp", Weight: 1}},
+		Profiles: []catalog.Profile{{Name: "p", Engine: "llamacpp", Weight: 1, MinMemoryGB: 2}},
 	}
 }

@@ -337,3 +337,43 @@ func TestMinZeroWithAMaxIsOnDemand(t *testing.T) {
 		t.Errorf("the idle timeout was lost; got %v", got.Scaling.IdleTimeout)
 	}
 }
+
+func TestACPUModelAddedNowMustSayHowMuchMemoryItNeeds(t *testing.T) {
+	s, st, _ := newService(t)
+	m := model("bare")
+	m.Profiles[0].MinMemoryGB = 0
+	err := s.Add(t.Context(), m, "tester")
+	if err == nil || !strings.Contains(err.Error(), "declares no memory") {
+		t.Fatalf("a CPU model with no memory was added: %v", err)
+	}
+	if len(st.rows) != 0 {
+		t.Error("it was stored anyway")
+	}
+}
+
+func TestResourcesAreSetOnAStoredModelAndSurviveARestart(t *testing.T) {
+	s, _, _ := newService(t)
+	if err := s.Add(t.Context(), model("m"), "tester"); err != nil {
+		t.Fatal(err)
+	}
+	got, err := s.Resources(t.Context(), "m", "", 6, 3, "tester")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if p := got.Profiles[0]; p.MinMemoryGB != 6 || p.CPUCores != 3 {
+		t.Errorf("profile %+v", p)
+	}
+	if _, err := s.Resources(t.Context(), "m", "nope", 6, 3, "tester"); err == nil {
+		t.Error("a profile the model does not have was accepted")
+	}
+	if _, err := s.Resources(t.Context(), "m", "", -1, 0, "tester"); err == nil {
+		t.Error("negative memory was accepted")
+	}
+	cat, err := s.Load(t.Context())
+	if err != nil {
+		t.Fatal(err)
+	}
+	if m, _ := cat.Resolve("m"); m.Profiles[0].CPUCores != 3 {
+		t.Error("the cores were lost on reload")
+	}
+}
@@ -1,6 +1,7 @@
 package placement

 import (
+	"strings"
 	"testing"

 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
@@ -100,3 +101,38 @@ func TestCPUProfileRespectsAvoid(t *testing.T) {
 		t.Errorf("placed on %q despite being told to avoid it", got)
 	}
 }
+
+func TestCPUCoresAreReservedAsMemoryIs(t *testing.T) {
+	n := node("n1", "", 0)
+	n.CPUCount = 8
+	st := store(n)
+	s := New(st)
+	p := cpuProfile("cpu")
+	p.CPUCores = 3
+
+	for i := range 2 {
+		res := s.Place([]Profile{p}, nil)
+		if res.Placement == nil {
+			t.Fatalf("replica %d did not fit in 8 cores:\n%s", i+1, res.Explain())
+		}
+		st.Allocate("n1", inventory.Allocation{ShardID: string(rune('a' + i)), CPUCores: 3})
+	}
+	// Six of eight given away: a third would need three.
+	res := s.Place([]Profile{p}, nil)
+	if res.Placement != nil {
+		t.Fatal("a third replica was given cores the node does not have")
+	}
+	if got := res.Explain(); !strings.Contains(got, "has 8 CPUs with 2 free, needs 3") {
+		t.Errorf("the refusal should say how many cores are left:\n%s", got)
+	}
+
+	// Unknown counts constrain nothing: an older agent, or a profile that
+	// reserves no cores.
+	old := node("n2", "", 0)
+	if res := New(store(old)).Place([]Profile{p}, nil); res.Placement == nil {
+		t.Error("a node that does not report its CPUs was excluded")
+	}
+	if res := s.Place([]Profile{cpuProfile("cpu")}, nil); res.Placement == nil {
+		t.Error("a profile reserving no cores was refused")
+	}
+}
@@ -39,6 +39,9 @@ type Profile struct {
 	// MinDiskGB is the free space needed to hold the weights, checked only on
 	// nodes that do not already have them. Zero is unconstrained.
 	MinDiskGB int
+	// CPUCores is the cores a profile with no GPU reserves. Zero is
+	// unconstrained.
+	CPUCores int
 	// Source is the model's weights, as the engine would be given it. Used to
 	// prefer nodes that already hold them; empty asks for no preference.
 	Source string
@@ -582,6 +585,18 @@ func (s *Scheduler) eligibleNodes(p Profile, fabric string, avoid []string, reje
 			}
 		}

+		// Cores, for a replica that computes on them: what is left after the
+		// cores already given to the node's other replicas. Same exemptions
+		// as memory: zero declares nothing, and a node that does not report
+		// its CPUs is an older agent, not one with none.
+		if p.CPUOnly() && p.CPUCores > 0 && n.CPUCount > 0 {
+			free := n.CPUCount - s.claimedCores(n.ID)
+			if free < p.CPUCores {
+				reject(fmt.Sprintf("has %d CPUs with %d free, needs %d", n.CPUCount, free, p.CPUCores))
+				continue
+			}
+		}
+
 		if p.CPUOnly() {
 			// Every ready node can run it; there is no card to be free.
 			out = append(out, candidate{node: n})
@@ -723,6 +738,15 @@ func (s *Scheduler) claimedMemory(nodeID string) int {
 	return total
 }

+// claimedCores is the CPU cores already given to replicas on a node.
+func (s *Scheduler) claimedCores(nodeID string) int {
+	total := 0
+	for _, a := range s.inv.Allocations(nodeID) {
+		total += a.CPUCores
+	}
+	return total
+}
+
 // installing says why an engine is missing, when the node has said.
 func installing(n inventory.Node, engine string) string {
 	for _, p := range n.Provision {
@@ -144,6 +144,7 @@ func (r *Reconciler) mergeAdopted(rep *Replica, group []AdoptedShard) {
 		r.inv.Allocate(s.NodeID, inventory.Allocation{
 			ShardID: s.ShardID, ReplicaID: rep.ID, GPUIndices: s.GPUIndices,
 			MemoryGB: r.profileMemory(rep.Model, rep.Profile),
+			CPUCores: r.adoptedCores(rep.Model, rep.Profile),
 		})
 		// The endpoint comes from the leader, which may be in any of the
 		// groups — the rank 0 node is not necessarily the first to register.
@@ -253,3 +254,9 @@ type AdoptedShard struct {
 	// Endpoint is the leader's base URL, set on the rank 0 shard.
 	Endpoint string
 }
+
+// adoptedCores is what an adopted shard is taken to hold of its node's CPUs.
+func (r *Reconciler) adoptedCores(model, profile string) int {
+	p, _ := r.profileOf(model, profile)
+	return cpuCores(p)
+}
@@ -0,0 +1,86 @@
+package reconciler
+
+import (
+	"slices"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/placement"
+)
+
+// Headroom is how many more replicas of a model the cluster has room for now.
+type Headroom struct {
+	// Current is the replicas the model has that hold capacity: starting or
+	// ready, not those on their way out.
+	Current int
+	// More is how many more could be placed now, at most the limit asked.
+	More int
+	// Nodes is where each of those would go, in order, a replica's nodes
+	// joined when it spans several.
+	Nodes [][]string
+	// Limited is set when More reached the limit asked, so there may be room
+	// for more than that. A CPU profile that declares no memory is always
+	// limited: nothing it asks for runs out.
+	Limited bool
+	// Blocked is why one more than More does not fit, a line per node that
+	// was considered. Empty when Limited.
+	Blocked []string
+}
+
+// Headroom tries placing up to limit more replicas of a model on a copy of
+// the inventory, each one's allocation made on the copy before the next is
+// tried, exactly as the reconciler would place them one pass at a time.
+//
+// The scheduler's own decisions, not an estimate beside them: what the page
+// offers is then what the next passes will do, and a count that the
+// arithmetic of a second implementation allowed but placement refuses would
+// leave a model short with nobody told why.
+func (r *Reconciler) Headroom(model string, limit int) (Headroom, bool) {
+	m, ok := r.cat.Load().Resolve(model)
+	if !ok {
+		return Headroom{}, false
+	}
+	var h Headroom
+	var avoid []string
+	r.mu.Lock()
+	for _, rep := range r.replicas {
+		if rep.Model == m.Name && rep.State.Counts() && rep.RetiringAt.IsZero() {
+			h.Current++
+			avoid = append(avoid, rep.NodeIDs()...)
+		}
+	}
+	r.mu.Unlock()
+
+	inv := r.inv.Clone()
+	sched := placement.New(inv)
+	for h.More < limit {
+		res := sched.Place(m.PlacementProfiles(), avoid)
+		if res.Placement == nil && len(avoid) > 0 {
+			// As scaleUp does: spreading is a preference, not a rule.
+			res = sched.Place(m.PlacementProfiles(), nil)
+		}
+		if res.Placement == nil {
+			for _, rej := range res.Rejections {
+				h.Blocked = append(h.Blocked, rej.String())
+			}
+			slices.Sort(h.Blocked)
+			h.Blocked = slices.Compact(h.Blocked)
+			return h, true
+		}
+		mem, cores := 0, 0
+		if prof, ok := m.ProfileByName(res.Placement.Profile.Name); ok {
+			mem, cores = prof.MinMemoryGB, cpuCores(prof)
+		}
+		var nodes []string
+		for _, a := range res.Placement.Nodes {
+			inv.Allocate(a.NodeID, inventory.Allocation{
+				ShardID: "headroom", ReplicaID: "headroom", GPUIndices: a.GPUIndices, MemoryGB: mem, CPUCores: cores,
+			})
+			nodes = append(nodes, a.NodeID)
+		}
+		avoid = append(avoid, nodes...)
+		h.Nodes = append(h.Nodes, nodes)
+		h.More++
+	}
+	h.Limited = true
+	return h, true
+}
@@ -0,0 +1,80 @@
+package reconciler
+
+import (
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
+)
+
+func TestHeadroomIsWhatPlacementWouldDoNext(t *testing.T) {
+	// Two 8-GPU nodes and a 4-GPU profile: four replicas in all, one placed.
+	h := newHarness(t, testModel(1), gpuNode("1", 8), gpuNode("2", 8))
+	h.r.Reconcile(t.Context())
+	h.markReady()
+
+	got, ok := h.r.Headroom("llama", 10)
+	if !ok {
+		t.Fatal("the model was not found")
+	}
+	if got.Current != 1 || got.More != 3 || got.Limited {
+		t.Fatalf("headroom %+v, expected 1 placed and room for 3 more", got)
+	}
+	// Spread first: the next one goes to the node without one.
+	if first := got.Nodes[0][0]; first == h.r.Replicas()[0].NodeIDs()[0] {
+		t.Errorf("the next replica would stack on %s beside the first", first)
+	}
+	if len(got.Blocked) == 0 || !strings.Contains(strings.Join(got.Blocked, "\n"), "GPU") {
+		t.Errorf("no reason given for the fifth: %v", got.Blocked)
+	}
+	// Tried on a copy: nothing was taken from the real inventory.
+	if free := len(h.inv.FreeGPUs("1")) + len(h.inv.FreeGPUs("2")); free != 12 {
+		t.Errorf("%d GPUs free after asking, expected the 12 the one replica leaves", free)
+	}
+
+	capped, _ := h.r.Headroom("llama", 2)
+	if capped.More != 2 || !capped.Limited || capped.Blocked != nil {
+		t.Errorf("asked for at most 2: %+v", capped)
+	}
+	if _, ok := h.r.Headroom("nope", 5); ok {
+		t.Error("an unknown model has headroom")
+	}
+}
+
+func TestHeadroomCountsMemoryAlreadyClaimed(t *testing.T) {
+	cpu := func(id string) inventory.Node {
+		return inventory.Node{ID: id, MgmtAddr: "10.0.0." + id, State: inventory.Ready, MemoryTotalMB: 16 * 1024}
+	}
+	m := catalog.Model{Name: "small", Source: "/m.gguf", Scaling: catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 6}}}
+	h := newHarness(t, m, cpu("a"), cpu("b"))
+	h.r.Reconcile(t.Context())
+
+	// 16 GB each at 6 GB a replica: two per node, one of them taken.
+	got, _ := h.r.Headroom("small", 10)
+	if got.Current != 1 || got.More != 3 {
+		t.Errorf("headroom %+v, expected room for 3 more", got)
+	}
+}
+
+func TestReservedCoresAreTheEnginesThreadsAndBoundTheRoom(t *testing.T) {
+	cpu := inventory.Node{ID: "a", MgmtAddr: "10.0.0.1", State: inventory.Ready, MemoryTotalMB: 64 * 1024, CPUCount: 8}
+	m := catalog.Model{Name: "coder", Source: "/m.gguf", Scaling: catalog.Scaling{Min: 1, Max: 1},
+		Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 6, CPUCores: 3}}}
+	h := newHarness(t, m, cpu)
+	h.r.Reconcile(t.Context())
+
+	started := h.disp.startedShards()
+	if len(started) != 1 || started[0].Spec.Threads != 3 {
+		t.Fatalf("the engine was not told its 3 cores: %+v", started)
+	}
+	if a := h.inv.Allocations("a"); len(a) != 1 || a[0].CPUCores != 3 {
+		t.Errorf("the cores were not reserved: %+v", a)
+	}
+	// Memory would allow nine more; cores allow one: 8 - 3 = 5, then 2.
+	got, _ := h.r.Headroom("coder", 10)
+	if got.More != 1 {
+		t.Errorf("room for %d more, expected 1 by cores: %+v", got.More, got)
+	}
+}
@@ -83,6 +83,12 @@ type ModelHealth struct {
 	// place one found nowhere; Placement says why.
 	Unplaceable bool
 	Placement   string
+	// Class is the capacity class it is treated as. Preempted counts the
+	// replicas it has lost to models of a higher class since the control
+	// plane started; HeldFor names the model it is giving room to now.
+	Class     string
+	Preempted int
+	HeldFor   string
 }

 // ModelHealth reports every model in the catalog.
@@ -106,7 +112,10 @@ func (r *Reconciler) ModelHealth() []ModelHealth {
 	out := make([]ModelHealth, 0, len(models))
 	for _, m := range models {
 		h := ModelHealth{Model: m.Name, Desired: desired[m.Name], Ready: ready[m.Name],
-			Failures: r.attempts[m.Name]}
+			Failures: r.attempts[m.Name], Class: m.EffectiveClass(), Preempted: r.preempted[m.Name]}
+		if hd, ok := r.held[m.Name]; ok && r.now().Before(hd.Until) {
+			h.HeldFor = hd.For
+		}
 		if p, ok := r.parked[m.Name]; ok {
 			h.Parked, h.ParkedSince, h.Reason = true, p.Since, p.Reason
 		}
@@ -0,0 +1,249 @@
+package reconciler
+
+import (
+	"context"
+	"slices"
+	"sort"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/placement"
+)
+
+// hold keeps a model that gave up room from taking it back.
+type hold struct {
+	Until time.Time
+	For   string
+}
+
+// maxVictims bounds how many replicas one placement may stop. Past this the
+// model is better reported as not fitting than paid for with half a cluster.
+const maxVictims = 3
+
+// victim is a replica that could be stopped for another model.
+type victim struct {
+	ID       string
+	Model    string
+	Class    string
+	AboveMin bool
+	Tier     int
+	Idle     time.Duration
+}
+
+// preemptFor stops replicas of lower classes when a model cannot otherwise be
+// placed, and says whether it did.
+//
+// What may be taken from whom:
+//
+//   - A lower class gives way to a higher one: opportunistic to standard,
+//     standard to critical.
+//   - Within a class, only a replica above its model's minimum, and only for
+//     a model still below its own. Room a model grew into under load is not a
+//     promise; its minimum is.
+//   - A critical model's minimum is never taken.
+//
+// Victims only ever come from a lower rank, or from above a minimum for a
+// model below one, so two models cannot take turns stopping each other.
+//
+// The set is tried on a copy of the inventory first, and only stopped if it
+// would let the model be placed: stopping replicas that do not make room is
+// the one outcome worse than not preempting.
+func (r *Reconciler) preemptFor(ctx context.Context, m catalog.Model, avoid []string) bool {
+	cands := r.victims(m)
+	if len(cands) == 0 {
+		return false
+	}
+	trial := r.inv.Clone()
+	sched := placement.New(trial)
+	var chosen []victim
+	for _, v := range cands {
+		if len(chosen) == maxVictims {
+			break
+		}
+		trial.ReleaseReplica(v.ID)
+		chosen = append(chosen, v)
+		res := sched.Place(m.PlacementProfiles(), avoid)
+		if res.Placement == nil && len(avoid) > 0 {
+			res = sched.Place(m.PlacementProfiles(), nil)
+		}
+		if res.Placement == nil {
+			continue
+		}
+		r.preempt(ctx, m, chosen)
+		return true
+	}
+	return false
+}
+
+// preempt drains the chosen replicas and holds their models back.
+func (r *Reconciler) preempt(ctx context.Context, m catalog.Model, chosen []victim) {
+	now := r.now()
+	var ids []string
+	r.mu.Lock()
+	for _, v := range chosen {
+		if rep, ok := r.replicas[v.ID]; ok {
+			rep.PreemptedFor = m.Name
+		}
+		ids = append(ids, v.ID)
+		r.held[v.Model] = hold{Until: now.Add(r.cfg.PreemptHold), For: m.Name}
+		r.preempted[v.Model]++
+	}
+	r.preempting[m.Name] = ids
+	r.mu.Unlock()
+
+	for _, v := range chosen {
+		r.log.Warn("preempting a replica to make room",
+			"replica", v.ID, "model", v.Model, "class", v.Class, "above_min", v.AboveMin,
+			"for", m.Name, "for_class", m.EffectiveClass())
+	}
+	r.scaleDown(ctx, ids)
+}
+
+// awaitingPreemption reports whether replicas stopped for a model are still
+// on their way out, so it waits for them rather than stopping more.
+func (r *Reconciler) awaitingPreemption(model string) bool {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	ids := r.preempting[model]
+	ids = slices.DeleteFunc(slices.Clone(ids), func(id string) bool {
+		_, ok := r.replicas[id]
+		return !ok
+	})
+	if len(ids) == 0 {
+		delete(r.preempting, model)
+		return false
+	}
+	r.preempting[model] = ids
+	return true
+}
+
+// isHeld reports whether a model gave up room and may not take it back yet.
+// The hold ends early once the model it was given to has what it asked for.
+//
+// Today a preempted replica is only removed by its own model's drain loop,
+// which runs after that model has decided not to scale up, so the room it
+// frees reaches the model it was freed for first anyway. The hold is what
+// keeps that true for any other way a replica can go — a stop reported
+// between passes, a reap at the start of one — rather than it resting on the
+// order of lines in reconcileModel.
+func (r *Reconciler) isHeld(model string) bool {
+	r.mu.Lock()
+	h, ok := r.held[model]
+	r.mu.Unlock()
+	if !ok {
+		return false
+	}
+	if r.now().After(h.Until) {
+		r.mu.Lock()
+		delete(r.held, model)
+		r.mu.Unlock()
+		return false
+	}
+	if to, ok := r.cat.Load().Resolve(h.For); ok && r.liveCount(h.For) >= r.desiredFor(to) {
+		r.mu.Lock()
+		delete(r.held, model)
+		r.mu.Unlock()
+		return false
+	}
+	return true
+}
+
+// liveCount is how many replicas of a model hold capacity.
+func (r *Reconciler) liveCount(model string) int {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	n := 0
+	for _, rep := range r.replicas {
+		if rep.Model == model && rep.State.Counts() && rep.RetiringAt.IsZero() {
+			n++
+		}
+	}
+	return n
+}
+
+// victims lists the replicas that may be stopped for m, in the order they
+// should be: idle on-demand ones first, then what lower classes grew into
+// under load, then the same within m's class, then lower classes' minimums;
+// within each, the longest unused first.
+func (r *Reconciler) victims(m catalog.Model) []victim {
+	cat := r.cat.Load()
+	now := r.now()
+	myRank := catalog.ClassRank(m.EffectiveClass())
+	belowMin := r.liveCount(m.Name) < m.Scaling.Min
+
+	r.mu.Lock()
+	defer r.mu.Unlock()
+
+	// Per model, its live replicas newest first: the newest are the ones
+	// above its minimum.
+	byModel := map[string][]*Replica{}
+	for _, rep := range r.replicas {
+		if rep.Model == m.Name || !rep.State.Counts() || !rep.RetiringAt.IsZero() || rep.PreemptedFor != "" {
+			continue
+		}
+		byModel[rep.Model] = append(byModel[rep.Model], rep)
+	}
+
+	var out []victim
+	for model, reps := range byModel {
+		other, ok := cat.Resolve(model)
+		if !ok {
+			continue
+		}
+		class := other.EffectiveClass()
+		rank := catalog.ClassRank(class)
+		sort.Slice(reps, func(i, j int) bool { return reps[i].CreatedAt.After(reps[j].CreatedAt) })
+		above := max(len(reps)-other.Scaling.Min, 0)
+		for i, rep := range reps {
+			aboveMin := i < above
+			tier := -1
+			switch {
+			case rank < myRank && other.Scaling.OnDemand() && idleFor(rep, now) >= r.cfg.EvictionGrace:
+				tier = 0
+			case rank < myRank && aboveMin:
+				tier = 1
+			case rank == myRank && aboveMin && belowMin:
+				tier = 2
+			case rank < myRank && class != catalog.ClassCritical:
+				tier = 3
+			}
+			if tier < 0 {
+				continue
+			}
+			// Just started: stopping it throws away a model load that has
+			// only now been paid for.
+			if !rep.ReadyAt.IsZero() && now.Sub(rep.ReadyAt) < r.cfg.MinLifetime {
+				continue
+			}
+			if len(nodesOf(rep)) > 1 {
+				continue // see makeRoomFor: a multi-node replica is its own decision
+			}
+			out = append(out, victim{ID: rep.ID, Model: model, Class: class,
+				AboveMin: aboveMin, Tier: tier, Idle: idleFor(rep, now)})
+		}
+	}
+	sort.SliceStable(out, func(i, j int) bool {
+		if out[i].Tier != out[j].Tier {
+			return out[i].Tier < out[j].Tier
+		}
+		return out[i].Idle > out[j].Idle
+	})
+	return out
+}
+
+// idleFor is how long a replica has gone unused; zero when nobody has said,
+// which is not the same as idle.
+func idleFor(rep *Replica, now time.Time) time.Duration {
+	if rep.LastUsedAt.IsZero() {
+		return 0
+	}
+	return now.Sub(rep.LastUsedAt)
+}
+
+func nodesOf(rep *Replica) map[string]struct{} {
+	out := map[string]struct{}{}
+	for _, sh := range rep.Shards {
+		out[sh.NodeID] = struct{}{}
+	}
+	return out
+}
@@ -0,0 +1,237 @@
+package reconciler
+
+import (
+	"log/slog"
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/inventory"
+)
+
+// newFleetHarness is newHarness with a catalog of several models.
+func newFleetHarness(t *testing.T, models []catalog.Model, nodes ...inventory.Node) *harness {
+	t.Helper()
+	cat, err := catalog.New(models)
+	if err != nil {
+		t.Fatal(err)
+	}
+	inv := inventory.NewStore()
+	c := newClock()
+	for _, n := range nodes {
+		inv.Upsert(n)
+	}
+	disp := &fakeDispatcher{}
+	cfg := DefaultConfig()
+	cfg.UnreachableAfter = 15 * time.Second
+	cfg.LostAfter = 60 * time.Second
+	r := New(cfg, catalog.NewLive(cat), inv, disp, slog.New(slog.DiscardHandler))
+	r.SetClock(c.now)
+	c.advance(cfg.SettleFor)
+	for _, n := range nodes {
+		touchAt(inv, n.ID, c.now())
+	}
+	return &harness{r: r, inv: inv, disp: disp, clock: c}
+}
+
+// gpuModel runs on one whole 8-GPU node.
+func gpuModel(name, class string, min int) catalog.Model {
+	return catalog.Model{Name: name, Source: "/models/" + name, Class: class,
+		Scaling:  catalog.Scaling{Min: min, Max: min},
+		Profiles: []catalog.Profile{{Name: "tp8", Engine: "vllm", GPUClass: "h100-80g", TP: 8}}}
+}
+
+// settle runs passes until nothing changes, marking what starts ready and
+// letting preempted replicas run out their grace.
+func (h *harness) settle(t *testing.T, passes int) {
+	t.Helper()
+	for range passes {
+		h.r.Reconcile(t.Context())
+		h.markReady()
+		h.advance(h.r.cfg.PreemptGrace + time.Second)
+	}
+}
+
+func replicasOf(h *harness, model string) (live int) {
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == model && rep.State.Counts() {
+			live++
+		}
+	}
+	return live
+}
+
+// fill places the standard model, then ages it past MinLifetime so it may be
+// taken from, and only then adds the model that wants its node.
+func fillThenWant(t *testing.T, want catalog.Model, have catalog.Model, nodes ...inventory.Node) *harness {
+	t.Helper()
+	h := newFleetHarness(t, []catalog.Model{have, withMin(want, 0)}, nodes...)
+	h.settle(t, 3)
+	if got := replicasOf(h, have.Name); got != have.Scaling.Min {
+		t.Fatalf("%s has %d replicas before anything wanted its room", have.Name, got)
+	}
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	cat, err := catalog.New([]catalog.Model{have, want})
+	if err != nil {
+		t.Fatal(err)
+	}
+	h.r.cat.Store(cat)
+	return h
+}
+
+func withMin(m catalog.Model, n int) catalog.Model {
+	m.Scaling = catalog.Scaling{Min: n, Max: n}
+	return m
+}
+
+func TestACriticalModelTakesAStandardModelsNode(t *testing.T) {
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+
+	h.r.Reconcile(t.Context())
+	var draining int
+	for _, rep := range h.r.Replicas() {
+		if rep.Model == "chat" && rep.State == Draining && rep.PreemptedFor == "vital" {
+			draining++
+		}
+	}
+	if draining != 1 {
+		t.Fatalf("the standard replica was not drained for the critical one: %+v", h.r.Replicas())
+	}
+	if hs := health(h, "chat"); hs.HeldFor != "vital" || hs.Preempted != 1 {
+		t.Errorf("the victim's health does not say it gave way: %+v", hs)
+	}
+
+	h.settle(t, 4)
+	if replicasOf(h, "vital") != 1 {
+		t.Fatalf("the critical model was not placed once room was made: %+v", h.r.Replicas())
+	}
+	// Held back until the critical model had its replica, then short again
+	// because there is no room: it must not have taken the node back — not
+	// even for a moment, which would load a model only to preempt it again.
+	if replicasOf(h, "chat") != 0 {
+		t.Error("the standard model took the node back")
+	}
+	started := 0
+	for _, cmd := range h.disp.startedShards() {
+		if strings.HasPrefix(cmd.ReplicaID, "chat-") {
+			started++
+		}
+	}
+	if started != 1 {
+		t.Errorf("the standard model started %d replicas; it should not have tried again", started)
+	}
+}
+
+func TestAStandardModelDoesNotTakeAnothersMinimum(t *testing.T) {
+	h := fillThenWant(t, gpuModel("second", "", 1), gpuModel("first", "", 1), gpuNode("1", 8))
+	h.settle(t, 3)
+	if replicasOf(h, "first") != 1 || replicasOf(h, "second") != 0 {
+		t.Errorf("a model of the same class took another's minimum: %+v", h.r.Replicas())
+	}
+	if hs := health(h, "second"); !hs.Unplaceable {
+		t.Error("the model left without room should say it cannot be placed")
+	}
+}
+
+func TestWithinAClassOnlyWhatWasGrownIntoIsTaken(t *testing.T) {
+	grown := gpuModel("busy", "", 1)
+	grown.Scaling = catalog.Scaling{Min: 1, Max: 2}
+	h := newFleetHarness(t, []catalog.Model{grown, withMin(gpuModel("new", "", 1), 0)}, gpuNode("1", 8), gpuNode("2", 8))
+	h.settle(t, 2)
+	// Load grows it to its max: the second replica is above its minimum.
+	h.r.mu.Lock()
+	h.r.extra["busy"] = 1
+	h.r.mu.Unlock()
+	h.settle(t, 3)
+	if replicasOf(h, "busy") != 2 {
+		t.Fatalf("setup: busy did not grow to 2: %+v", h.r.Replicas())
+	}
+	h.advance(h.r.cfg.MinLifetime + time.Minute)
+	cat, _ := catalog.New([]catalog.Model{grown, gpuModel("new", "", 1)})
+	h.r.cat.Store(cat)
+
+	h.settle(t, 5)
+	if replicasOf(h, "new") != 1 || replicasOf(h, "busy") != 1 {
+		t.Errorf("the grown replica should give way to a minimum: busy %d, new %d",
+			replicasOf(h, "busy"), replicasOf(h, "new"))
+	}
+}
+
+func TestNothingIsStoppedThatWouldNotMakeRoom(t *testing.T) {
+	// Two nodes, each half used by an opportunistic model; the critical one
+	// needs a whole node. Stopping one frees half a node, which is no use —
+	// and both together are on two nodes, which is no use either.
+	half := catalog.Model{Name: "half", Source: "/m", Class: catalog.ClassOpportunistic,
+		Scaling:  catalog.Scaling{Min: 2, Max: 2},
+		Profiles: []catalog.Profile{{Name: "tp4", Engine: "vllm", GPUClass: "h100-80g", TP: 4}}}
+	five := func(id string) inventory.Node { return gpuNode(id, 5) }
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), half, five("1"), five("2"))
+	h.r.Reconcile(t.Context())
+	for _, rep := range h.r.Replicas() {
+		if rep.PreemptedFor != "" || rep.State == Draining {
+			t.Fatalf("a replica was stopped without making room: %+v", rep)
+		}
+	}
+}
+
+func TestACriticalMinimumIsNeverTaken(t *testing.T) {
+	h := fillThenWant(t, gpuModel("also-vital", catalog.ClassCritical, 1), gpuModel("vital", catalog.ClassCritical, 1), gpuNode("1", 8))
+	h.settle(t, 3)
+	if replicasOf(h, "vital") != 1 {
+		t.Error("a critical model's minimum was taken")
+	}
+}
+
+func TestAReplicaThatJustStartedIsLeftAlone(t *testing.T) {
+	h := newFleetHarness(t, []catalog.Model{gpuModel("chat", "", 1), withMin(gpuModel("vital", catalog.ClassCritical, 1), 0)}, gpuNode("1", 8))
+	h.settle(t, 2)
+	cat, _ := catalog.New([]catalog.Model{gpuModel("chat", "", 1), gpuModel("vital", catalog.ClassCritical, 1)})
+	h.r.cat.Store(cat)
+	h.r.Reconcile(t.Context())
+	for _, rep := range h.r.Replicas() {
+		if rep.State == Draining {
+			t.Fatal("a replica that had only just loaded was preempted")
+		}
+	}
+}
+
+func health(h *harness, model string) ModelHealth {
+	for _, x := range h.r.ModelHealth() {
+		if x.Model == model {
+			return x
+		}
+	}
+	return ModelHealth{}
+}
+
+func TestAHoldLastsUntilTheRoomIsUsedOrItExpires(t *testing.T) {
+	h := fillThenWant(t, gpuModel("vital", catalog.ClassCritical, 1), gpuModel("chat", "", 1), gpuNode("1", 8))
+	h.r.Reconcile(t.Context())
+	if !h.r.isHeld("chat") {
+		t.Fatal("the model that gave way is not held")
+	}
+	if h.r.isHeld("vital") {
+		t.Error("the model room was made for is held")
+	}
+	// Released as soon as the model it was given to has what it asked for.
+	h.settle(t, 3)
+	if replicasOf(h, "vital") != 1 {
+		t.Fatalf("setup: vital was not placed: %+v", h.r.Replicas())
+	}
+	if h.r.isHeld("chat") {
+		t.Error("the hold outlasted its purpose")
+	}
+
+	// And, whatever happens, after PreemptHold.
+	h.r.mu.Lock()
+	h.r.held["other"] = hold{Until: h.r.now().Add(time.Minute), For: "never-placed"}
+	h.r.mu.Unlock()
+	if !h.r.isHeld("other") {
+		t.Fatal("a fresh hold is not in force")
+	}
+	h.advance(2 * time.Minute)
+	if h.r.isHeld("other") {
+		t.Error("a hold outlived its time")
+	}
+}
@@ -65,6 +65,12 @@ type Config struct {
 	// Without a bound, draining a node whose model cannot be placed anywhere
 	// else would never finish.
 	DrainGrace time.Duration
+	// PreemptGrace is how long a replica stopped to make room for another
+	// is given to finish what it is serving: a request, not a replacement.
+	PreemptGrace time.Duration
+	// PreemptHold is how long a model that gave up room may not take it back,
+	// so the replica it was given up for is placed first.
+	PreemptHold time.Duration
 	// ScaleUpWaiting is the average number of requests waiting per ready
 	// replica, held for ScaleUpAfter, that adds a replica — up to the model's
 	// max. ScaleDownAfter is how long with nothing waiting before one of the
@@ -141,6 +147,8 @@ func DefaultConfig() Config {
 		EvictionGrace:    2 * time.Minute,
 		MinLifetime:      5 * time.Minute,
 		DrainGrace:       5 * time.Minute,
+		PreemptGrace:     30 * time.Second,
+		PreemptHold:      5 * time.Minute,
 		ParkAfter:        5,
 		ScaleUpWaiting:   1,
 		ScaleUpAfter:     time.Minute,
@@ -158,6 +166,12 @@ func (c *Config) normalize() {
 	if c.BaseBackoff <= 0 {
 		c.BaseBackoff = d.BaseBackoff
 	}
+	if c.PreemptGrace <= 0 {
+		c.PreemptGrace = d.PreemptGrace
+	}
+	if c.PreemptHold <= 0 {
+		c.PreemptHold = d.PreemptHold
+	}
 	if c.MaxBackoff < c.BaseBackoff {
 		c.MaxBackoff = max(c.BaseBackoff, d.MaxBackoff)
 	}
@@ -250,6 +264,12 @@ type Reconciler struct {
 	// placementLogged remembers the last "no placement" said per model, so it
 	// is said when it changes rather than on every five-second pass.
 	placementLogged map[string]placementNote
+	// preempting is, per model, the replicas stopped to make room for it and
+	// not gone yet; held is, per model that gave them up, until when it may
+	// not take the room back; preempted counts, per model, replicas it lost.
+	preempting map[string][]string
+	held       map[string]hold
+	preempted  map[string]int

 	wake chan struct{}
 	stop chan struct{}
@@ -285,6 +305,9 @@ func New(cfg Config, cat *catalog.Live, inv *inventory.Store, disp Dispatcher, l
 		quietSince:      map[string]time.Time{},
 		placementLogged: map[string]placementNote{},
 		lastExplain:     map[string]placement.Result{},
+		preempting:      map[string][]string{},
+		held:            map[string]hold{},
+		preempted:       map[string]int{},
 		wake:            make(chan struct{}, 1),
 		stop:            make(chan struct{}),
 		done:            make(chan struct{}),
@@ -704,6 +727,9 @@ func (r *Reconciler) reconcileModel(ctx context.Context, m catalog.Model) {
 	switch {
 	case len(liveIDs) < desired && isParked:
 		// Not placed. Its failure is on every status the operator reads.
+	case len(liveIDs) < desired && r.isHeld(m.Name):
+		// It gave up room to a model of a higher class, which is placed
+		// first; taking it back now would undo the preemption.
 	case len(liveIDs) < desired:
 		if r.now().Before(blockedUntil) {
 			// Still in backoff. Rebuilding immediately would hot-loop on a
@@ -725,7 +751,11 @@ func (r *Reconciler) reconcileModel(ctx context.Context, m catalog.Model) {
 		rep, ok := r.replicas[id]
 		var expired bool
 		if ok {
-			expired = !rep.DrainingAt.IsZero() && r.now().Sub(rep.DrainingAt) > r.cfg.DrainGrace
+			grace := r.cfg.DrainGrace
+			if rep.PreemptedFor != "" {
+				grace = r.cfg.PreemptGrace
+			}
+			expired = !rep.DrainingAt.IsZero() && r.now().Sub(rep.DrainingAt) > grace
 		}
 		r.mu.Unlock()

@@ -775,6 +805,9 @@ func (r *Reconciler) scaleUp(ctx context.Context, m catalog.Model, avoid []strin
 		if m.Scaling.OnDemand() && r.makeRoomFor(ctx, m) {
 			return
 		}
+		if r.awaitingPreemption(m.Name) || r.preemptFor(ctx, m, avoid) {
+			return
+		}
 		r.notePlacementFailure(m.Name, res.Explain())
 		return
 	}
@@ -825,7 +858,7 @@ func (r *Reconciler) startReplica(ctx context.Context, m catalog.Model, pl *plac
 		})
 		r.inv.Allocate(a.NodeID, inventory.Allocation{
 			ShardID: shardID, ReplicaID: id, GPUIndices: a.GPUIndices,
-			MemoryGB: prof.MinMemoryGB,
+			MemoryGB: prof.MinMemoryGB, CPUCores: cpuCores(prof),
 		})
 	}
 	rep.Endpoint = fmt.Sprintf("http://%s:%d", addrOf(leaderNode), basePort)
@@ -869,6 +902,7 @@ func (r *Reconciler) startReplica(ctx context.Context, m catalog.Model, pl *plac
 				HTTPPort:      s.HTTPPort,
 				Profile:       espec,
 				APIKey:        r.engineKey(id),
+				Threads:       cpuCores(prof),
 			},
 			StartupTimeout: prof.StartupTimeout,
 			Pool:           pl.Pool,
@@ -1108,16 +1142,31 @@ func addrOf(n inventory.Node) string {
 // claim on restart would place a second large model onto a node already full of
 // the first.
 func (r *Reconciler) profileMemory(model, profile string) int {
+	p, _ := r.profileOf(model, profile)
+	return p.MinMemoryGB
+}
+
+// profileOf is a replica's profile as the catalog holds it now.
+func (r *Reconciler) profileOf(model, profile string) (catalog.Profile, bool) {
 	m, ok := r.cat.Load().Resolve(model)
 	if !ok {
-		return 0
+		return catalog.Profile{}, false
 	}
 	for _, p := range m.Profiles {
 		if p.Name == profile {
-			return p.MinMemoryGB
+			return p, true
 		}
 	}
-	return 0
+	return catalog.Profile{}, false
+}
+
+// cpuCores is what a profile reserves of a node's CPUs: its cores when it
+// computes on them, nothing when a GPU does the work.
+func cpuCores(p catalog.Profile) int {
+	if !p.CPUOnly() {
+		return 0
+	}
+	return p.CPUCores
 }

 func (r *Reconciler) engineKey(replicaID string) string {
@@ -156,6 +156,8 @@ type Replica struct {
 	// DrainingAt is when the replica stopped taking new work, so the wait for a
 	// replacement can be bounded.
 	DrainingAt time.Time
+	// PreemptedFor is the model this replica was stopped to make room for.
+	PreemptedFor string
 	// RetiringAt is set while the replica is being restarted: it keeps serving
 	// but no longer counts toward the model's replicas, so a replacement is
 	// placed beside it, and it is drained only once that replacement is ready.
@@ -7,11 +7,13 @@ import (
 	"fmt"
 	"html/template"
 	"net/http"
+	"slices"
 	"sort"
 	"strconv"
 	"strings"
 	"time"

+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/gguf"
 	"gitea.mynodes.xyz/ilker/llmgw/internal/hf"
@@ -67,6 +69,18 @@ type modelUse struct {
 	Name     string
 	Mode     string // always, ondemand or stopped
 	Replicas int
+	// Min and Max are its scaling as the catalog holds it.
+	Min, Max int
+	// Ready, Wanted and Nodes are how it runs now, for the row itself:
+	// replicas ready of those it should have, and the nodes they are on.
+	// Fleet is false where nothing could be asked, so the row says nothing
+	// rather than "0 of 0".
+	Fleet         bool
+	Ready, Wanted int
+	Starting      int
+	Nodes         []nodeName
+	Parked        bool
+	Unplaceable   bool
 	// Declared models are read from server.yaml on every start, so a mode
 	// changed here would revert; the page says where to change it instead.
 	Declared bool
@@ -83,6 +97,11 @@ type mirrorRow struct {
 	// operator clicked. Nil on a row that is still arriving: a catalog entry
 	// pointing at a partial file is the thing versioned paths exist to prevent.
 	Serve *serveForm
+	// Detail is the card opened beneath the row: what runs it, on which
+	// nodes, and how many replicas it runs.
+	Detail *heldDetail
+	// Open is the row's ?open= value.
+	Open string

 	// HasPercent is false when the upstream never sent a length. The bar is
 	// then not drawn at all: a progress bar that has to invent its own total is
@@ -100,7 +119,11 @@ type mirrorRow struct {
 }

 type adminMirrorData struct {
-	Nav  Nav
+	Nav Nav
+	// Tab is the part of the page shown: held, explore or fetch. Tabs are the
+	// links to each.
+	Tab  string
+	Tabs []mirrorTab
 	Addr string
 	Base string
 	Rows []mirrorRow
@@ -270,6 +293,8 @@ func (p *AccountPages) RegisterAdminMirror(mux *http.ServeMux, d MirrorDeps) {
 				return name + " now runs all the time", nil
 			}))

+		mux.HandleFunc("POST /admin/mirror/replicas", p.mirrorAction(m, p.setReplicas))
+
 		mux.HandleFunc("POST /admin/mirror/serve", p.mirrorAction(m,
 			func(ctx context.Context, admin store.User, r *http.Request) (string, error) {
 				model, err := buildModel(r.FormValue)
@@ -361,7 +386,14 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
 	}

 	d.HasCatalog = p.catalog != nil
-	p.explore(r, &d)
+	d.HasRegistry = p.registry != nil
+	d.Tab = mirrorTabFor(r, errMsg)
+	// Only the tab shown does its work: the registry is a server across the
+	// internet, and asking it for sixty repositories to show somebody who came
+	// to look at a download's progress made every visit wait on it.
+	if d.Tab == tabExplore {
+		p.explore(r, &d)
+	}

 	got, err := m.List(r.Context())
 	if err != nil {
@@ -370,6 +402,7 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
 		if d.Error == "" {
 			d.Error = err.Error()
 		}
+		d.Tabs = mirrorTabs(d)
 		render(w, "admin_mirror", d)
 		return
 	}
@@ -412,12 +445,46 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
 		}
 		d.Rows = append(d.Rows, row)
 	}
+	var views map[string]adminapi.ModelView
+	prefix := ""
+	if p.fleet != nil && len(sources) > 0 {
+		views = map[string]adminapi.ModelView{}
+		for _, v := range p.fleet.Models() {
+			views[v.Name] = v
+		}
+		var ids []string
+		for _, n := range p.fleet.Nodes() {
+			ids = append(ids, n.ID)
+		}
+		prefix = sharedPrefix(ids)
+	}
 	for i := range d.Rows {
 		for _, m := range sources[d.Rows[i].URL] {
-			d.Rows[i].UsedBy = append(d.Rows[i].UsedBy, modelUse{
+			u := modelUse{
 				Name: m.Name, Mode: modeOf(m.Scaling), Replicas: max(m.Scaling.Max, 1),
+				Min: m.Scaling.Min, Max: m.Scaling.Max,
 				Declared: p.catalog.Declared(m.Name),
-			})
+			}
+			if v, ok := views[m.Name]; ok {
+				u.Fleet, u.Ready, u.Wanted = true, v.Ready, v.Desired
+				u.Parked, u.Unplaceable = v.Parked, v.Unplaceable
+				var nodes []string
+				nodes, u.Starting = runningOn(p.fleet.Replicas(m.Name))
+				for _, n := range nodes {
+					u.Nodes = append(u.Nodes, nodeName{Full: n, Short: strings.TrimPrefix(n, prefix)})
+				}
+			}
+			d.Rows[i].UsedBy = append(d.Rows[i].UsedBy, u)
+		}
+	}
+
+	// The one row whose card is open. From the form too, so a change made in
+	// the card comes back with the card still open to show it.
+	open := r.FormValue("open")
+	for i := range d.Rows {
+		d.Rows[i].Open = openKey(d.Rows[i].Model, d.Rows[i].Version)
+		if open != "" && d.Rows[i].Open == open && d.Rows[i].State == mirror.Present {
+			d.Rows[i].Detail = p.heldDetailFor(d.Rows[i])
 		}
 	}

@@ -462,12 +529,78 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,

 	d.Groups = groupByModel(d.Rows)

-	if d.Fetching > 0 {
+	// Only on the list: a reload anywhere else takes with it a search, an
+	// open repository or a URL half typed.
+	if d.Fetching > 0 && d.Tab == tabHeld && open == "" {
 		d.RefreshSec = 5
 	}
+	d.Tabs = mirrorTabs(d)
 	render(w, "admin_mirror", d)
 }

+const (
+	tabHeld    = "held"
+	tabExplore = "explore"
+	tabFetch   = "fetch"
+)
+
+// mirrorTab is one of the page's tabs.
+type mirrorTab struct {
+	Label  string
+	Count  string // beside the label, empty for none
+	Href   string
+	Active bool
+}
+
+// mirrorTabFor is the tab a request is for.
+//
+// Named in the address when it is chosen, and otherwise read from what the
+// address carries, so links made before the page had tabs — a search, an
+// open repository — still land where they meant. After a change the list is
+// shown, where its effect is, unless it was refused: then the form it came
+// from, to try again.
+func mirrorTabFor(r *http.Request, errMsg string) string {
+	valid := func(t string) bool { return t == tabHeld || t == tabExplore || t == tabFetch }
+	if r.Method == http.MethodPost {
+		if t := r.PostFormValue("tab"); errMsg != "" && valid(t) {
+			return t
+		}
+		return tabHeld
+	}
+	q := r.URL.Query()
+	if t := q.Get("tab"); valid(t) {
+		return t
+	}
+	for _, k := range []string{"q", "repo", "view", "task", "size", "x"} {
+		if q.Has(k) {
+			return tabExplore
+		}
+	}
+	return tabHeld
+}
+
+func mirrorTabs(d adminMirrorData) []mirrorTab {
+	held := mirrorTab{Label: "Held", Href: "/admin/mirror?tab=held", Active: d.Tab == tabHeld}
+	switch {
+	case d.Unreachable:
+		held.Count = "unreachable"
+	case d.Fetching > 0:
+		held.Count = fmt.Sprintf("%d · %d downloading", d.Present, d.Fetching)
+	default:
+		held.Count = strconv.Itoa(d.Present)
+	}
+	if d.Failed > 0 {
+		held.Count += fmt.Sprintf(" · %d failed", d.Failed)
+	}
+	tabs := []mirrorTab{held}
+	if d.HasRegistry {
+		tabs = append(tabs, mirrorTab{Label: "Explore Hugging Face",
+			Href: "/admin/mirror?tab=explore", Active: d.Tab == tabExplore})
+	}
+	return append(tabs, mirrorTab{Label: "Fetch by URL",
+		Href: "/admin/mirror?tab=fetch", Active: d.Tab == tabFetch})
+}
+
 // mirrorGroup is a family of models the mirror holds — every qwen, every
 // llama — and their versions.
 type mirrorGroup struct {
@@ -663,3 +796,43 @@ func writeJSONStatus(w http.ResponseWriter, code int, v any) {
 	w.WriteHeader(code)
 	_ = json.NewEncoder(w).Encode(v)
 }
+
+// runningOn is the nodes a model's replicas are on, each once and in order,
+// and how many of its replicas are not ready yet.
+func runningOn(reps []adminapi.ReplicaView) (nodes []string, starting int) {
+	for _, r := range reps {
+		if r.State != "ready" {
+			starting++
+		}
+		for _, sh := range r.Shards {
+			if !slices.Contains(nodes, sh.Node) {
+				nodes = append(nodes, sh.Node)
+			}
+		}
+	}
+	slices.Sort(nodes)
+	return nodes, starting
+}
+
+// nodeName is a node as a row names it: short, with the full name beside.
+type nodeName struct{ Short, Full string }
+
+// sharedPrefix is what every node's name starts with, up to and including a
+// separator — "llmgw-" of llmgw-node01 and llmgw-node02 — so a row can say
+// node01 and fit on a line. Nothing when the names share no such part, or
+// when there is one node and dropping it would leave only a number.
+func sharedPrefix(ids []string) string {
+	if len(ids) < 2 {
+		return ""
+	}
+	p := ids[0]
+	for _, id := range ids[1:] {
+		for !strings.HasPrefix(id, p) {
+			p = p[:len(p)-1]
+		}
+	}
+	if i := strings.LastIndexAny(p, "-._"); i >= 0 {
+		return p[:i+1]
+	}
+	return ""
+}
@@ -341,16 +341,18 @@ func TestAFailedRowOffersRetryRatherThanRemove(t *testing.T) {
 }

 type fakeRegistry struct {
-	results []hf.Model
-	repo    hf.Repo
-	err     error
-	asked   string
-	query   hf.Query
+	results  []hf.Model
+	repo     hf.Repo
+	err      error
+	asked    string
+	query    hf.Query
+	explored int
 }

 func (f *fakeRegistry) Explore(_ context.Context, q hf.Query) ([]hf.Model, error) {
 	f.asked = q.Search
 	f.query = q
+	f.explored++
 	return f.results, f.err
 }

@@ -441,7 +443,11 @@ func TestWithNoRegistryThereIsNoSearchBox(t *testing.T) {
 	if strings.Contains(body, "Search Hugging Face") {
 		t.Error("the explorer should be absent without a registry")
 	}
-	if !strings.Contains(body, "Fetch a model") {
+	if strings.Contains(body, "Explore Hugging Face") {
+		t.Error("a tab for an explorer that is not there")
+	}
+	fetch := mirrorGet(t, mirrorPage(t, &fakeMirror{}, true), "/admin/mirror?tab=fetch").Body.String()
+	if !strings.Contains(fetch, `name="source" required`) {
 		t.Error("fetching by hand must still be there")
 	}
 }
@@ -464,8 +470,7 @@ func TestAShardedModelIsOneRowAndOneButton(t *testing.T) {
 		"/admin/mirror?repo=Qwen/Qwen2.5-7B-Instruct").Body.String()

 	// Fifteen shards is one decision, not fifteen. Four here, and one form.
-	if n := strings.Count(body, `action="/admin/mirror/fetch"`); n != 2 {
-		// One for the set, one for the manual form at the bottom.
+	if n := strings.Count(body, `action="/admin/mirror/fetch"`); n != 1 {
 		t.Errorf("expected the four shards as a single form, found %d forms", n)
 	}
 	if !strings.Contains(body, "4 shards, fetched together") {
@@ -534,8 +539,8 @@ func TestAlternativeQuantisationsStaySeparateRows(t *testing.T) {

 	// Two GGUFs are alternatives, not halves. Offering "fetch all" here would
 	// spend gigabytes on a copy nobody asked for.
-	if n := strings.Count(body, `action="/admin/mirror/fetch"`); n != 3 {
-		t.Errorf("expected a form each plus the manual one, found %d", n)
+	if n := strings.Count(body, `action="/admin/mirror/fetch"`); n != 2 {
+		t.Errorf("expected a form each, found %d", n)
 	}
 }

@@ -587,7 +592,7 @@ func TestHowAModelRunsIsChangedWhereItsFileIsShown(t *testing.T) {
 		t.Error("the running model's mode is not shown as chosen")
 	}
 	// Declared in server.yaml: shown, not offered, since it would revert.
-	if !strings.Contains(body, "on demand<br>set in server.yaml") {
+	if !strings.Contains(body, `title="set in server.yaml, which is read again on every start">on demand · yaml</span>`) {
 		t.Error("a declared model should say where its mode is set")
 	}
 	if strings.Count(body, `name="mode" class="mode"`) != 1 {
@@ -655,3 +660,53 @@ func privateOK(m *mirror.Manager) *mirror.Manager {
 	m.AllowPrivateSources()
 	return m
 }
+
+func TestEachTabDoesOnlyItsOwnWork(t *testing.T) {
+	reg := &fakeRegistry{}
+	mux := mirrorPageWith(t, &fakeMirror{models: []mirror.Status{
+		{Model: "m", Version: "v", File: "m.gguf", State: mirror.Present, BytesDone: 1},
+		{Model: "n", Version: "v", File: "n.gguf", State: mirror.Fetching, BytesDone: 1},
+	}}, reg, true)
+
+	held := mirrorGet(t, mux, "/admin/mirror").Body.String()
+	if reg.explored != 0 {
+		t.Error("the list asked the registry for repositories it does not show")
+	}
+	if !strings.Contains(held, `class="on" aria-current="page">Held <span class="count">1 · 1 downloading</span>`) {
+		t.Errorf("the Held tab should be chosen and count what it holds: %s", held)
+	}
+	if strings.Contains(held, `name="source" required`) || strings.Contains(held, "Search Hugging Face") {
+		t.Error("other tabs' forms are on the list")
+	}
+	if !strings.Contains(held, `http-equiv="refresh"`) {
+		t.Error("the list stopped following a download")
+	}
+
+	explore := mirrorGet(t, mux, "/admin/mirror?tab=explore").Body.String()
+	if reg.explored != 1 || !strings.Contains(explore, "Search Hugging Face") {
+		t.Error("the explorer tab did not explore")
+	}
+	if strings.Contains(explore, `id="held"`) || strings.Contains(explore, `http-equiv="refresh"`) {
+		t.Error("the explorer shows the list, or reloads away a search")
+	}
+	// A link from before there were tabs still lands on the explorer.
+	if old := mirrorGet(t, mux, "/admin/mirror?q=qwen").Body.String(); !strings.Contains(old, "Search Hugging Face") {
+		t.Error("a search address no longer opens the explorer")
+	}
+}
+
+func TestARefusedFetchReturnsToItsFormAndAStartedOneToTheList(t *testing.T) {
+	m := &fakeMirror{fetchErr: errors.New("upstream said 404")}
+	mux := mirrorPage(t, m, true)
+	body := mirrorPost(t, mux, "/admin/mirror/fetch", url.Values{"csrf": {"tok"}, "source": {"https://h.example/m.gguf"}, "model": {"m"},
+		"version": {"v"}, "force": {"1"}, "tab": {"fetch"}}).Body.String()
+	if !strings.Contains(body, "upstream said 404") || !strings.Contains(body, `name="source" required`) {
+		t.Errorf("a refusal should come back on the form it came from: %s", body)
+	}
+	m.fetchErr = nil
+	body = mirrorPost(t, mux, "/admin/mirror/fetch", url.Values{"csrf": {"tok"}, "source": {"https://h.example/m.gguf"}, "model": {"m"},
+		"version": {"v"}, "force": {"1"}, "tab": {"fetch"}}).Body.String()
+	if !strings.Contains(body, `id="held"`) {
+		t.Errorf("a started download should be shown where it is followed: %s", body)
+	}
+}
@@ -49,6 +49,9 @@ type serveForm struct {
 	// 3.5 GB for a KV pool. Nothing was wrong with placement — it was never
 	// told what the model needed.
 	MinMemoryGB int
+	// CPUCores is the cores a replica computes on: half the smallest node it
+	// fits on, or zero when no node says how many it has.
+	CPUCores int
 	// MemoryBasis says how MinMemoryGB was arrived at, for the form to show.
 	MemoryBasis string
 	// CtxPool is the engine context pool the form starts with, and the one
@@ -152,6 +155,15 @@ func planServe(st mirror.Status, files []mirror.Status, base string,
 			defaultCtxPool, header.BlockCount, kvHeads,
 			float64(int64(1<<30)+weights/20)/(1<<30))
 	}
+	if fleet != nil {
+		var cpus []int
+		for _, n := range fleet.Nodes() {
+			if n.State == "ready" && (n.MemoryGB == 0 || n.MemoryGB >= f.MinMemoryGB) {
+				cpus = append(cpus, n.CPUCount)
+			}
+		}
+		f.CPUCores = catalog.SuggestCores(cpus)
+	}
 	if biggest := biggestNode(fleet); biggest > 0 && f.MinMemoryGB > biggest {
 		f.Warning = strings.TrimSuffix(f.Warning, ".") +
 			fmt.Sprintf("%sthe largest node has %d GB and this needs about %d",
@@ -277,6 +289,11 @@ func buildModel(get func(string) string) (catalog.Model, error) {
 			return m, err
 		}
 	}
+	if v := strings.TrimSpace(get("cpu_cores")); v != "" {
+		if p.CPUCores, err = positive(v, "the cores"); err != nil {
+			return m, err
+		}
+	}
 	if k, v := strings.TrimSpace(get("selector_key")), strings.TrimSpace(get("selector_value")); k != "" && v != "" {
 		// What keeps a large model off the small nodes. A CPU profile has no
 		// card to be short of, so without this every ready node looks like a
@@ -45,6 +45,13 @@ type DashboardData struct {

 	HasPeople bool

+	// NodePrefix is what every node's name starts with, left off in tables
+	// so a row fits on a line; the full name is on hover.
+	NodePrefix string
+	// Columns shown only when some row has something in them: a CPU-only
+	// cluster's nodes table was three columns of dashes wide.
+	AnyGPUs, AnyClasses, AnyFabrics, AnyFallbacks bool
+
 	// Charts are pre-rendered rather than assembled in the template: building
 	// SVG out of template fragments is how a chart ends up unescaped.
 	RequestChart template.HTML
@@ -177,6 +184,17 @@ func Dashboard(src Source, usage UsageSource, version string, refresh time.Durat
 				}
 			}
 			d.Unplaced = unplaced(src, d.Models)
+			ids := make([]string, 0, len(d.Nodes))
+			for _, n := range d.Nodes {
+				ids = append(ids, n.ID)
+				d.AnyGPUs = d.AnyGPUs || n.GPUTotal > 0
+				d.AnyClasses = d.AnyClasses || len(n.Classes) > 0
+				d.AnyFabrics = d.AnyFabrics || len(n.Fabrics) > 0
+			}
+			d.NodePrefix = sharedPrefix(ids)
+			for _, m := range d.Models {
+				d.AnyFallbacks = d.AnyFallbacks || len(m.Fallbacks) > 0
+			}
 			d.Summary = src.Summary()
 			if fs, ok := src.(FabricSource); ok {
 				r := fs.Fabrics()
@@ -0,0 +1,207 @@
+package web
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"net/http"
+	"strconv"
+	"strings"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/store"
+)
+
+// HeadroomSource is a fleet that can say how many more replicas of a model
+// fit now. Optional: without it the card shows what runs and offers no change
+// it cannot check.
+type HeadroomSource interface {
+	Headroom(model string, limit int) (adminapi.HeadroomView, bool)
+}
+
+// maxReplicas bounds what the form accepts. Not a cluster limit, a typing one:
+// 300 where 3 was meant would otherwise be taken as a ceiling.
+const maxReplicas = 64
+
+// heldDetail is the card opened beneath a held version: what runs it, where,
+// and room to change it.
+type heldDetail struct {
+	Models []heldModel
+}
+
+// heldModel is one catalog model running a held version.
+type heldModel struct {
+	modelUse
+	View     adminapi.ModelView
+	Replicas []heldReplica
+	Headroom *adminapi.HeadroomView
+	// MostNow is how many replicas could always run, Current plus the room
+	// found; zero when the room was not measured or had no end in sight.
+	MostNow int
+}
+
+// heldReplica is one replica, by the nodes it runs on.
+type heldReplica struct {
+	ID       string
+	State    string
+	Restart  string
+	Age      int64
+	Endpoint string
+	Nodes    []heldShard
+}
+
+type heldShard struct {
+	Node     string
+	MemoryGB int // the node's RAM, 0 when unknown
+	CPU      float64
+	MemBytes int64
+}
+
+// openKey names a row the way ?open= does.
+func openKey(model, version string) string { return model + ":" + version }
+
+// heldDetailFor reads, for each model using a row, how it runs and how much
+// more room it has.
+func (p *AccountPages) heldDetailFor(row mirrorRow) *heldDetail {
+	d := &heldDetail{}
+	if p.fleet == nil {
+		for _, u := range row.UsedBy {
+			d.Models = append(d.Models, heldModel{modelUse: u})
+		}
+		return d
+	}
+	views := map[string]adminapi.ModelView{}
+	for _, v := range p.fleet.Models() {
+		views[v.Name] = v
+	}
+	memory := map[string]int{}
+	for _, n := range p.fleet.Nodes() {
+		memory[n.ID] = n.MemoryGB
+	}
+	hs, _ := p.fleet.(HeadroomSource)
+	for _, u := range row.UsedBy {
+		hm := heldModel{modelUse: u, View: views[u.Name]}
+		for _, rep := range p.fleet.Replicas(u.Name) {
+			// Built field by field: a ReplicaView carries the engine's key,
+			// which is for the admin listener and never for a page.
+			hr := heldReplica{ID: rep.ID, State: rep.State, Restart: rep.Restart,
+				Age: rep.AgeSec, Endpoint: rep.Endpoint}
+			for _, sh := range rep.Shards {
+				hr.Nodes = append(hr.Nodes, heldShard{Node: sh.Node, MemoryGB: memory[sh.Node],
+					CPU: sh.CPUCores, MemBytes: sh.MemoryBytes})
+			}
+			hm.Replicas = append(hm.Replicas, hr)
+		}
+		if hs != nil {
+			if h, ok := hs.Headroom(u.Name, 16); ok {
+				hm.Headroom = &h
+				if !h.Limited {
+					hm.MostNow = h.Current + h.More
+				}
+			}
+		}
+		d.Models = append(d.Models, hm)
+	}
+	return d
+}
+
+// setReplicas changes how many replicas a model always runs and how far it
+// may grow, refusing an always-on count the cluster has no room for.
+//
+// Refused rather than taken and left short: a minimum placement cannot meet
+// is a model reported as unplaceable on every pass from then on, which is the
+// same answer given later and less plainly. A maximum above the room is
+// accepted and said: it is reached only under load, by which time something
+// may have made room, and refusing it would make the ceiling depend on what
+// happened to be running when it was set.
+func (p *AccountPages) setReplicas(ctx context.Context, admin store.User, r *http.Request) (string, error) {
+	name := r.FormValue("model")
+	lo, err1 := strconv.Atoi(strings.TrimSpace(r.FormValue("min")))
+	hi, err2 := strconv.Atoi(strings.TrimSpace(r.FormValue("max")))
+	switch {
+	case name == "":
+		return "", errors.New("which model?")
+	case err1 != nil || err2 != nil:
+		return "", errors.New("replicas are whole numbers")
+	case lo < 0 || hi < lo:
+		return "", fmt.Errorf("always running (%d) cannot be more than the most under load (%d)", lo, hi)
+	case hi > maxReplicas:
+		return "", fmt.Errorf("at most %d replicas", maxReplicas)
+	}
+	current, err := p.currentModel(ctx, name)
+	if err != nil {
+		return "", err
+	}
+	if p.catalog.Declared(name) {
+		return "", fmt.Errorf("%s is set in server.yaml; change it there", name)
+	}
+
+	note := ""
+	if hs, ok := p.fleet.(HeadroomSource); ok && hi > 0 {
+		h, ok := hs.Headroom(name, hi)
+		if !ok {
+			return "", fmt.Errorf("no model %q", name)
+		}
+		fits := h.Current + h.More
+		if lo > fits && !h.Limited {
+			why := "nothing more fits"
+			if len(h.Blocked) > 0 {
+				why = strings.Join(h.Blocked, "; ")
+			}
+			return "", fmt.Errorf("%s can always run at most %d now (%d running, room for %d more): %s",
+				name, fits, h.Current, h.More, why)
+		}
+		if hi > fits && !h.Limited {
+			note = fmt.Sprintf("; there is room for %d now, so it grows past that only if room is made", fits)
+		}
+	}
+
+	sc := catalog.Scaling{Min: lo, Max: hi, IdleTimeout: current.Scaling.IdleTimeout}
+	if sc.OnDemand() && sc.IdleTimeout <= 0 {
+		sc.IdleTimeout = defaultIdle
+	}
+	if _, err := p.catalog.Scale(ctx, name, sc, admin.Email); err != nil {
+		return "", err
+	}
+	switch {
+	case hi == 0:
+		return name + " is stopped; its replicas are being taken down", nil
+	case lo == 0:
+		return fmt.Sprintf("%s loads when asked for, up to %d under load%s", name, hi, note), nil
+	case lo == hi:
+		return fmt.Sprintf("%s runs %s", name, plural(lo, "replica", "replicas")), nil
+	}
+	return fmt.Sprintf("%s runs %d, and up to %d under load%s", name, lo, hi, note), nil
+}
+
+// currentModel is a model as the catalog holds it now.
+func (p *AccountPages) currentModel(ctx context.Context, name string) (catalog.Model, error) {
+	sources, err := p.catalog.Sources(ctx)
+	if err != nil {
+		return catalog.Model{}, err
+	}
+	for _, ms := range sources {
+		for _, m := range ms {
+			if m.Name == name {
+				return m, nil
+			}
+		}
+	}
+	return catalog.Model{}, fmt.Errorf("no model %q runs from the mirror", name)
+}
+
+// age renders seconds the way the replicas table does.
+func age(sec int64) string {
+	d := time.Duration(sec) * time.Second
+	switch {
+	case d < time.Minute:
+		return fmt.Sprintf("%ds", sec)
+	case d < time.Hour:
+		return fmt.Sprintf("%dm", int(d.Minutes()))
+	case d < 48*time.Hour:
+		return fmt.Sprintf("%dh%02dm", int(d.Hours()), int(d.Minutes())%60)
+	}
+	return fmt.Sprintf("%dd", int(d.Hours()/24))
+}
@@ -0,0 +1,183 @@
+package web
+
+import (
+	"net/url"
+	"strings"
+	"testing"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
+	"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
+)
+
+// roomyFleet runs one replica of "coder" on n2 and has room for `more`.
+type roomyFleet struct {
+	fakeFleet
+	more    int
+	blocked []string
+	asked   int
+}
+
+func (f *roomyFleet) Nodes() []adminapi.NodeView {
+	return []adminapi.NodeView{{ID: "n1", MemoryGB: 16}, {ID: "n2", MemoryGB: 32}}
+}
+func (f *roomyFleet) Models() []adminapi.ModelView {
+	return []adminapi.ModelView{{Name: "coder", Min: 1, Max: 1, Ready: 1, Total: 1, Desired: 1}}
+}
+func (f *roomyFleet) Replicas(string) []adminapi.ReplicaView {
+	return []adminapi.ReplicaView{{ID: "coder-abc", Model: "coder", State: "ready", AgeSec: 3700,
+		EngineKey: "secret-engine-key",
+		Shards:    []adminapi.ShardView{{Node: "n2", CPUCores: 3.5, MemoryBytes: 6 << 30}}}}
+}
+func (f *roomyFleet) Headroom(model string, limit int) (adminapi.HeadroomView, bool) {
+	f.asked = limit
+	more := min(f.more, limit)
+	h := adminapi.HeadroomView{Model: model, Current: 1, More: more}
+	for range more {
+		h.Nodes = append(h.Nodes, []string{"n1"})
+	}
+	if more < limit {
+		h.Blocked = f.blocked
+	} else {
+		h.Limited = true
+	}
+	return h, true
+}
+
+func detailPage(t *testing.T, fleet *roomyFleet, cat *fakeCatalog) func(string) string {
+	t.Helper()
+	m := &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 5 << 30},
+	}}
+	mux := mirrorPageDeps(t, MirrorDeps{Mirror: m, Catalog: cat, Fleet: fleet}, true)
+	return func(path string) string { return mirrorGet(t, mux, path).Body.String() }
+}
+
+func coderCatalog() *fakeCatalog {
+	return &fakeCatalog{sources: map[string][]catalog.Model{"http://mirror:8081/coder/v1/c.gguf": {
+		{Name: "coder", Scaling: catalog.Scaling{Min: 1, Max: 1}}}}}
+}
+
+func TestAHeldModelOpensToWhereItRunsAndHowMuchRoomIsLeft(t *testing.T) {
+	fleet := &roomyFleet{more: 2, blocked: []string{"n2: 2 GB free of 32, profile needs 6"}}
+	get := detailPage(t, fleet, coderCatalog())
+
+	closed := get("/admin/mirror")
+	if strings.Contains(closed, "held-card") {
+		t.Error("a card is open before anything was clicked")
+	}
+	if !strings.Contains(closed, `href="/admin/mirror?tab=held&amp;open=coder%3av1"`) {
+		t.Error("the model's name does not open its card")
+	}
+
+	body := get("/admin/mirror?tab=held&open=coder:v1")
+	for _, want := range []string{
+		"held-card", `<a href="/admin/nodes/n2">n2</a>`, "32 GB", "coder-abc", "1h01m", "3.5 cores", "6.0 GiB",
+		"Room for 2 more now, on n1, n1.", "n2: 2 GB free of 32, profile needs 6",
+		`name="min" value="1" min="0" max="3"`, `action="/admin/mirror/replicas"`,
+		`name="open" value="coder:v1"`,
+	} {
+		if !strings.Contains(body, want) {
+			t.Errorf("the card is missing %q", want)
+		}
+	}
+	if strings.Contains(body, "secret-engine-key") {
+		t.Error("the engine's key reached the page")
+	}
+	if strings.Contains(body, `http-equiv="refresh"`) {
+		t.Error("a page with an open form reloads under the operator")
+	}
+}
+
+func TestAReplicaCountTheClusterHasNoRoomForIsRefused(t *testing.T) {
+	fleet := &roomyFleet{more: 1, blocked: []string{"n1: 1 GB free of 16, profile needs 6"}}
+	cat := coderCatalog()
+	mux := mirrorPageDeps(t, MirrorDeps{Mirror: &fakeMirror{models: []mirror.Status{
+		{Model: "coder", Version: "v1", File: "c.gguf", State: mirror.Present, BytesDone: 1},
+	}}, Catalog: cat, Fleet: fleet}, true)
+	post := func(lo, hi string) string {
+		return mirrorPost(t, mux, "/admin/mirror/replicas", url.Values{"csrf": {"tok"},
+			"model": {"coder"}, "min": {lo}, "max": {hi}, "open": {"coder:v1"}}).Body.String()
+	}
+
+	body := post("3", "3")
+	if len(cat.scaled) != 0 {
+		t.Fatalf("three always-on replicas were set with room for two: %+v", cat.lastScale)
+	}
+	if !strings.Contains(body, "can always run at most 2 now") || !strings.Contains(body, "1 GB free of 16") {
+		t.Errorf("the refusal should say how many fit and why: %s", body)
+	}
+	if !strings.Contains(body, "held-card") {
+		t.Error("the card closed on the refusal")
+	}
+
+	body = post("2", "4")
+	if len(cat.scaled) != 1 || cat.lastScale.Min != 2 || cat.lastScale.Max != 4 {
+		t.Fatalf("what fits was not set: %+v", cat.lastScale)
+	}
+	if !strings.Contains(body, "coder runs 2, and up to 4 under load; there is room for 2 now") {
+		t.Errorf("a ceiling above the room should be taken and said: %s", body)
+	}
+
+	post("0", "2")
+	if cat.lastScale.Min != 0 || cat.lastScale.Max != 2 || cat.lastScale.IdleTimeout == 0 {
+		t.Errorf("on demand was set as %+v", cat.lastScale)
+	}
+	for _, bad := range [][2]string{{"3", "2"}, {"-1", "1"}, {"x", "1"}, {"1", "100"}} {
+		n := len(cat.scaled)
+		post(bad[0], bad[1])
+		if len(cat.scaled) != n {
+			t.Errorf("min %s max %s was accepted", bad[0], bad[1])
+		}
+	}
+}
+
+func TestAModelSetInConfigurationIsShownButNotChanged(t *testing.T) {
+	cat := coderCatalog()
+	cat.declared = "coder"
+	body := detailPage(t, &roomyFleet{more: 20}, cat)("/admin/mirror?open=coder:v1")
+	if strings.Contains(body, `action="/admin/mirror/replicas"`) || !strings.Contains(body, "change its replicas there") {
+		t.Error("a declared model offered a change that would revert")
+	}
+	if !strings.Contains(body, "Room for at least 16 more") {
+		t.Error("room that reached the limit asked should read as at least")
+	}
+}
+
+func TestAHeldRowSaysHowItsModelRunsWithoutOpening(t *testing.T) {
+	body := detailPage(t, &roomyFleet{more: 1}, coderCatalog())("/admin/mirror")
+	if strings.Contains(body, "held-card") {
+		t.Fatal("the card opened by itself")
+	}
+	if !strings.Contains(body, `title="replicas ready / wanted">1/1 · <span class="node" title="n2">n2</span></span>`) {
+		t.Errorf("the row does not say where its replicas run: %s", body)
+	}
+}
+
+func TestRunningOnNamesEachNodeOnceAndCountsWhatIsStarting(t *testing.T) {
+	nodes, starting := runningOn([]adminapi.ReplicaView{
+		{State: "ready", Shards: []adminapi.ShardView{{Node: "n2"}}},
+		{State: "starting", Shards: []adminapi.ShardView{{Node: "n1"}}},
+		{State: "ready", Shards: []adminapi.ShardView{{Node: "n2"}}},
+	})
+	if strings.Join(nodes, ",") != "n1,n2" || starting != 1 {
+		t.Errorf("nodes %v, starting %d", nodes, starting)
+	}
+}
+
+func TestNodeNamesLoseOnlyWhatEveryNodeShares(t *testing.T) {
+	for _, c := range []struct {
+		ids  []string
+		want string
+	}{
+		{[]string{"llmgw-node01", "llmgw-node02", "llmgw-node06"}, "llmgw-"},
+		{[]string{"gpu-a.dc1", "gpu-b.dc1"}, "gpu-"},
+		{[]string{"n1", "n2"}, ""},
+		{[]string{"llmgw-node01"}, ""},
+		{[]string{"alpha", "beta"}, ""},
+	} {
+		if got := sharedPrefix(c.ids); got != c.want {
+			t.Errorf("%v: %q, expected %q", c.ids, got, c.want)
+		}
+	}
+}
@@ -155,7 +155,7 @@ func oneOf(s string, options ...string) bool {

 // href is the page with these settings and one of them changed.
 func (e explorer) href(query, key, value string) template.URL {
-	v := url.Values{"view": {e.View}, "task": {e.Task}, "size": {e.Size}, "x": {"1"}}
+	v := url.Values{"tab": {tabExplore}, "view": {e.View}, "task": {e.Task}, "size": {e.Size}, "x": {"1"}}
 	if e.Here {
 		v.Set("here", "1")
 	}
@@ -27,7 +27,7 @@ func TestTheExplorerOpensOnTrendingModelsTheNodesCanRun(t *testing.T) {
 		ggufModel("odd/Kev-GGUF", "ggmlc", 5e8, now.Add(-2*24*time.Hour)),
 	}}
 	body := mirrorGet(t, mirrorPageLlama(t, MirrorDeps{Mirror: &fakeMirror{}, Registry: reg}),
-		"/admin/mirror").Body.String()
+		"/admin/mirror?tab=explore").Body.String()

 	q := reg.query
 	if q.Sort != hf.SortTrending || q.Task != "text-generation" || !q.GGUF || q.Search != "" {
@@ -96,7 +96,7 @@ func TestAnAdapterIsNotListedAsAModel(t *testing.T) {
 		ggufModel("a/Tiny-GGUF", "qwen2", 135e6, time.Now()),
 	}}
 	body := mirrorGet(t, mirrorPageLlama(t, MirrorDeps{Mirror: &fakeMirror{}, Registry: reg}),
-		"/admin/mirror").Body.String()
+		"/admin/mirror?tab=explore").Body.String()
 	if strings.Contains(body, "Some-LoRA") || !strings.Contains(body, "a/Tiny-GGUF") {
 		t.Errorf("adapters should be left out and small models kept: %s", body)
 	}
@@ -0,0 +1,54 @@
+package web
+
+import (
+	"strings"
+	"testing"
+	"time"
+
+	"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
+)
+
+// cpuFleet is a CPU-only cluster whose nodes share a name prefix.
+type cpuFleet struct{ fakeSource }
+
+func (cpuFleet) Nodes() []adminapi.NodeView {
+	return []adminapi.NodeView{
+		{ID: "llmgw-node01", State: "ready", MgmtAddr: "10.0.0.141", MemoryGB: 8},
+		{ID: "llmgw-node02", State: "ready", MgmtAddr: "10.0.0.142", MemoryGB: 8},
+	}
+}
+func (cpuFleet) Summary() adminapi.Summary { return adminapi.Summary{} }
+func (cpuFleet) Models() []adminapi.ModelView {
+	return []adminapi.ModelView{{Name: "qwen", Min: 1, Max: 1, Ready: 1,
+		Description: "A sentence long enough that it once had a row of its own under the model."}}
+}
+func (cpuFleet) Replicas(string) []adminapi.ReplicaView {
+	return []adminapi.ReplicaView{{ID: "qwen-1", Model: "qwen", State: "ready",
+		Shards: []adminapi.ShardView{{Node: "llmgw-node02"}}}}
+}
+
+func TestTheClusterPageKeepsEachRecordOnOneLine(t *testing.T) {
+	h := Dashboard(cpuFleet{}, nil, "v1", time.Second, true, NavExtras{}, nil)
+	_, body := get(t, h, "http://127.0.0.1:9090/", nil)
+
+	for _, gone := range []string{"<th>Classes</th>", "<th>Fabrics</th>", `<th class="num">GPUs</th>`, "<th>Fallbacks</th>"} {
+		if strings.Contains(body, gone) {
+			t.Errorf("%s is shown with nothing in it on any row", gone)
+		}
+	}
+	if !strings.Contains(body, `title="llmgw-node01"><code>node01</code>`) {
+		t.Error("node names should drop what every node shares, keeping the whole on hover")
+	}
+	if !strings.Contains(body, `<code title="llmgw-node02">node02</code>`) {
+		t.Error("the replicas table should name nodes the same way")
+	}
+	if strings.Contains(body, `class="url-row"`) || !strings.Contains(body, `<td class="clip" title="A sentence long enough`) {
+		t.Error("a model's description should be a clipped cell, not a row of its own")
+	}
+
+	// A cluster with GPUs still gets its GPU column.
+	_, gpu := get(t, Dashboard(fakeSource{}, nil, "v1", time.Second, true, NavExtras{}, nil), "http://127.0.0.1:9090/", nil)
+	if !strings.Contains(gpu, `<th class="num">GPUs</th>`) || !strings.Contains(gpu, "<th>Fabrics</th>") {
+		t.Error("columns with something in them were hidden")
+	}
+}
@@ -60,7 +60,7 @@ li { margin: .25rem 0; }
 a { color: var(--accent); }
 .wrap.narrow { max-width: 30rem; }
 label { display: block; margin: .9rem 0; font-size: .9rem; font-weight: 600; }
-input[type=text], input[type=email], input[type=password], input[type=search] {
+input[type=text], input[type=email], input[type=password], input[type=search], input[type=number] {
  display: block; width: 100%; margin-top: .3rem; font: inherit; font-weight: 400;
  padding: .5rem .6rem; border: 1px solid var(--line); border-radius: 6px;
  background: var(--card); color: var(--fg);
@@ -444,18 +444,29 @@ h3.limits-for { font-size: .92rem; margin: 1rem 0 .4rem; }
 table.narrow-table { max-width: 30rem; }
 .q-act a + a { margin-left: .8rem; }
 .chip-role { font-size: .68rem; font-weight: 600; color: var(--accent); text-transform: uppercase; letter-spacing: .04em; }
-/* Runs: one fixed-width selector per model, so they line up down the column
-   whatever the names; a catalog name differing from the mirror's sits above. */
-table.held .use { width: 9.5rem; margin: .1rem 0; }
-table.held .use .as { display: block; font-size: .72rem; color: var(--muted);
-                      overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
-table.held .use .opt { font-size: .75rem; line-height: 1.3; }
+/* Runs: one line per model — a catalog name that differs from the mirror's,
+   how it runs, and where — with the selector at a fixed width so the
+   replica counts line up down the column. */
+table.held .use { display: flex; align-items: center; gap: .5rem; margin: .1rem 0; white-space: nowrap; }
+table.held .use .as { font-size: .72rem; color: var(--muted); max-width: 10rem;
+                      overflow: hidden; text-overflow: ellipsis; }
+table.held .use form.inline { margin: 0; }
+table.held .use .mode-fixed { width: 7.5rem; font-size: .8rem; color: var(--muted); }
 table.held td.num { white-space: nowrap; }
-table.held select.mode { width: 100%; padding: .15rem .3rem; font-size: .8rem; }
+table.held select.mode { width: 7.5rem; margin: 0; padding: .1rem .3rem; font-size: .8rem; }

 /* Link pills: the same look as .tabs, for choices that are addresses rather
   than panels on one page — a filtered listing should survive a reload and be
   worth sending to somebody. */
+/* A page's own sections, one address each. Underlined rather than pills so
+ * they do not read as one more filter beside the ones inside a section. */
+.page-tabs { display: flex; flex-wrap: wrap; gap: 0 1.5rem; margin: -1rem 0 1.5rem;
+             border-bottom: 1px solid var(--line); }
+.page-tabs a { padding: .55rem 0; margin-bottom: -1px; font-weight: 600; color: var(--muted);
+               text-decoration: none; border-bottom: 2px solid transparent; }
+.page-tabs a:hover { color: var(--fg); }
+.page-tabs a.on { color: var(--fg); border-bottom-color: var(--accent); }
+.page-tabs .count { font-weight: 500; font-size: .8rem; color: var(--muted); margin-left: .25rem; }
 .pills { display: flex; flex-wrap: wrap; gap: .35rem; margin: .5rem 0; }
 .pills a { font-size: .85rem; font-weight: 600; color: var(--muted); padding: .35rem .85rem;
           border: 1px solid transparent; border-radius: 999px; text-decoration: none; }
@@ -804,3 +815,43 @@ details.chat-settings output { font-variant-numeric: tabular-nums; font-weight:
 .copyable button { margin-right: .75rem; font-size: .8rem; }
 .connect-form .field-row label { flex: 1 1 12rem; }
 table.connect-fields th { white-space: nowrap; }
+
+/* The card opened beneath a held version on the mirror page. */
+tr.held-detail > td { background: var(--bg); padding: .75rem 1rem 1rem; }
+.held-card .held-model { margin-top: .75rem; padding-top: .75rem; border-top: 1px solid var(--line); }
+.held-card .held-model:first-of-type { border-top: 0; padding-top: 0; }
+.held-card h3 { font-size: 1rem; margin: 0 0 .5rem; display: flex; gap: .6rem; align-items: baseline; flex-wrap: wrap; }
+.held-card table.replicas { margin: .25rem 0 .5rem; }
+.held-card .headroom { margin: .5rem 0 .25rem; font-weight: 600; }
+.held-card details.blocked { font-size: .85rem; color: var(--muted); margin-bottom: .5rem; }
+.held-card details.blocked ul { margin: .35rem 0 0 1.1rem; padding: 0; }
+.held-card .replicas-form .field-row { align-items: flex-end; }
+.held-card .replicas-form .field-row label { flex: 0 0 auto; }
+.held-card .replicas-form input[type=number] { width: 8rem; margin: .25rem 0 0; }
+.held-card .replicas-form button { margin: 0; }
+a.open-held { text-decoration: none; color: inherit; }
+a.open-held:hover strong { text-decoration: underline; }
+a.open-held[aria-expanded=true]::before { content: "▾ "; color: var(--muted); }
+a.open-held[aria-expanded=false]::before { content: "▸ "; color: var(--muted); }
+.use .runs-now { font-size: .78rem; color: var(--muted); }
+.use .runs-now.short { color: var(--warn); }
+.use .runs-now.bad { color: var(--bad); }
+.use .runs-now .node { white-space: nowrap; }
+
+/* One line per record, in every table. A header never wraps; a cell of free
+ * text is cut to its column with the whole of it on hover, instead of a row of
+ * its own or a line break that makes one record look like two. */
+th { white-space: nowrap; }
+table.rows td { white-space: nowrap; }
+td.clip { max-width: 22rem; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
+table.rows td.clip { color: var(--muted); font-size: .85rem; }
+/* A row across the whole table is a note or an open form, not a record: it
+ * wraps as prose does. */
+table.rows td[colspan] { white-space: normal; }
+table.rows td.where { max-width: none; }
+table.rows .more { cursor: help; margin-left: .2rem; }
+table.explore td:first-child { max-width: 34rem; overflow: hidden; text-overflow: ellipsis; }
+table.explore .from { margin-left: .5rem; font-size: .78rem; color: var(--muted); }
+.clip-inline { display: inline-block; max-width: 24rem; overflow: hidden; text-overflow: ellipsis;
+               vertical-align: bottom; font-size: .8rem; color: var(--muted); margin-left: .4rem; }
+table.rows td.warn { color: var(--warn); cursor: help; }
@@ -84,7 +84,7 @@ Tell whoever you asked for access that you have signed up.</p>
 {{end}}
 {{if .Fields}}
 <h3>Settings to fill in</h3>
-<table class="compact narrow-table connect-fields">{{range .Fields}}
+<table class="rows compact narrow-table connect-fields">{{range .Fields}}
  <tr><th>{{.Label}}</th><td><code>{{.Value}}</code></td><td><button type="button" class="link copy" data-value="{{.Value}}">copy</button></td></tr>{{end}}
 </table>
 {{end}}
@@ -149,7 +149,7 @@ document.addEventListener("click", (e) => {

 <h2>Keys</h2>
 {{if .Keys}}
-<table><tr><th>Name</th><th>Key</th><th>Spends against</th><th>Created</th><th>Last used</th><th>State</th><th></th></tr>
+<table class="rows"><tr><th>Name</th><th>Key</th><th>Spends against</th><th>Created</th><th>Last used</th><th>State</th><th></th></tr>
 {{range .Keys}}<tr>
  <td>{{.Name}}</td>
  <td><code>{{.Prefix}}</code></td>
@@ -192,7 +192,7 @@ old one to stop moving, then revoke it.</p>
 </div>
 {{end}}
 {{if .Rows}}
-<table class="compact narrow-table"><tr><th>Limit</th><th class="num">Value</th></tr>
+<table class="rows compact narrow-table"><tr><th>Limit</th><th class="num">Value</th></tr>
 {{range .Rows}}<tr><td>{{.Name}}</td><td class="num">{{.Value}}</td></tr>{{end}}</table>
 {{else}}<p class="empty">No limits here.</p>{{end}}
 {{else}}<p class="empty">Nowhere to spend yet.</p>{{end}}
@@ -8,16 +8,20 @@
 </head><body><div class="wrap">
 <header>
  <h1>Model mirror</h1>
-  <p class="sub">
-    {{if .Unreachable}}{{.Addr}} did not answer{{else}}{{.Present}} held ·
-    {{.Fetching}} downloading · {{.Failed}} failed · {{.Addr}}{{end}}
-  </p>
+  <p class="sub">{{.Addr}}{{if .Unreachable}} did not answer{{end}}</p>
  {{template "nav" .Nav}}
 </header>

+{{/* Links rather than a script's panels: each tab has an address, the back
+     button works, and only the one shown asks anything of the registry. */}}
+<nav class="page-tabs" aria-label="Mirror">
+{{range .Tabs}}<a href="{{.Href}}"{{if .Active}} class="on" aria-current="page"{{end}}>{{.Label}}{{with .Count}} <span class="count">{{.}}</span>{{end}}</a>
+{{end}}</nav>
+
 {{if .Error}}<p class="error">{{.Error}}</p>{{end}}
 {{if .Message}}<p class="ok">{{.Message}}</p>{{end}}

+{{if eq .Tab "held"}}
 {{if .Unreachable}}
 {{/* Said outright rather than shown as an empty table. A mirror holding
     nothing and a mirror that is not there look the same in a list of zero
@@ -29,9 +33,8 @@ reachable from this machine.</p>
 {{else}}

 <div id="held">
-<h2>What the mirror holds</h2>
 {{if .Rows}}
-<table class="compact held">
+<table class="rows compact held">
 <tr><th>Model</th><th>Files</th><th class="num">Size</th><th>Runs</th><th></th></tr>
 {{range .Groups}}{{$grouped := gt .Models 1}}
 {{if $grouped}}
@@ -41,8 +44,10 @@ reachable from this machine.</p>
  {{if .Failed}}<span class="tag bad">failed</span>{{end}}
  {{if .InUse}}<span class="tag">in use</span>{{end}}</td></tr>
 {{end}}
-{{range .Rows}}<tr{{if $grouped}} class="model-version"{{end}}>
-  <td><strong>{{.Model}}</strong> <span class="opt">{{.Version}}</span></td>
+{{range .Rows}}{{$row0 := .}}<tr{{if $grouped}} class="model-version"{{end}}{{if .Detail}} class="open"{{end}}>
+  <td>{{if eq (printf "%s" .State) "present"}}<a class="open-held" aria-expanded="{{if .Detail}}true{{else}}false{{end}}"
+       href="/admin/mirror?tab=held{{if not .Detail}}&amp;open={{.Open}}{{end}}"><strong>{{.Model}}</strong></a>
+    {{else}}<strong>{{.Model}}</strong>{{end}} <span class="opt">{{.Version}}</span></td>
  <td class="file" title="{{.File}}{{if .SHA256}} · sha256 {{.SHA256}}{{end}}">
    {{if gt .Files 1}}{{.Files}} files{{else}}<code>{{.File}}</code>{{end}}</td>
  <td class="num">
@@ -62,7 +67,8 @@ reachable from this machine.</p>
      {{/* Named only when the catalog calls it something other than the
           mirror does; otherwise it is the model column again. */}}
      {{if ne .Name $row.Model}}<span class="as" title="{{.Name}}">as {{.Name}}</span>{{end}}
-      {{if .Declared}}<span class="opt">{{if eq .Mode "always"}}always on{{else if eq .Mode "ondemand"}}on demand{{else}}stopped{{end}}<br>set in server.yaml</span>
+      {{if .Declared}}<span class="mode-fixed" title="set in server.yaml, which is read again on every start">
+        {{- if eq .Mode "always"}}always on{{else if eq .Mode "ondemand"}}on demand{{else}}stopped{{end}} · yaml</span>
      {{else}}
      {{/* Submitted on change: the choice is the action, and a second click
           to confirm it would only be a way to forget to. */}}
@@ -78,6 +84,12 @@ reachable from this machine.</p>
        <noscript><button type="submit" class="link">set</button></noscript>
      </form>
      {{end}}
+      {{if .Fleet}}<span class="runs-now{{if or .Parked .Unplaceable}} bad{{else if lt .Ready .Wanted}} short{{end}}" title="replicas ready / wanted{{if .Starting}}, + starting{{end}}">
+        {{- if .Parked}}parked
+        {{- else if and (eq .Wanted 0) (eq .Ready 0)}}none running
+        {{- else}}{{.Ready}}/{{.Wanted}}{{if .Starting}} +{{.Starting}}{{end}}{{end}}
+        {{- if .Unplaceable}} · no room{{end}}
+        {{- with .Nodes}} · {{range $i, $n := .}}{{if $i}}, {{end}}<span class="node" title="{{$n.Full}}">{{$n.Short}}</span>{{end}}{{end}}</span>{{end}}
    </div>{{end}}
    {{else}}<span class="opt">not served</span>{{end}}
  </td>
@@ -109,6 +121,7 @@ reachable from this machine.</p>
  </td>
 </tr>
 {{if .Err}}<tr><td colspan="5" class="error-row">{{.Err}}</td></tr>{{end}}
+{{with .Detail}}<tr class="held-detail"><td colspan="5">{{template "mirror_held_detail" dict "D" . "Row" $row0 "CSRF" $.CSRF}}</td></tr>{{end}}

 {{with .Serve}}
 <tr class="editing"><td colspan="5">
@@ -152,7 +165,9 @@ reachable from this machine.</p>
  <div class="field-row">
    <label>Profile name <input type="text" name="profile" value="{{.Profile}}"></label>
    <label>Needs RAM (GB) <span class="opt">for the context pool beside it</span>
-      <input type="text" name="min_memory_gb" value="{{.MinMemoryGB}}"></label>
+      <input type="text" name="min_memory_gb" value="{{.MinMemoryGB}}" required></label>
+    <label>CPU cores <span class="opt">reserved; the engine's threads</span>
+      <input type="text" name="cpu_cores" value="{{.CPUCores}}"></label>
    <label>Only nodes labelled <span class="opt">key / value, optional</span>
      <input type="text" name="selector_key" placeholder="role" size="8">
      <input type="text" name="selector_value" placeholder="cpu-large" size="10"></label>
@@ -190,11 +205,14 @@ document.addEventListener('click', function (e) {
  } else { window.prompt('Address', b.dataset.url); }
 });
 </script>
-{{else}}<p class="empty">The mirror is reachable and holds nothing yet.</p>{{end}}
+{{else}}<p class="empty">The mirror is reachable and holds nothing yet.
+{{if .HasRegistry}}<a href="/admin/mirror?tab=explore">Explore Hugging Face</a> for a model, or
+{{end}}<a href="/admin/mirror?tab=fetch">fetch one by URL</a>.</p>{{end}}
 </div>
+{{end}}
+{{end}}

-{{if .HasRegistry}}
-<h2>Explore models</h2>
+{{if and (eq .Tab "explore") .HasRegistry}}
 {{if not .Repo}}{{with .Explore}}
 <nav class="pills" aria-label="Which models">{{range .Views}}<a href="{{.Href}}"{{if .Active}} class="on" aria-current="page"{{end}}>{{.Label}}</a>{{end}}</nav>
 <div class="explore-filters">
@@ -203,6 +221,7 @@ document.addEventListener('click', function (e) {
 </div>
 {{end}}
 <form method="get" action="/admin/mirror" class="explore-search">
+  <input type="hidden" name="tab" value="explore">
  {{with .Explore}}<input type="hidden" name="view" value="{{.View}}">
  <input type="hidden" name="task" value="{{.Task}}">
  <input type="hidden" name="size" value="{{.Size}}">
@@ -222,13 +241,13 @@ document.addEventListener('click', function (e) {

 {{if not .Repo}}{{with .Explore}}
 {{if .Rows}}
-<table class="explore">
+<table class="rows explore">
 <tr><th>Repository</th><th class="num">Size</th><th>Architecture</th><th class="num">Context</th>
 <th class="num">Downloads</th><th class="num">Likes</th><th class="num">Updated</th></tr>
 {{range .Rows}}<tr class="explore-row">
  <td><a href="{{.Href}}" class="open-repo" data-repo="{{.ID}}" aria-expanded="false"><code>{{.ID}}</code></a>
    {{if .New}}<span class="tag new">new</span>{{end}}{{if .Gated.Is}}<span class="tag">gated</span>{{end}}
-    {{if or .BaseModel .License}}<div class="from">{{with .BaseModel}}from {{.}}{{end}}{{if and .BaseModel .License}} · {{end}}{{.License}}</div>{{end}}</td>
+    {{if or .BaseModel .License}}<span class="from" title="{{with .BaseModel}}from {{.}}{{end}}{{if and .BaseModel .License}} · {{end}}{{.License}}">{{with .BaseModel}}from {{.}}{{end}}{{if and .BaseModel .License}} · {{end}}{{.License}}</span>{{end}}</td>
  <td class="num">{{.ParamsLabel}}</td>
  <td>{{if .Architecture}}<span{{if not .Verdict.Servable}} class="cannot"{{end}} title="{{.Verdict.Reason}}">{{.Architecture}}{{if .Verdict.Checked}}{{if .Verdict.Servable}} ✓{{else}} ✗{{end}}{{end}}</span>{{else}}—{{end}}</td>
  <td class="num">{{.ContextLabel}}</td>
@@ -307,9 +326,12 @@ document.querySelector("table.explore").addEventListener("submit", async functio
      return;
    }
    note.classList.remove("bad");
-    note.textContent = "✓ " + out.message + " — its progress is in the table above.";
+    note.textContent = "✓ " + out.message + " — ";
+    const link = document.createElement("a");
+    link.href = "/admin/mirror?tab=held";
+    link.textContent = "follow it under Held";
+    note.appendChild(link);
    btn.textContent = "Started";
-    followHeld();
  } catch (err) {
    note.textContent = "Could not reach the gateway: " + err;
    note.classList.add("bad");
@@ -317,28 +339,6 @@ document.querySelector("table.explore").addEventListener("submit", async functio
    btn.textContent = was;
  }
 });
-
-// Refreshes only the mirror's own table, every five seconds while anything
-// in it is downloading.
-let following = false;
-async function followHeld() {
-  if (following) return;
-  following = true;
-  try {
-    for (;;) {
-      const r = await fetch(location.pathname + location.search, { headers: { Accept: "text/html" } });
-      if (!r.ok) break;
-      const doc = new DOMParser().parseFromString(await r.text(), "text/html");
-      const fresh = doc.getElementById("held"), held = document.getElementById("held");
-      if (!fresh || !held) break;
-      held.replaceWith(fresh);
-      if (!fresh.querySelector(".state-starting")) break;
-      await new Promise((ok) => setTimeout(ok, 5000));
-    }
-  } finally {
-    following = false;
-  }
-}
 </script>
 {{if .Hidden}}<p class="note">{{.Hidden}} more {{if eq .Hidden 1}}is{{else}}are{{end}} hidden: their architecture is one the nodes' llama.cpp cannot load.
 Untick “only what these nodes can run” to see them.</p>{{end}}
@@ -354,10 +354,11 @@ directory, or give the mirror a token.</p>
 {{if .Repo}}{{template "mirror_repo" .}}{{end}}
 {{end}}

-<h2>Fetch a model</h2>
+{{if eq .Tab "fetch"}}
 <p class="note">By hand, for anything the search does not reach.</p>
 <form method="post" action="/admin/mirror/fetch">
  <input type="hidden" name="csrf" value="{{.CSRF}}">
+  <input type="hidden" name="tab" value="fetch">
  <label>Source URL
    <input type="text" name="source" required
           placeholder="https://huggingface.co/…/resolve/main/model.gguf"></label>
@@ -373,7 +374,7 @@ directory, or give the mirror a token.</p>
 </form>
 <p class="note">The mirror answers as soon as the download is under way, not when
 it finishes — a model is tens of gigabytes and nothing here holds a connection
-open for that. The row above will show it arriving. A version is never
+open for that. The Held tab shows it arriving. A version is never
 overwritten: fetching into one that exists is refused, so a new build is a new
 version and the old weights stay where they are.</p>
 {{end}}
@@ -400,7 +401,7 @@ version and the old weights stay where they are.</p>
 </details>
 {{end}}
 {{if .RepoSets}}
-<table class="repo-files">
+<table class="rows repo-files">
 <tr><th>What</th><th class="num">Size</th><th>Published checksum</th><th></th></tr>
 {{range .RepoSets}}<tr>
  <td><code>{{.Name}}</code>
@@ -410,11 +411,12 @@ version and the old weights stay where they are.</p>
  <td>
    {{if not .Verdict.Servable}}
    <span class="state state-failed" title="{{.Verdict.Reason}}">cannot be served</span>
-    <div class="note">{{.Verdict.Reason}}</div>
+    <span class="clip-inline" title="{{.Verdict.Reason}}">{{.Verdict.Reason}}</span>
    {{else}}
    {{if .Verdict.Architecture}}<span class="opt" title="{{.Verdict.Reason}}">{{.Verdict.Architecture}}{{if .Verdict.Checked}} ✓{{end}}</span>{{end}}
    <form method="post" action="/admin/mirror/fetch" class="inline">
      <input type="hidden" name="csrf" value="{{$.CSRF}}">
+      <input type="hidden" name="tab" value="explore">
      {{range .Sources}}<input type="hidden" name="source" value="{{.URL}}">
      <input type="hidden" name="file" value="{{.Path}}">
      {{end}}
@@ -439,3 +441,73 @@ computes once it has it.</p>
 <p class="empty">No weight files in {{.Repo.ID}}.</p>
 {{end}}
 {{end}}
+
+{{/* The card beneath a held version: what runs it, where, and how many. */}}
+{{define "mirror_held_detail"}}{{$row := .Row}}{{$csrf := .CSRF}}
+<div class="held-card">
+<p class="repo-meta">
+  <span>{{if gt $row.Files 1}}{{$row.Files}} files{{else}}<code>{{$row.File}}</code>{{end}}</span>
+  <span>{{bytes $row.BytesDone}}</span>
+  {{with $row.SHA256}}<span>sha256 <code title="{{.}}">{{slice . 0 12}}</code></span>{{end}}
+  <span><code>{{$row.URL}}</code></span>
+</p>
+{{if not .D.Models}}
+<p class="note">Not in the catalog: nothing runs this version.
+{{if $row.Serve}}{{else}}Use <strong>serve</strong> on its row to put it into service.{{end}}</p>
+{{end}}
+{{range .D.Models}}
+<section class="held-model">
+<h3>{{.Name}}
+  {{with .View}}<span class="opt">{{.Ready}} of {{.Desired}} ready{{if ne .Total .Ready}} · {{.Total}} in all{{end}}</span>
+  {{if .Parked}}<span class="tag bad">parked</span>{{else if .Unplaceable}}<span class="tag bad">does not fit</span>{{end}}{{end}}
+</h3>
+{{with .View.Reason}}<p class="note error-row">{{.}}</p>{{end}}
+
+{{if .Replicas}}
+<table class="rows compact replicas">
+<tr><th>Replica</th><th>Node</th><th>State</th><th class="num">Age</th><th class="num">CPU</th><th class="num">Memory</th></tr>
+{{range .Replicas}}{{$rep := .}}{{range $i, $s := .Nodes}}<tr>
+  <td>{{if eq $i 0}}<code>{{$rep.ID}}</code>{{end}}</td>
+  <td><a href="/admin/nodes/{{$s.Node}}">{{$s.Node}}</a>{{if $s.MemoryGB}} <span class="opt">{{$s.MemoryGB}} GB</span>{{end}}</td>
+  <td>{{if eq $i 0}}<span class="state state-{{if eq $rep.State "ready"}}ready{{else if eq $rep.State "failed"}}failed{{else}}starting{{end}}">{{$rep.State}}</span>
+    {{with $rep.Restart}}<span class="opt">restart {{.}}</span>{{end}}{{end}}</td>
+  <td class="num">{{if eq $i 0}}{{age $rep.Age}}{{end}}</td>
+  <td class="num">{{if $s.CPU}}{{printf "%.1f" $s.CPU}} cores{{else}}—{{end}}</td>
+  <td class="num">{{if $s.MemBytes}}{{bytes $s.MemBytes}}{{else}}—{{end}}</td>
+</tr>{{end}}{{end}}
+</table>
+{{else}}<p class="note">No replica runs now{{if eq .Mode "ondemand"}}: it starts when somebody asks for it{{end}}.</p>{{end}}
+
+{{with .Headroom}}
+<p class="headroom">
+{{if .Limited}}Room for at least {{.More}} more replicas.
+{{else if .More}}Room for {{.More}} more now{{if .Nodes}}, on {{range $i, $n := .Nodes}}{{if $i}}, {{end}}{{join $n}}{{end}}{{end}}.
+{{else}}No room for another replica now.{{end}}
+</p>
+{{if and (not .Limited) .Blocked}}<details class="blocked"><summary>Why not {{if .More}}more{{else}}another{{end}}</summary>
+<ul>{{range .Blocked}}<li>{{.}}</li>{{end}}</ul></details>{{end}}
+{{end}}
+
+{{if .Declared}}
+<p class="note">Set in server.yaml, which is read again on every start: change its replicas there.</p>
+{{else}}
+<form method="post" action="/admin/mirror/replicas" class="replicas-form">
+  <input type="hidden" name="csrf" value="{{$csrf}}">
+  <input type="hidden" name="model" value="{{.Name}}">
+  <input type="hidden" name="open" value="{{$row.Open}}">
+  <div class="field-row">
+    <label>Always running
+      <input type="number" name="min" value="{{.Min}}" min="0"{{if .MostNow}} max="{{.MostNow}}"{{end}} required></label>
+    <label>Up to, under load
+      <input type="number" name="max" value="{{.Max}}" min="0" max="64" required></label>
+    <button type="submit">Update</button>
+  </div>
+  <p class="note">{{if .MostNow}}At most {{.MostNow}} can always run with what is free now. {{end}}0 always running with a
+  higher limit loads it when asked for; 0 and 0 stops it. A replica is added under load when requests wait, and
+  taken away after ten quiet minutes.</p>
+</form>
+{{end}}
+</section>
+{{end}}
+</div>
+{{end}}
@@ -34,7 +34,7 @@ budgets bind.</p>
 {{/* The tree is the interface. A quota belongs to a branch, so the form to set
     one opens on that branch rather than asking somebody to name it again in a
     box at the bottom. */}}
-<table class="quota-tree">
+<table class="rows quota-tree">
 <tr><th>Where</th><th class="num">Requests/min</th><th class="num">Tokens/min</th>
    <th class="num">Budget</th><th>Spent</th><th></th></tr>
 {{range .Rows}}
@@ -119,7 +119,7 @@ up — hover it to see where. A value set here is in normal type.</p>
 <p class="note">A person's own limit covers everything they do, across all their
 keys, projects and the chat page. {{.Personal}}</p>
 {{if .People}}
-<table class="quota-tree people-limits">
+<table class="rows quota-tree people-limits">
 <tr><th>Person</th><th class="num">Requests/min</th><th class="num">Tokens/min</th>
    <th class="num">Budget</th><th>Spent</th><th></th></tr>
 {{range .People}}
@@ -47,7 +47,7 @@
 </div>

 {{if .Rows}}
-<table>
+<table class="rows">
  <tr><th>Group</th><th class="num">Requests</th><th class="num">Tokens</th><th class="num">Spend</th><th class="num">Errors</th></tr>
  {{range .Rows}}
  <tr>
@@ -22,7 +22,7 @@

 {{if .Pending}}
 <h2>Waiting for approval</h2>
-<table><tr><th>Email</th><th>Name</th><th>Signed up</th><th>Approve, and put them in</th><th></th></tr>
+<table class="rows"><tr><th>Email</th><th>Name</th><th>Signed up</th><th>Approve, and put them in</th><th></th></tr>
 {{range .Pending}}<tr>
  <td>{{.Email}}</td><td>{{.Name}}</td><td>{{ago .CreatedAt}}</td>
  <td>
@@ -149,14 +149,14 @@ project in the team — an account can only be approved into a project.</p>
        </div>

        {{with index $.Members (printf "project:%s" $p.ID)}}
-        <table class="compact"><tr><th>Member</th><th>State</th><th>Last sign-in</th><th></th></tr>
+        <table class="rows compact"><tr><th>Member</th><th>State</th><th>Last sign-in</th><th></th></tr>
        {{range .}}{{template "member-row" (dict "U" . "D" $ "Scope" (printf "project:%s" $p.ID) "Open" $p.ID)}}{{end}}
        </table>
        {{else}}<p class="empty">No members of its own. Members of its team or organization can use it too.</p>{{end}}
        {{template "add-member" (dict "Kind" "project" "ID" $p.ID "D" $ "Open" $p.ID)}}

        {{with index $.Keys $p.ID}}
-        <table class="compact"><tr><th>Key</th><th>Owner</th><th>State</th><th>Last used</th><th></th></tr>
+        <table class="rows compact"><tr><th>Key</th><th>Owner</th><th>State</th><th>Last used</th><th></th></tr>
        {{range .}}<tr>
          <td>{{.Name}} <code>{{.Prefix}}</code></td>
          <td>{{or .OwnerEmail "issued by an operator"}}</td>
@@ -210,7 +210,7 @@ project in the team — an account can only be approved into a project.</p>

 <h2 id="accounts">All accounts</h2>
 {{if .Accounts}}
-<table id="account-list"><tr><th>Person</th><th>State</th><th>Member of</th><th>Last sign-in</th><th>Add to</th><th></th></tr>
+<table class="rows" id="account-list"><tr><th>Person</th><th>State</th><th>Member of</th><th>Last sign-in</th><th>Add to</th><th></th></tr>
 {{range .Accounts}}{{template "account-row" (dict "U" . "D" $)}}{{end}}
 </table>
 <p class="note">Someone who is a member of nothing uses the gateway on their own:
@@ -298,7 +298,7 @@ access than it had, so they need a shell on the control plane —
  <td>{{$u.Email}}{{if $u.IsAdmin}} <span class="tag">admin</span>{{end}}</td>
  <td class="state state-{{$u.State}}">{{$u.State}}</td>
  <td class="where">
-    {{range $u.Memberships}}<span class="chip-m">{{.Path}}{{if eq .Role "manager"}} <b class="chip-role">manager</b>{{end}}
+    {{range $i, $m := $u.Memberships}}{{if lt $i 2}}{{with $m}}<span class="chip-m">{{.Path}}{{if eq .Role "manager"}} <b class="chip-role">manager</b>{{end}}
      {{if eq (printf "%s" .Scope) "org"}}
      <form method="post" action="/admin/users/{{$u.ID}}/role" class="inline">
        <input type="hidden" name="csrf" value="{{$d.CSRF}}">
@@ -313,8 +313,12 @@ access than it had, so they need a shell on the control plane —
        <input type="hidden" name="csrf" value="{{$d.CSRF}}">
        <input type="hidden" name="scope" value="{{.Scope}}:{{.ScopeID}}">
        <button type="submit" class="chip-x" aria-label="Take out of {{.Path}}" title="Take out of {{.Path}}">×</button>
-      </form></span>
+      </form></span>{{end}}{{end}}
    {{else}}<span class="opt">on their own</span>{{end}}
+    {{/* Two, then a count: a person in six teams was six chips wrapping the
+         row into a paragraph. The rest are named on hover and in their own
+         scopes' member lists, where taking them out is done. */}}
+    {{if gt (len $u.Memberships) 2}}<span class="opt more" title="{{range $i, $m := $u.Memberships}}{{if ge $i 2}}{{$m.Path}}&#10;{{end}}{{end}}">+{{sub (len $u.Memberships) 2}}</span>{{end}}
  </td>
  <td>{{ago $u.LastLoginAt}}</td>
  <td>
@@ -365,7 +369,7 @@ access than it had, so they need a shell on the control plane —
 <details class="edit"><summary>{{len (index $d.Members $k)}} member{{if ne (len (index $d.Members $k)) 1}}s{{end}}</summary>
  <div class="scope-members">
  {{with index $d.Members $k}}
-  <table class="compact">
+  <table class="rows compact">
  {{range .}}{{template "member-row" (dict "U" . "D" $d "Scope" $k "Open" "")}}{{end}}
  </table>
  {{else}}<p class="note">Nobody is a member of {{.Name}} itself; a member here could use every project in it.</p>{{end}}
@@ -51,23 +51,23 @@

 <h2>Nodes</h2>
 {{if .Nodes}}
-<table class="nodes"><tr><th>Node</th><th>State</th><th>CPU</th><th>RAM</th><th class="num">GPUs</th><th class="num">Disk</th><th>Engines</th><th>Classes</th><th>Fabrics</th><th class="num">Models</th><th class="num">Replicas</th><th>Last seen</th></tr>
+<table class="nodes rows"><tr><th>Node</th><th>State</th><th>CPU</th><th>RAM</th>{{if .AnyGPUs}}<th class="num">GPUs</th>{{end}}<th class="num">Disk</th><th>Engines</th>{{if .AnyClasses}}<th>Classes</th>{{end}}{{if .AnyFabrics}}<th>Fabrics</th>{{end}}<th class="num">Models</th><th class="num">Replicas</th><th>Seen</th></tr>
 {{range .Nodes}}<tr>
-  <td><a href="{{if $.OnAdminListener}}/nodes/{{.ID}}{{else}}/admin/nodes/{{.ID}}{{end}}"><code>{{.ID}}</code></a>{{if .MgmtAddr}} <span class="note">{{.MgmtAddr}}</span>{{end}}
+  <td><a href="{{if $.OnAdminListener}}/nodes/{{.ID}}{{else}}/admin/nodes/{{.ID}}{{end}}" title="{{.ID}}"><code>{{shortNode .ID $.NodePrefix}}</code></a>{{if .MgmtAddr}} <span class="note">{{.MgmtAddr}}</span>{{end}}
    <a class="note" href="{{if $.OnAdminListener}}/nodes/{{.ID}}/logs{{else}}/admin/nodes/{{.ID}}/logs{{end}}">logs</a></td>
  <td class="state state-{{.State}}">{{.State}}</td>
  {{/* A dash, not an empty bar, when the agent has not measured it lately:
       unknown is not idle. */}}
  <td class="load">{{with .Load}}{{template "minibar" pctbar .CPUPercent}}<span class="load-v">{{printf "%.0f" .CPUPercent}}%{{if .CPUs}} <span class="note">of {{.CPUs}}</span>{{end}}</span>{{else}}—{{end}}</td>
  <td class="load">{{with .Load}}{{template "minibar" pctbar .MemPercent}}<span class="load-v">{{bytes .MemUsed}} <span class="note">of {{bytes .MemTotal}}</span></span>{{else}}{{if .MemoryGB}}{{.MemoryGB}}G{{else}}—{{end}}{{end}}</td>
-  <td class="num">{{if .GPUTotal}}{{.GPUFree}}/{{.GPUTotal}}{{else}}—{{end}}</td>
+  {{if $.AnyGPUs}}<td class="num">{{if .GPUTotal}}{{.GPUFree}}/{{.GPUTotal}}{{else}}—{{end}}</td>{{end}}
  {{/* Free out of total, because the first is meaningless without the
       second, and an em dash where nothing was reported: an agent that
       was never asked about its disk must not read as a full one. */}}
  <td class="num">{{if .DiskTotalGB}}{{.DiskFreeGB}}G/{{.DiskTotalGB}}G{{else}}—{{end}}</td>
  <td>{{if .Engines}}{{join .Engines}}{{else}}<span class="note">none yet</span>{{end}}</td>
-  <td>{{join .Classes}}</td>
-  <td>{{join .Fabrics}}</td>
+  {{if $.AnyClasses}}<td>{{join .Classes}}</td>{{end}}
+  {{if $.AnyFabrics}}<td>{{join .Fabrics}}</td>{{end}}
  <td class="num">{{if .Models}}{{len .Models}}{{else}}—{{end}}</td>
  <td class="num">{{.Replicas}}</td>
  <td>{{or .LastSeenAgo "—"}}</td>
@@ -80,18 +80,21 @@
 <h2>Models</h2>

 {{if .Models}}
-<table><tr><th>Model</th><th>Aliases</th><th class="num">Ready</th><th class="num">Wanted</th><th>Profiles</th><th>Fallbacks</th></tr>
+<table class="rows"><tr><th>Model</th><th>Aliases</th><th class="num">Ready</th><th class="num">Wanted</th><th>Class</th><th>Reserves</th><th>Profiles</th>{{if .AnyFallbacks}}<th>Fallbacks</th>{{end}}<th>What it is for</th></tr>
 {{range .Models}}<tr>
  <td><code>{{.Name}}</code></td>
  <td>{{join .Aliases}}</td>
  <td class="num">{{.Ready}}</td>
  <td class="num">{{.Min}}–{{.Max}}</td>
+  <td{{if .HeldFor}} class="warn" title="gave room to {{.HeldFor}}"{{end}}>{{.Class}}{{if .Preempted}} <span class="note" title="replicas lost to a higher class">−{{.Preempted}}</span>{{end}}</td>
+  <td{{with .Warnings}} class="warn" title="{{join .}}"{{end}}>{{if or .MemoryGB .CPUCores}}{{.MemoryGB}}G{{if .CPUCores}} · {{.CPUCores}}c{{end}}{{else}}—{{end}}
+    {{- if .MeasuredMemoryGB}} <span class="note">{{printf "%.1f" .MeasuredMemoryGB}}G used</span>{{end}}{{if .Warnings}} ⚠{{end}}</td>
  <td>{{join .Profiles}}</td>
-  <td>{{join .Fallbacks}}</td>
+  {{if $.AnyFallbacks}}<td>{{join .Fallbacks}}</td>{{end}}
+  {{/* Cut to the column, the whole sentence on hover: a row of its own for
+       each description made the table twice as long as the catalog. */}}
+  <td class="clip"{{with .Description}} title="{{.}}"{{end}}>{{or .Description "—"}}</td>
 </tr>
-{{/* Under the row rather than in a column: a sentence in a table cell either
-     wraps every other column into a stripe or gets cut off. */}}
-{{if .Description}}<tr><td colspan="6" class="url-row">{{.Description}}</td></tr>{{end}}
 {{end}}</table>
 {{else}}<p class="empty">The catalog is empty.</p>{{end}}

@@ -106,7 +109,7 @@

 <h2>Replicas</h2>
 {{if .Replicas}}
-<table><tr><th>Replica</th><th>Model</th><th>State</th><th>Pool</th><th>Endpoint</th><th class="num">Age</th><th>Nodes</th></tr>
+<table class="rows"><tr><th>Replica</th><th>Model</th><th>State</th><th>Pool</th><th>Endpoint</th><th class="num">Age</th><th>Nodes</th></tr>
 {{range .Replicas}}<tr>
  <td><code>{{.ID}}</code></td>
  <td>{{.Model}}</td>
@@ -114,7 +117,7 @@
  <td>{{or .Pool "—"}}</td>
  <td>{{if .Endpoint}}<code>{{.Endpoint}}</code>{{else}}—{{end}}</td>
  <td class="num">{{duration .AgeSec}}</td>
-  <td>{{range .Shards}}<code>{{.Node}}</code> {{end}}</td>
+  <td>{{range .Shards}}<code title="{{.Node}}">{{shortNode .Node $.NodePrefix}}</code> {{end}}</td>
 </tr>{{if .Err}}<tr><td colspan="7" class="note">{{.Err}}</td></tr>{{end}}
 {{end}}</table>
 {{else}}<p class="empty">No replicas.</p>{{end}}
@@ -35,7 +35,7 @@
 <h2>Fabric</h2>
 <p class="sub">{{if .Node.Fabrics}}On {{join .Node.Fabrics}} · <a href="{{if $.OnAdminListener}}/fabrics{{else}}/admin/fabrics{{end}}">the map →</a>{{else}}<span class="bad">On no fabric: no port a declared fabric names can carry traffic, so no multi-node replica is placed here.</span>{{end}}</p>
 {{if .Node.IBPorts}}
-<table><tr><th>Adapter</th><th>State</th><th class="num">Rate</th><th>Fabric</th><th>Physically on</th><th>Errors since boot</th><th class="num">Sent</th><th class="num">Received</th></tr>
+<table class="rows"><tr><th>Adapter</th><th>State</th><th class="num">Rate</th><th>Fabric</th><th>Physically on</th><th>Errors since boot</th><th class="num">Sent</th><th class="num">Received</th></tr>
 {{range .Node.IBPorts}}<tr>
  <td><code>{{.HCA}}/{{.Port}}</code>{{if eq .LinkLayer "Ethernet"}} <span class="tag">RoCE</span>{{end}}</td>
  <td class="{{if .Active}}state-ready{{else}}bad{{end}}">{{.State}}{{with .PhysState}} <span class="note">{{.}}</span>{{end}}</td>
@@ -60,13 +60,13 @@ the adapter came up: a number that grows is the one to look at.</p>
  {{if or .Node.NVLinkDomain .Node.IMEX}} · IMEX {{if eq .Node.IMEX "active"}}<span class="state-ready">active</span>, {{.Node.IMEXChannels}} channel{{if ne .Node.IMEXChannels 1}}s{{end}}{{else}}<span class="bad">{{or .Node.IMEX "not installed"}}</span>{{end}}{{end}}
 </p>
 <div class="gpu-wiring">
-<table class="nvl-matrix" aria-label="NVLink between GPUs">
+<table class="rows nvl-matrix" aria-label="NVLink between GPUs">
  <tr><th></th>{{range .Node.GPUs}}<th>{{.Index}}</th>{{end}}</tr>
  {{range $a := .Node.GPUs}}<tr><th>GPU {{$a.Index}}</th>
    {{range $b := $.Node.GPUs}}{{if eq $a.Index $b.Index}}<td class="self"></td>{{else if hasInt $a.NVLinkPeers $b.Index}}<td class="nv" title="GPU {{$a.Index}} ↔ GPU {{$b.Index}} over NVLink">NV</td>{{else}}<td class="pcie" title="GPU {{$a.Index}} ↔ GPU {{$b.Index}} over PCIe">·</td>{{end}}{{end}}
  </tr>{{end}}
 </table>
-<table class="compact gpu-list">
+<table class="rows compact gpu-list">
  <tr><th>GPU</th><th>Name</th><th class="num">NUMA</th><th>Nearest adapter</th><th>Multi-node NVLink</th></tr>
  {{range .Node.GPUs}}<tr>
    <td>{{.Index}}</td><td>{{.Name}}</td><td class="num">{{.NUMA}}</td>
@@ -82,7 +82,7 @@ traffic between nodes should go: PIX and PXB are under the same PCIe switch, SYS

 <h2>What runs here</h2>
 {{if .Engines}}
-<table><tr><th>Model</th><th>Replica</th><th>State</th><th>CPU</th><th class="num">Memory</th></tr>
+<table class="rows"><tr><th>Model</th><th>Replica</th><th>State</th><th>CPU</th><th class="num">Memory</th></tr>
 {{range .Engines}}<tr>
  <td>{{.Model}}</td>
  <td><code>{{.Replica}}</code></td>
@@ -42,7 +42,7 @@
 {{end}}

 {{if .Rows}}
-<table class="compact">
+<table class="rows compact">
  <tr><th>{{if eq .By "user"}}Person{{else if eq .By "project"}}Project{{else}}Team{{end}}</th>
      <th class="num">Requests</th><th class="num">Tokens</th><th class="num">Spend</th><th class="num">Errors</th></tr>
  {{range .Rows}}<tr>
@@ -56,7 +56,7 @@ people do elsewhere — in another organization, or on their own — is not show

 <h2>Teams, projects and limits</h2>
 {{if .Quotas}}
-<table class="quota-tree">
+<table class="rows quota-tree">
 <tr><th>Where</th><th class="num">Requests/min</th><th class="num">Tokens/min</th>
    <th class="num">Budget</th><th>Spent · 30 days</th></tr>
 {{range .Quotas}}
@@ -79,7 +79,7 @@ people do elsewhere — in another organization, or on their own — is not show

 <h2>Members</h2>
 {{if .Members}}
-<table class="compact">
+<table class="rows compact">
 <tr><th>Person</th><th>Member of</th><th>State</th></tr>
 {{range .Members}}<tr>
  <td>{{.Email}}</td>
@@ -109,6 +109,9 @@ var funcs = template.FuncMap{
 		return p
 	},
 	"ago": ago,
+	// shortNode is a node's name without what every node's name starts with.
+	"shortNode": strings.TrimPrefix,
+	"age":       age,
 	// pctbar is a percentage as the minibar template takes it: whole, and
 	// held to 0..100 so a measurement a little over does not draw past its box.
 	"pctbar": func(v float64) struct{ P int } {
Author	SHA1	Message	Date
Ilker ManapandClaude Opus 5.5	ac24e204a6	Add capacity classes and preempt lower ones to place a model Release / build (push) Successful in 1m7s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m46s Details A model's class (critical, standard, opportunistic; unset follows its scaling) decides what may be stopped to make room for it: a lower class for a higher, and within a class only what another grew into under load, for a model below its minimum. Victims are tried on a copy of the inventory and stopped only if they make the room. Preempted replicas are drained with a short grace and their model is held back until the room is used. ctl model resources and class move under ctl model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 18:02:59 +02:00
Ilker ManapandClaude Opus 5.5	990d82d801	Reserve CPU cores and require memory for CPU models Release / build (push) Successful in 1m5s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m47s Details A profile's cpu_cores is reserved out of its node's CPUs by placement, as memory is, and becomes the engine's thread count. Agents report their CPU count in registration and heartbeats. New CPU models must declare memory; model views show reserved against measured memory with warnings, and ctl model resources sets both for a stored model. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 17:43:13 +02:00
Ilker ManapandClaude Opus 5.5	26151ec25a	Format web.go Release / build (push) Successful in 1m4s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m50s Details Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 11:02:32 +02:00
Ilker ManapandClaude Opus 5.5	235bf33dcd	Keep every admin table to one line per record CI / test (push) Failing after 31s Details Release / notes (push) Successful in 4s Details Release / build (push) Successful in 1m4s Details Headers never wrap, free text is clipped with the whole on hover, full-width rows are only notes and forms. The cluster page hides columns empty on every row and shortens node names; People shows two memberships and a count. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 11:02:18 +02:00
Ilker ManapandClaude Opus 5.5	7099043c75	Keep each held row on one line Release / build (push) Successful in 1m8s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m53s Details The replica count, its nodes and the mode selector sit side by side instead of stacking in a fixed-width cell, and node names drop the prefix every node shares, with the full name on hover. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 10:50:02 +02:00
Ilker ManapandClaude Opus 5.5	a32ea0e5ab	Say on each held row how its model runs and where Release / build (push) Successful in 1m8s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 4m0s Details Ready of wanted, how many are starting, and the nodes, from the fleet data the card already reads, so the list answers it without a click. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 10:38:49 +02:00
Ilker ManapandClaude Opus 5.5	d7f1c1d545	Open a held model to its replicas, nodes and room for more Release / build (push) Successful in 1m6s Details Release / notes (push) Successful in 3s Details CI / test (push) Successful in 3m48s Details The Held tab's model name opens a card: each replica with its node, state, age, CPU and memory, the room for more found by dry-running the scheduler on a copy of the inventory, and a form for the always-running and under-load replica counts. An always-running count placement cannot meet is refused with the reason. Headroom is also on the admin API. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 10:19:31 +02:00
Ilker ManapandClaude Opus 5.5	2ae12ae1fe	Split the mirror page into Held, Explore and Fetch tabs CI / test (push) Successful in 3m45s Details Release / build (push) Successful in 58s Details Release / notes (push) Successful in 3s Details One long page made every visit wait on Hugging Face for a list it might not show. Each tab now has its own address and does only its own work; only Held reloads while a download runs, and a refused fetch returns to the form it came from. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 08:31:28 +02:00