BACKLOG: O1 measured, the router's ceiling on the PoC

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

BACKLOG.md

@@ -1918,9 +1918,21 @@ Every client request goes through the leader's router today: the floating addres
 passes `/v1` on to it (a standby's router sees an empty cluster). That is failover, not scale: the cluster serves what
 one control plane's router can. In order:
 
-- O1. Measure the ceiling first: one router in front of a fast fake engine (`sim-engine`, through an external model),
-  streams at 1 to 256 at once from another machine, the leader's CPU beside it. Is it the router, the guardrails, the
-  usage writes or TLS? Written down with the host's load (pve1 is oversubscribed).
+- O1. Measured 2026-10-30/31 on the PoC: the leader (2 vCPU, 2 GB, pve1 at load 6–10 of 28 threads) in front of
+  a fake engine on node06 (`sim-engine`, 500 tok/s a stream, no batch slowdown, as an external model), 64-token
+  streams from mirror 146, the default `observe` guardrails and usage written to PostgreSQL:
+
+  | at once | req/s | tok/s | first token p50/p95 | leader llmgw CPU |
+  |---|---|---|---|---|
+  | 64 | 465 | 29,752 | 9 / 34 ms | 159% |
+  | 128 | 572 | 36,607 | 91 / 167 ms | 181% |
+  | 256 | 577 | 36,900 | 310 / 392 ms | 182% |
+  | 512 | 551 | 35,240 | 793 / 883 ms | 181% |
+
+  The ceiling is the router's CPU: ~575 requests/s, ~37k streamed tokens/s on two cores, no errors; past it the
+  first token waits. Found on the way and fixed: an external model held to 3–4 at once by admission; a valid key
+  throttled as "unrecognised" when its cache entry expired under load. Next: a profile of where the CPU goes
+  (pprof under the same load) before O2, and the same run on a machine the size of a customer's control plane.
 - O2. Every control plane answers `/v1` itself (active-active):
   - the leader's replica table followed live by the standbys, so their routers route;
   - rate limits and budgets kept cluster-wide, not per process (J6), or three routers allow three times as much;