BACKLOG: O2 tried on the PoC, two routers carry 1.5x the leader alone
CI / test (push) Successful in 9m26s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-01 16:51:54 +01:00
1 parent 8b7eefef16
commit 95fb5eeb6e
1 file changed
+13 -1
+13 -1
View File
@@ -1957,7 +1957,19 @@ one control plane's router can. In order:
every router for the same replica set; what is left is the same router, which is the balancer's (source hash).
- [ ] in front: IPVS on the keepalived the setup already installs, DNS round-robin, or the customer's balancer
(J7, L9). Until then the standbys' own addresses are what spread the load.
- [ ] tried on the PoC: 140 and 150 both taking `/v1`, the O1 load split between them.
- [x] tried on the PoC (2026-11-01, v0.16.1-14-g8b7eefe, setting on since): 140 leads, 150 follows. A request
through 150 is answered and logged by 150; a model added on the leader is routable on 150 within a tick; a
CPU replica used only through 150 shows idle 6 s on the leader. Load from 146, 64-token streams, a sim engine on
node06 at 5,000 tok/s a stream (2,300 req/s on its own; at 500 tok/s it was the ceiling, ~490 req/s, routers or
not), pve1 at load 14 of 28 threads:
| streams | leader alone | split over 140 + 150 | first token p50, alone / split |
|---|---|---|---|
| 128 | 971 req/s (140 at 176%) | 832 + 692 = 1,524 req/s (178% / 166%) | 115 / 59–75 ms |
| 256 | 990 req/s (177%) | 816 + 677 = 1,493 req/s (177% / 164%) | 243 / 135–166 ms |
1.5× with two routers on an oversubscribed host, no errors. Not 2×: both routers' VMs and the load generator
share pve1's cores.
- Known and accepted for now, per router: the precise KV index is empty on a standby (agents stream to the leader
only), a revoked key works on a standby until `auth.cache_ttl`, canary promotion sees the leader's share only,
the answer cache and prefix tree are split, an MCP caller's rate is per router.