From 95fb5eeb6e552c5a1301500800bf4b71283454fe Mon Sep 17 00:00:00 2001 From: Ilker Manap Date: Sun, 1 Nov 2026 16:51:54 +0100 Subject: [PATCH] BACKLOG: O2 tried on the PoC, two routers carry 1.5x the leader alone Co-Authored-By: Claude Opus 5.5 (1M context) --- BACKLOG.md | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/BACKLOG.md b/BACKLOG.md index 24a8e64..4026f4a 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -1957,7 +1957,19 @@ one control plane's router can. In order: every router for the same replica set; what is left is the same router, which is the balancer's (source hash). - [ ] in front: IPVS on the keepalived the setup already installs, DNS round-robin, or the customer's balancer (J7, L9). Until then the standbys' own addresses are what spread the load. - - [ ] tried on the PoC: 140 and 150 both taking `/v1`, the O1 load split between them. + - [x] tried on the PoC (2026-11-01, v0.16.1-14-g8b7eefe, setting on since): 140 leads, 150 follows. A request + through 150 is answered and logged by 150; a model added on the leader is routable on 150 within a tick; a + CPU replica used only through 150 shows idle 6 s on the leader. Load from 146, 64-token streams, a sim engine on + node06 at 5,000 tok/s a stream (2,300 req/s on its own; at 500 tok/s it was the ceiling, ~490 req/s, routers or + not), pve1 at load 14 of 28 threads: + + | streams | leader alone | split over 140 + 150 | first token p50, alone / split | + |---|---|---|---| + | 128 | 971 req/s (140 at 176%) | 832 + 692 = 1,524 req/s (178% / 166%) | 115 / 59–75 ms | + | 256 | 990 req/s (177%) | 816 + 677 = 1,493 req/s (177% / 164%) | 243 / 135–166 ms | + + 1.5× with two routers on an oversubscribed host, no errors. Not 2×: both routers' VMs and the load generator + share pve1's cores. - Known and accepted for now, per router: the precise KV index is empty on a standby (agents stream to the leader only), a revoked key works on a standby until `auth.cache_ttl`, canary promotion sees the leader's share only, the answer cache and prefix tree are split, an MCP caller's rate is per router.