Standbys that answer the client API themselves, behind router.active_active
CI / test (push) Successful in 9m35s

A standby follows the leader's Ready replicas over the agent port, every
two seconds, and reports back the replicas its router used and the
on-demand models it was asked for. No engine keys travel: every control
plane derives them from the cluster secret. Budgets are raised to the
store's every 30 s. Unheard from the leader for a minute, a standby
passes /v1 on as before. Off by default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-01 13:47:47 +01:00
1 parent f664592948
commit 8b7eefef16
22 files changed
+936 -15

No files matched your search

+16
View File
@@ -320,6 +320,7 @@ rather than repeated.
| `batch` | `Batch` | [Router](#router) |
| `cache` | `Cache` | [Router](#router) |
| `trusted_proxies` | `[]string` | [Router](#router) |
| `active_active` | `bool` | [Router](#router) |
| `timeouts` | `Timeouts` | [Router](#router) |
| `scoring` | `Scoring` | [Router](#router) |
| `admission` | `Admission` | [Router](#router) |
@@ -2810,6 +2811,21 @@ as addresses or CIDR prefixes. Their X-Forwarded-For, -Proto and -Host
are believed; anyone else's are not, since they decide the address
written into the setup scripts and whose attempts are counted where.
### `active_active`
`bool`
ActiveActive has standby control planes answer the client API
themselves, routing to the leader's replicas, rather than passing
every request to the leader. More machines carry the traffic, which
is the point; what each router keeps in memory is then its own:
rate limits are counted per control plane, so a key may make up to
as many times its limit as there are control planes taking its
requests; a budget is checked against the store every half minute;
cached answers and prefix affinity are split between them. Off by
default. Which machine a client reaches is the load balancer's or
DNS's to say; the floating address stays on the leader.
### `timeouts`
`Timeouts`