Standbys that answer the client API themselves, behind router.active_active
CI / test (push) Successful in 9m35s
CI / test (push) Successful in 9m35s
A standby follows the leader's Ready replicas over the agent port, every two seconds, and reports back the replicas its router used and the on-demand models it was asked for. No engine keys travel: every control plane derives them from the cluster secret. Budgets are raised to the store's every 30 s. Unheard from the leader for a minute, a standby passes /v1 on as before. Off by default. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f664592948
commit
8b7eefef16
22 files changed
+936
-15
No files matched your search
@@ -320,6 +320,7 @@ rather than repeated.
|
||||
| `batch` | `Batch` | [Router](#router) |
|
||||
| `cache` | `Cache` | [Router](#router) |
|
||||
| `trusted_proxies` | `[]string` | [Router](#router) |
|
||||
| `active_active` | `bool` | [Router](#router) |
|
||||
| `timeouts` | `Timeouts` | [Router](#router) |
|
||||
| `scoring` | `Scoring` | [Router](#router) |
|
||||
| `admission` | `Admission` | [Router](#router) |
|
||||
@@ -2810,6 +2811,21 @@ as addresses or CIDR prefixes. Their X-Forwarded-For, -Proto and -Host
|
||||
are believed; anyone else's are not, since they decide the address
|
||||
written into the setup scripts and whose attempts are counted where.
|
||||
|
||||
### `active_active`
|
||||
|
||||
`bool`
|
||||
|
||||
ActiveActive has standby control planes answer the client API
|
||||
themselves, routing to the leader's replicas, rather than passing
|
||||
every request to the leader. More machines carry the traffic, which
|
||||
is the point; what each router keeps in memory is then its own:
|
||||
rate limits are counted per control plane, so a key may make up to
|
||||
as many times its limit as there are control planes taking its
|
||||
requests; a budget is checked against the store every half minute;
|
||||
cached answers and prefix affinity are split between them. Off by
|
||||
default. Which machine a client reaches is the load balancer's or
|
||||
DNS's to say; the floating address stays on the leader.
|
||||
|
||||
### `timeouts`
|
||||
|
||||
`Timeouts`
|
||||
|
||||
Reference in new issue
Block a user