Standbys that answer the client API themselves, behind router.active_active
CI / test (push) Successful in 9m35s
CI / test (push) Successful in 9m35s
A standby follows the leader's Ready replicas over the agent port, every two seconds, and reports back the replicas its router used and the on-demand models it was asked for. No engine keys travel: every control plane derives them from the cluster secret. Budgets are raised to the store's every 30 s. Unheard from the leader for a minute, a standby passes /v1 on as before. Off by default. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f664592948
commit
8b7eefef16
22 files changed
+936
-15
No files matched your search
@@ -641,6 +641,18 @@ type Router struct {
|
||||
// written into the setup scripts and whose attempts are counted where.
|
||||
TrustedProxies []string `koanf:"trusted_proxies"`
|
||||
|
||||
// ActiveActive has standby control planes answer the client API
|
||||
// themselves, routing to the leader's replicas, rather than passing
|
||||
// every request to the leader. More machines carry the traffic, which
|
||||
// is the point; what each router keeps in memory is then its own:
|
||||
// rate limits are counted per control plane, so a key may make up to
|
||||
// as many times its limit as there are control planes taking its
|
||||
// requests; a budget is checked against the store every half minute;
|
||||
// cached answers and prefix affinity are split between them. Off by
|
||||
// default. Which machine a client reaches is the load balancer's or
|
||||
// DNS's to say; the floating address stays on the leader.
|
||||
ActiveActive bool `koanf:"active_active"`
|
||||
|
||||
Timeouts Timeouts `koanf:"timeouts"`
|
||||
Scoring Scoring `koanf:"scoring"`
|
||||
Admission Admission `koanf:"admission"`
|
||||
|
||||
Reference in new issue
Block a user