Standbys that answer the client API themselves, behind router.active_active
CI / test (push) Successful in 9m35s

A standby follows the leader's Ready replicas over the agent port, every
two seconds, and reports back the replicas its router used and the
on-demand models it was asked for. No engine keys travel: every control
plane derives them from the cluster secret. Budgets are raised to the
store's every 30 s. Unheard from the leader for a minute, a standby
passes /v1 on as before. Off by default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-01 13:47:47 +01:00
1 parent f664592948
commit 8b7eefef16
22 files changed
+936 -15

No files matched your search

+133
View File
@@ -110,6 +110,9 @@
{
"name": "restart"
},
{
"name": "routing"
},
{
"name": "settings"
},
@@ -4793,6 +4796,77 @@
}
}
},
"/api/v1/routing": {
"get": {
"operationId": "getRouting",
"summary": "The Ready replicas a standby routes to",
"description": "Read by a standby control plane when router.active_active is on: each replica's address, model, profile, pool and weight, without its key, and which catalog is in force.",
"tags": [
"routing"
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/control.Routing"
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/routing/activity": {
"post": {
"operationId": "postRoutingActivity",
"summary": "A standby's router reports what it used",
"description": "When it last sent each replica a request, and the on-demand models it was asked for, so the leader does not evict a replica busy behind a standby. Answered with the routing, so one request a tick does both.",
"tags": [
"routing"
],
"parameters": [
{
"$ref": "#/components/parameters/Actor"
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/control.Activity"
}
}
}
},
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/control.Routing"
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/settings": {
"get": {
"operationId": "getSettings",
@@ -10554,6 +10628,65 @@
}
}
},
"control.Activity": {
"type": "object",
"description": "Activity is what a standby's router reports back: when it last sent each replica a request, and the on-demand models it was asked for. The leader evicts and loads by these, and without them a replica busy behind a standby looks idle to the one machine that decides.",
"properties": {
"last_used": {
"type": "object",
"additionalProperties": {
"type": "string",
"format": "date-time"
}
},
"want": {
"type": "array",
"items": {
"type": "string"
}
}
}
},
"control.RoutedReplica": {
"type": "object",
"description": "RoutedReplica is a Ready replica as a router needs it: where it is, what it serves and how much it may take. Not its key — every control plane derives that from the replica's id and the cluster's secret, so none travels.",
"properties": {
"id": {
"type": "string"
},
"model": {
"type": "string"
},
"profile": {
"type": "string"
},
"pool": {
"type": "string"
},
"endpoint": {
"type": "string"
},
"weight": {
"type": "number"
}
}
},
"control.Routing": {
"type": "object",
"description": "Routing is what a standby that answers requests itself reads from the leader: the replicas it may send to, and which catalog they belong to.",
"properties": {
"replicas": {
"type": "array",
"items": {
"$ref": "#/components/schemas/control.RoutedReplica"
}
},
"catalog": {
"type": "string",
"description": "Catalog changes whenever the leader puts a catalog in force. Only compared with the last one a standby saw, never with its own: a model added, scaled or given a canary is in the store, and the standby reads it again from there."
}
}
},
"eval.Live": {
"type": "object",
"description": "Live is a run as it stands: one running, with how many of its cases are done, or one finished.",
+16
View File
@@ -320,6 +320,7 @@ rather than repeated.
| `batch` | `Batch` | [Router](#router) |
| `cache` | `Cache` | [Router](#router) |
| `trusted_proxies` | `[]string` | [Router](#router) |
| `active_active` | `bool` | [Router](#router) |
| `timeouts` | `Timeouts` | [Router](#router) |
| `scoring` | `Scoring` | [Router](#router) |
| `admission` | `Admission` | [Router](#router) |
@@ -2810,6 +2811,21 @@ as addresses or CIDR prefixes. Their X-Forwarded-For, -Proto and -Host
are believed; anyone else's are not, since they decide the address
written into the setup scripts and whose attempts are counted where.
### `active_active`
`bool`
ActiveActive has standby control planes answer the client API
themselves, routing to the leader's replicas, rather than passing
every request to the leader. More machines carry the traffic, which
is the point; what each router keeps in memory is then its own:
rate limits are counted per control plane, so a key may make up to
as many times its limit as there are control planes taking its
requests; a budget is checked against the store every half minute;
cached answers and prefix affinity are split between them. Off by
default. Which machine a client reaches is the load balancer's or
DNS's to say; the floating address stays on the leader.
### `timeouts`
`Timeouts`