Standbys that answer the client API themselves, behind router.active_active
CI / test (push) Successful in 9m35s

A standby follows the leader's Ready replicas over the agent port, every
two seconds, and reports back the replicas its router used and the
on-demand models it was asked for. No engine keys travel: every control
plane derives them from the cluster secret. Budgets are raised to the
store's every 30 s. Unheard from the leader for a minute, a standby
passes /v1 on as before. Off by default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-01 13:47:47 +01:00
1 parent f664592948
commit 8b7eefef16
22 files changed
+936 -15

No files matched your search

+14 -1
View File
@@ -49,6 +49,9 @@ type leaderForward struct {
self string
tls tlsconf.Config
log *slog.Logger
// routes, with router.active_active, is the leader's replicas this
// standby routes to; nil when every request goes to the leader.
routes *standbyRoutes
}
// standby wraps the admin handler: /api/ goes to the leader while this
@@ -117,7 +120,8 @@ func (f *leaderForward) public(next http.Handler, port string) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Once only: a request another control plane forwarded is answered
// here, leading or not, rather than sent on while the lease moves.
if publicStaysHere(r.URL.Path) || f.lead.IsLeader() || r.Header.Get(hdrForwardedBy) != "" {
if publicStaysHere(r.URL.Path) || f.lead.IsLeader() || r.Header.Get(hdrForwardedBy) != "" ||
f.answersHere(r.URL.Path) {
next.ServeHTTP(w, r)
return
}
@@ -154,6 +158,15 @@ func (f *leaderForward) public(next http.Handler, port string) http.Handler {
})
}
// answersHere is whether a standby answers this client request itself: the
// API under /v1, with router.active_active, while it has the leader's
// replicas to send it to. Not the portal, its admin pages or /mcp, whose
// llmgw:admin tools read the cluster from memory only the leader has; and
// not while the leader has gone unheard, when it is passed on as before.
func (f *leaderForward) answersHere(path string) bool {
return f.routes != nil && strings.HasPrefix(path, "/v1/") && f.routes.serving()
}
// trustControlPlanes keeps the client port trusting the other control
// planes' X-Forwarded-For, besides router.trusted_proxies: a standby passes
// requests on, and the client is the address it was given, not the standby.