Standbys that answer the client API themselves, behind router.active_active
A standby follows the leader's Ready replicas over the agent port, every two seconds, and reports back the replicas its router used and the on-demand models it was asked for. No engine keys travel: every control plane derives them from the cluster secret. Budgets are raised to the store's every 30 s. Unheard from the leader for a minute, a standby passes /v1 on as before. Off by default. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f664592948
commit
8b7eefef16
22 files changed
+936
-15
No files matched your search
@@ -49,6 +49,9 @@ type leaderForward struct {
|
||||
self string
|
||||
tls tlsconf.Config
|
||||
log *slog.Logger
|
||||
// routes, with router.active_active, is the leader's replicas this
|
||||
// standby routes to; nil when every request goes to the leader.
|
||||
routes *standbyRoutes
|
||||
}
|
||||
|
||||
// standby wraps the admin handler: /api/ goes to the leader while this
|
||||
@@ -117,7 +120,8 @@ func (f *leaderForward) public(next http.Handler, port string) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
// Once only: a request another control plane forwarded is answered
|
||||
// here, leading or not, rather than sent on while the lease moves.
|
||||
if publicStaysHere(r.URL.Path) || f.lead.IsLeader() || r.Header.Get(hdrForwardedBy) != "" {
|
||||
if publicStaysHere(r.URL.Path) || f.lead.IsLeader() || r.Header.Get(hdrForwardedBy) != "" ||
|
||||
f.answersHere(r.URL.Path) {
|
||||
next.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
@@ -154,6 +158,15 @@ func (f *leaderForward) public(next http.Handler, port string) http.Handler {
|
||||
})
|
||||
}
|
||||
|
||||
// answersHere is whether a standby answers this client request itself: the
|
||||
// API under /v1, with router.active_active, while it has the leader's
|
||||
// replicas to send it to. Not the portal, its admin pages or /mcp, whose
|
||||
// llmgw:admin tools read the cluster from memory only the leader has; and
|
||||
// not while the leader has gone unheard, when it is passed on as before.
|
||||
func (f *leaderForward) answersHere(path string) bool {
|
||||
return f.routes != nil && strings.HasPrefix(path, "/v1/") && f.routes.serving()
|
||||
}
|
||||
|
||||
// trustControlPlanes keeps the client port trusting the other control
|
||||
// planes' X-Forwarded-For, besides router.trusted_proxies: a standby passes
|
||||
// requests on, and the client is the address it was given, not the standby.
|
||||
|
||||
Reference in new issue
Block a user