Two control planes: a failover in seconds, and the console to set one up

A control plane being stopped held every agent's stream for the thirty
seconds of a graceful shutdown, so the standby led within four seconds and
had no nodes for thirty. The agent server is now closed, not drained. A
control plane that loses its lease while it lives on lets its agents go,
rather than keeping nodes the new leader cannot reach.

ctl status and the console's Overview say which control plane leads, which
stand by, and whether the floating address is on the leader. The console
adds a control plane (Do → Add a control plane), asking for the machine and
the floating address. server add says when the mirror would refuse the new
control plane.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 02:17:51 +02:00
1 parent 289a4ff8e8
commit ec1918b23b
14 files changed
+623 -17

No files matched your search

+7
View File
@@ -54,6 +54,9 @@ type API struct {
releases string
// storeStatus reads a replicated store's members; nil for one database.
storeStatus func() ([]StoreMember, error)
// controlPlanes says what each control plane answers; nil for a cluster
// of one.
controlPlanes func() ControlPlanes
// serve runs registry-to-replica jobs; nil without a mirror and a store.
serve *modelserve.Server
// history is the Prometheus the console's charts read from.
@@ -870,6 +873,10 @@ func (a *API) SetUsage(u Usage) { a.usage = u }
// SetStoreStatus supplies the state of a replicated store, for the summary.
func (a *API) SetStoreStatus(f func() ([]StoreMember, error)) { a.storeStatus = f }
// SetControlPlanes sets where the summary learns which control plane leads
// and which stand by.
func (a *API) SetControlPlanes(f func() ControlPlanes) { a.controlPlanes = f }
// ScopeUsageView is one group's spend, with its quota when it has one.
type ScopeUsageView struct {
Scope string `json:"scope"`
+27
View File
@@ -35,6 +35,9 @@ type Summary struct {
// StoreError why it could not be read; both empty for a single database.
Store []StoreMember `json:"store,omitempty"`
StoreError string `json:"store_error,omitempty"`
// ControlPlanes is the cluster's control planes when there is more than
// one: which leads, which stand by, and the floating address in front.
ControlPlanes ControlPlanes `json:"control_planes,omitzero"`
}
// StoreMember is one member of a replicated store, as its Patroni reports it.
@@ -47,6 +50,27 @@ type StoreMember struct {
Lag int64 `json:"lag"`
}
// ControlPlanes is every control plane of a cluster that has several, as
// each answered when asked.
type ControlPlanes struct {
Members []ControlPlane `json:"members,omitempty"`
// Floating is the address keepalived keeps on the leader (10.0.0.161),
// and FloatingUp whether it answered as a leader when asked; empty when
// nothing floats.
Floating string `json:"floating,omitempty"`
FloatingUp bool `json:"floating_up,omitempty"`
}
// ControlPlane is one control plane.
type ControlPlane struct {
Name string `json:"name"`
Address string `json:"address"`
// State is "leading", "standing by" or "not answering".
State string `json:"state"`
// This is the control plane that answered the summary.
This bool `json:"this,omitempty"`
}
// NodeTotals counts nodes by the states that need different responses.
type NodeTotals struct {
Total int `json:"total"`
@@ -128,6 +152,9 @@ func (a *API) Summary() Summary {
s.StoreError = err.Error()
}
}
if a.controlPlanes != nil {
s.ControlPlanes = a.controlPlanes()
}
nodes := a.Nodes()
byClass := map[string]*ClassUse{}