Leadership stays put while the store's primary is replaced
CI / test (push) Failing after 30m3s

Measured again after the store fixes. Cutting the store primary's power no
longer hung requests, but leadership moved anyway: the lease had lapsed for
everyone while Patroni promoted the standby, and a standby asked first. The
new leader then spent about 25 s without its nodes, and so without its
judges.

A standby that saw the store out of reach now leaves the lease alone for one
renew round after the store returns. The default lease is 10 s, and a node
detects a control plane that died silently in 8 s (5 s idle, 3 s ping)
rather than 15.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-15 23:17:58 +02:00
1 parent 5a4b022dcd
commit b09922b29b
7 files changed
+151 -9

No files matched your search

+7 -4
View File
@@ -1417,11 +1417,14 @@ func Client(cfg tlsconf.Config, timeout time.Duration) (*http.Client, error) {
// off — is never closed: nothing sends the FIN, and the agent's heartbeats sit
// in the socket's buffer until the kernel gives up retransmitting, fifteen
// minutes later. Found by switching off a leader: the standby took over in 14
// seconds and the node stayed with the dead one. An HTTP/2 ping after ten
// quiet seconds, given five to answer, finds it in a quarter of a minute.
// seconds and the node stayed with the dead one. An HTTP/2 ping after five
// quiet seconds, given three to answer, finds it in eight: by the time the
// standby leads, as it does within the lease's ten seconds, the node is on
// its way to it. At fifteen, with the lease at fifteen too, the new leader
// went without its nodes, and so its judges, for half a minute.
const (
streamIdle = 10 * time.Second
streamPing = 5 * time.Second
streamIdle = 5 * time.Second
streamPing = 3 * time.Second
)
// localizeSpec fills in what only this node knows.