Commit Graph
3 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 975c93994f Set up a replicated store with server init --patroni, and watch it
init --patroni user@host ... installs PostgreSQL, Patroni and etcd on the
given machines over ssh, with this one as the third etcd vote, makes the role
and database, and moves the store in use into the cluster. Each step checks
first, so it changes nothing on a cluster that is up. For a store of several
hosts the control plane publishes each member's role, state and lag from
Patroni, a stopped member as missing, prints it in ctl status, and alerts on
it. Making an administrator on PostgreSQL sent a bool to an integer column;
it sends an integer now.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 20:57:41 +02:00
Ilker ManapandClaude Opus 5.5 7a562ac1e7 Keep the store under Patroni, and back up from a store of several hosts
deploy/ha documents PostgreSQL under Patroni with a three-member etcd, as
running on the reference cluster, and the address that follows the primary.
Backups read such an address as libpq host and port lists and prefer a
standby, rather than resolving the list as one name.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-03 20:31:19 +02:00
Ilker ManapandClaude Opus 5 2bf7de1a43 Give two control planes a shared address, and a probe that means something
The last piece of HA, and it is a deployment pattern rather than code: llmgw
picks one leader and agents follow it, but client traffic arrives in front of
both and llmgw is not in front of anything.

Writing the pattern found a bug in the pattern. The first draft said to
health-check `/health` and claimed a standby still routes to replicas the
leader placed, so sending it traffic was fine. It does not. A standby's
reconciler places nothing and it holds no agent streams to adopt from, so it
publishes an empty endpoint list — a load balancer following that advice would
have sent half the traffic to something that answers every request with 503.

So there is now a `/ready` that means "this process has somewhere to send a
request": 503 on a standby, 503 on a leader whose replicas have all failed,
200 when there is something to serve. `/health` keeps meaning the process is
up, which is a different and less useful question. Both are open, because a
load balancer has no key.

There is a gap during failover where neither passes — three seconds when the
leader stops cleanly, up to the lease TTL when it does not — and the document
says so rather than implying otherwise. Traffic cannot go to a control plane
that has nothing to route to.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-27 07:52:54 +02:00