Give two control planes a shared address, and a probe that means something
CI / test (push) Successful in 1m36s
CI / test (push) Successful in 1m36s
The last piece of HA, and it is a deployment pattern rather than code: llmgw picks one leader and agents follow it, but client traffic arrives in front of both and llmgw is not in front of anything. Writing the pattern found a bug in the pattern. The first draft said to health-check `/health` and claimed a standby still routes to replicas the leader placed, so sending it traffic was fine. It does not. A standby's reconciler places nothing and it holds no agent streams to adopt from, so it publishes an empty endpoint list — a load balancer following that advice would have sent half the traffic to something that answers every request with 503. So there is now a `/ready` that means "this process has somewhere to send a request": 503 on a standby, 503 on a leader whose replicas have all failed, 200 when there is something to serve. `/health` keeps meaning the process is up, which is a different and less useful question. Both are open, because a load balancer has no key. There is a gap during failover where neither passes — three seconds when the leader stops cleanly, up to the lease TTL when it does not — and the document says so rather than implying otherwise. Traffic cannot go to a control plane that has nothing to route to. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
a35e8796d2
commit
2bf7de1a43
6 files changed
+257
-1
No files matched your search
@@ -718,6 +718,18 @@ Without authentication, `threads.store` and `threads.retain_hours` in
|
||||
`server.yaml` keep both conversations and notes in one shared pool. That is for
|
||||
one trusted machine only; see [quickstart.md](quickstart.md).
|
||||
|
||||
## Two control planes
|
||||
|
||||
llmgw handles the hard half itself: only one places work, and agents find
|
||||
whichever is leading. What it does not do is move client traffic, which
|
||||
happens in front of both.
|
||||
|
||||
[deploy/ha/](../deploy/ha/) has the pattern and a keepalived example. The one
|
||||
thing to get right is the health check: point it at `/ready`, not `/health`.
|
||||
`/health` is 200 on a standby, which is alive and holds no endpoints at all,
|
||||
so a load balancer using it would send half the traffic to something that
|
||||
refuses every request.
|
||||
|
||||
## Reclaiming disk
|
||||
|
||||
`llmgw ctl nodes` has a DISK column: free space against the size it is free
|
||||
|
||||
Reference in new issue
Block a user