Federation: another llmgw cluster as a provider, its models as PEER/MODEL

ctl federation add/list/remove/sync keeps peers in the store (migration
0029); their /v1/models are offered here as external models named
PEER/MODEL and kept in step, a model still named by a fallback kept
withdrawn. Peers are checked at /ready, timed, and taken out of routing
while down. Requests to a peer carry x-llmgw-hop and are refused (508) when
they come back to a cluster they passed through; residency and no_store
keep requests off peers; X-Request-Id and, with --send-tags, attribution
tags go along. ctl model fallbacks sets overflow to a peer. Console view,
metrics, admin API routes and docs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 01:21:30 +02:00
1 parent d41104a452
commit 40e9fd2ab1
33 files changed
+2944 -23

No files matched your search

+17
View File
@@ -174,12 +174,29 @@ type Server struct {
Control Control `koanf:"control"`
Models []catalog.Model `koanf:"models"`
// Federation is how this cluster takes part in federation with other
// llmgw clusters. The peers themselves are added at runtime, with
// `llmgw ctl federation add`, and kept in the store.
Federation Federation `koanf:"federation"`
// Endpoints is the phase-1 static replica list. It is only consulted when
// no model declares a profile, which is what makes a router-only smoke test
// against sim-engines possible without a control plane.
Endpoints []Endpoint `koanf:"endpoints"`
}
// Federation is this cluster's side of federation with other llmgw clusters.
type Federation struct {
// Name is this cluster as a peer reads it, in the federated_from tag
// sent with a request to a peer set to receive tags (`ctl federation add
// --send-tags`). Empty is this machine's host name.
Name string `koanf:"name"`
// Interval is how often each peer is asked whether it is ready, timed,
// and what it offers. A peer that does not answer is taken out of
// routing until it does. Zero is 10s.
Interval time.Duration `koanf:"interval"`
}
// Monitoring points the dashboard at a Prometheus holding this cluster's
// history.
//