Federation: another llmgw cluster as a provider, its models as PEER/MODEL
ctl federation add/list/remove/sync keeps peers in the store (migration 0029); their /v1/models are offered here as external models named PEER/MODEL and kept in step, a model still named by a fallback kept withdrawn. Peers are checked at /ready, timed, and taken out of routing while down. Requests to a peer carry x-llmgw-hop and are refused (508) when they come back to a cluster they passed through; residency and no_store keep requests off peers; X-Request-Id and, with --send-tags, attribution tags go along. ctl model fallbacks sets overflow to a peer. Console view, metrics, admin API routes and docs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d41104a452
commit
40e9fd2ab1
33 files changed
+2944
-23
No files matched your search
@@ -174,12 +174,29 @@ type Server struct {
|
||||
Control Control `koanf:"control"`
|
||||
Models []catalog.Model `koanf:"models"`
|
||||
|
||||
// Federation is how this cluster takes part in federation with other
|
||||
// llmgw clusters. The peers themselves are added at runtime, with
|
||||
// `llmgw ctl federation add`, and kept in the store.
|
||||
Federation Federation `koanf:"federation"`
|
||||
|
||||
// Endpoints is the phase-1 static replica list. It is only consulted when
|
||||
// no model declares a profile, which is what makes a router-only smoke test
|
||||
// against sim-engines possible without a control plane.
|
||||
Endpoints []Endpoint `koanf:"endpoints"`
|
||||
}
|
||||
|
||||
// Federation is this cluster's side of federation with other llmgw clusters.
|
||||
type Federation struct {
|
||||
// Name is this cluster as a peer reads it, in the federated_from tag
|
||||
// sent with a request to a peer set to receive tags (`ctl federation add
|
||||
// --send-tags`). Empty is this machine's host name.
|
||||
Name string `koanf:"name"`
|
||||
// Interval is how often each peer is asked whether it is ready, timed,
|
||||
// and what it offers. A peer that does not answer is taken out of
|
||||
// routing until it does. Zero is 10s.
|
||||
Interval time.Duration `koanf:"interval"`
|
||||
}
|
||||
|
||||
// Monitoring points the dashboard at a Prometheus holding this cluster's
|
||||
// history.
|
||||
//
|
||||
|
||||
Reference in new issue
Block a user