Federation: another llmgw cluster as a provider, its models as PEER/MODEL

ctl federation add/list/remove/sync keeps peers in the store (migration
0029); their /v1/models are offered here as external models named
PEER/MODEL and kept in step, a model still named by a fallback kept
withdrawn. Peers are checked at /ready, timed, and taken out of routing
while down. Requests to a peer carry x-llmgw-hop and are refused (508) when
they come back to a cluster they passed through; residency and no_store
keep requests off peers; X-Request-Id and, with --send-tags, attribution
tags go along. ctl model fallbacks sets overflow to a peer. Console view,
metrics, admin API routes and docs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 01:21:30 +02:00
1 parent d41104a452
commit 40e9fd2ab1
33 files changed
+2944 -23

No files matched your search

+7
View File
@@ -41,6 +41,13 @@ type Info struct {
// from the catalog's name: an external provider's. Requests are rewritten
// to it. Empty sends the name as the client did.
UpstreamModel string
// Peer names the federated llmgw cluster behind an external endpoint,
// empty for every other. PeerCluster is that cluster's own id, which a
// request that has already passed through it carries in its hop header,
// and PeerTags whether requests sent there name who sent them.
Peer string
PeerCluster string
PeerTags bool
}
// EngineExternal marks an endpoint that is a provider outside the cluster.