Federation: another llmgw cluster as a provider, its models as PEER/MODEL
ctl federation add/list/remove/sync keeps peers in the store (migration 0029); their /v1/models are offered here as external models named PEER/MODEL and kept in step, a model still named by a fallback kept withdrawn. Peers are checked at /ready, timed, and taken out of routing while down. Requests to a peer carry x-llmgw-hop and are refused (508) when they come back to a cluster they passed through; residency and no_store keep requests off peers; X-Request-Id and, with --send-tags, attribution tags go along. ctl model fallbacks sets overflow to a peer. Console view, metrics, admin API routes and docs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d41104a452
commit
40e9fd2ab1
33 files changed
+2944
-23
No files matched your search
@@ -1017,8 +1017,16 @@ type External struct {
|
||||
// rather than the key itself, so the key is in neither the catalog, the
|
||||
// store nor the audit log; empty for a provider that asks for none.
|
||||
APIKeyFile string `json:"api_key_file,omitempty" koanf:"api_key_file"`
|
||||
// Peer names the federated llmgw cluster this model is offered by, for
|
||||
// a model that federation added (`llmgw ctl federation add`) rather
|
||||
// than one written by hand. Such a model is named PEER/MODEL, comes and
|
||||
// goes with the peer's own list, and is sent only while the peer is up.
|
||||
Peer string `json:"peer,omitempty" koanf:"peer"`
|
||||
}
|
||||
|
||||
// IsPeer reports whether a model is offered by a federated cluster.
|
||||
func (m Model) IsPeer() bool { return m.External != nil && m.External.Peer != "" }
|
||||
|
||||
func (e *External) validate(model string, profiles int) error {
|
||||
if profiles > 0 {
|
||||
return fmt.Errorf("model %q: an external model is served elsewhere and has no profiles", model)
|
||||
|
||||
Reference in new issue
Block a user