Federation: another llmgw cluster as a provider, its models as PEER/MODEL
ctl federation add/list/remove/sync keeps peers in the store (migration 0029); their /v1/models are offered here as external models named PEER/MODEL and kept in step, a model still named by a fallback kept withdrawn. Peers are checked at /ready, timed, and taken out of routing while down. Requests to a peer carry x-llmgw-hop and are refused (508) when they come back to a cluster they passed through; residency and no_store keep requests off peers; X-Request-Id and, with --send-tags, attribution tags go along. ctl model fallbacks sets overflow to a peer. Console view, metrics, admin API routes and docs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d41104a452
commit
40e9fd2ab1
33 files changed
+2944
-23
No files matched your search
@@ -0,0 +1,22 @@
|
||||
-- Other llmgw clusters this one federates with (`llmgw ctl federation add`):
|
||||
-- each peer's address, the file on the control plane holding the key this
|
||||
-- cluster uses there, whether requests sent to it carry tags naming who sent
|
||||
-- them, the peer's own cluster id, and the models it last offered.
|
||||
--
|
||||
-- The models are kept, not only fetched, so a catalog that names one of them
|
||||
-- as a fallback is whole at startup, before the peer has been asked again:
|
||||
-- a local model falling back to a peer's would otherwise be skipped as
|
||||
-- naming a model that is not there. Stored as JSON for the same reason a
|
||||
-- runtime model's definition is: a list of what the peer says about each,
|
||||
-- which grows.
|
||||
CREATE TABLE federation_peers (
|
||||
name TEXT PRIMARY KEY,
|
||||
url TEXT NOT NULL,
|
||||
key_file TEXT NOT NULL DEFAULT '',
|
||||
send_tags INTEGER NOT NULL DEFAULT 0,
|
||||
cluster TEXT NOT NULL DEFAULT '',
|
||||
models TEXT NOT NULL DEFAULT '[]',
|
||||
synced_at INTEGER NOT NULL DEFAULT 0,
|
||||
created_at INTEGER NOT NULL,
|
||||
created_by TEXT NOT NULL DEFAULT ''
|
||||
);
|
||||
Reference in new issue
Block a user