Merge C24 federation: another llmgw cluster as a provider
Conflicts with facility-aware placement (config, control, admin API routes, store test tables) all keep both; the generated reference and OpenAPI document regenerated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
commit
33334c8f33
33 files changed
+2945
-24
No files matched your search
@@ -86,6 +86,7 @@ rather than repeated.
|
||||
| `url` | `string` | [External](#external) |
|
||||
| `model` | `string` | [External](#external) |
|
||||
| `api_key_file` | `string` | [External](#external) |
|
||||
| `peer` | `string` | [External](#external) |
|
||||
| `budgets` | `[]FacilityBudget` | [Facility](#facility) |
|
||||
| `margin` | `float64` | [Facility](#facility) |
|
||||
| `gpu_watts` | `float64` | [Facility](#facility) |
|
||||
@@ -95,6 +96,8 @@ rather than repeated.
|
||||
| `domain` | `string` | [FacilityBudget](#facilitybudget) |
|
||||
| `cap_watts` | `float64` | [FacilityBudget](#facilitybudget) |
|
||||
| `other_watts` | `float64` | [FacilityBudget](#facilitybudget) |
|
||||
| `name` | `string` | [Federation](#federation) |
|
||||
| `interval` | `time.Duration` | [Federation](#federation) |
|
||||
| `format` | `string` | [GuardModel](#guardmodel) |
|
||||
| `categories` | `[]string` | [GuardModel](#guardmodel) |
|
||||
| `model` | `string` | [JudgePolicy](#judgepolicy) |
|
||||
@@ -293,6 +296,7 @@ rather than repeated.
|
||||
| `control` | `Control` | [Server](#server) |
|
||||
| `models` | `[]catalog.Model` | [Server](#server) |
|
||||
| `facility` | `Facility` | [Server](#server) |
|
||||
| `federation` | `Federation` | [Server](#server) |
|
||||
| `endpoints` | `[]Endpoint` | [Server](#server) |
|
||||
| `bin` | `string` | [SimConfig](#simconfig) |
|
||||
| `python` | `string` | [TRTLLMConfig](#trtllmconfig) |
|
||||
@@ -863,6 +867,15 @@ APIKeyFile holds the provider's key, on the control plane. A file
|
||||
rather than the key itself, so the key is in neither the catalog, the
|
||||
store nor the audit log; empty for a provider that asks for none.
|
||||
|
||||
### `peer`
|
||||
|
||||
`string`
|
||||
|
||||
Peer names the federated llmgw cluster this model is offered by, for
|
||||
a model that federation added (`llmgw ctl federation add`) rather
|
||||
than one written by hand. Such a model is named PEER/MODEL, comes and
|
||||
goes with the peer's own list, and is sent only while the peer is up.
|
||||
|
||||
## Facility
|
||||
|
||||
### `budgets`
|
||||
@@ -932,6 +945,24 @@ one feed carries alone, derated as the facility derates it (usually
|
||||
OtherWatts is drawn in the domain by what llmgw does not place on:
|
||||
switches, storage, machines outside the cluster.
|
||||
|
||||
## Federation
|
||||
|
||||
### `name`
|
||||
|
||||
`string`
|
||||
|
||||
Name is this cluster as a peer reads it, in the federated_from tag
|
||||
sent with a request to a peer set to receive tags (`ctl federation add
|
||||
--send-tags`). Empty is this machine's host name.
|
||||
|
||||
### `interval`
|
||||
|
||||
`time.Duration`
|
||||
|
||||
Interval is how often each peer is asked whether it is ready, timed,
|
||||
and what it offers. A peer that does not answer is taken out of
|
||||
routing until it does. Zero is 10s.
|
||||
|
||||
## GuardModel
|
||||
|
||||
### `format`
|
||||
@@ -2482,6 +2513,16 @@ maintenance is `llmgw ctl facility hold`, not configuration.
|
||||
|
||||
Takes a [Facility](#facility) block.
|
||||
|
||||
### `federation`
|
||||
|
||||
`Federation`
|
||||
|
||||
Federation is how this cluster takes part in federation with other
|
||||
llmgw clusters. The peers themselves are added at runtime, with
|
||||
`llmgw ctl federation add`, and kept in the store.
|
||||
|
||||
Takes a [Federation](#federation) block.
|
||||
|
||||
### `endpoints`
|
||||
|
||||
`[]Endpoint`
|
||||
|
||||
Reference in new issue
Block a user