Several keys for one outside provider
An external model takes api_key_files, each a file and an optional weight, in place of api_key_file. Each key is an endpoint of its own (external:MODEL#1, #2), so the router's scoring, retries and breakers work per key. A key answering 429 is passed over while its breaker is open. A key answering 401 is passed over too, and external_key_refused is sent, naming the endpoint and never the key; with a single key the 401 is still the client's. A key whose file cannot be read is left out, and usage records which key answered by its endpoint. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
688b205604
commit
fe310576e7
14 files changed
+283
-15
No files matched your search
@@ -43,10 +43,15 @@ const (
|
||||
// CanaryRolledBack is a model's canary taken out because it served
|
||||
// worse than the model it was a candidate to replace.
|
||||
CanaryRolledBack = "canary_rolled_back"
|
||||
// ExternalKeyRefused is an outside provider answering 401 to one of a
|
||||
// model's keys: revoked, expired or mistyped. Requests go to its other
|
||||
// keys meanwhile, where it has any.
|
||||
ExternalKeyRefused = "external_key_refused"
|
||||
)
|
||||
|
||||
// Kinds are every event there is.
|
||||
var Kinds = []string{ModelParked, NodeLost, BudgetSpent, UpdateAvailable, JudgeUnavailable, CanaryRolledBack}
|
||||
var Kinds = []string{ModelParked, NodeLost, BudgetSpent, UpdateAvailable, JudgeUnavailable, CanaryRolledBack,
|
||||
ExternalKeyRefused}
|
||||
|
||||
// Event is one thing that happened.
|
||||
type Event struct {
|
||||
|
||||
Reference in new issue
Block a user