Several keys for one outside provider
CI / test (push) Successful in 8m52s

An external model takes api_key_files, each a file and an optional weight,
in place of api_key_file. Each key is an endpoint of its own
(external:MODEL#1, #2), so the router's scoring, retries and breakers work
per key. A key answering 429 is passed over while its breaker is open. A
key answering 401 is passed over too, and external_key_refused is sent,
naming the endpoint and never the key; with a single key the 401 is still
the client's. A key whose file cannot be read is left out, and usage
records which key answered by its endpoint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-17 08:56:30 +02:00
1 parent 688b205604
commit fe310576e7
14 files changed
+283 -15

No files matched your search

+6 -1
View File
@@ -43,10 +43,15 @@ const (
// CanaryRolledBack is a model's canary taken out because it served
// worse than the model it was a candidate to replace.
CanaryRolledBack = "canary_rolled_back"
// ExternalKeyRefused is an outside provider answering 401 to one of a
// model's keys: revoked, expired or mistyped. Requests go to its other
// keys meanwhile, where it has any.
ExternalKeyRefused = "external_key_refused"
)
// Kinds are every event there is.
var Kinds = []string{ModelParked, NodeLost, BudgetSpent, UpdateAvailable, JudgeUnavailable, CanaryRolledBack}
var Kinds = []string{ModelParked, NodeLost, BudgetSpent, UpdateAvailable, JudgeUnavailable, CanaryRolledBack,
ExternalKeyRefused}
// Event is one thing that happened.
type Event struct {