Several keys for one outside provider
CI / test (push) Successful in 8m52s

An external model takes api_key_files, each a file and an optional weight,
in place of api_key_file. Each key is an endpoint of its own
(external:MODEL#1, #2), so the router's scoring, retries and breakers work
per key. A key answering 429 is passed over while its breaker is open. A
key answering 401 is passed over too, and external_key_refused is sent,
naming the endpoint and never the key; with a single key the 401 is still
the client's. A key whose file cannot be read is left out, and usage
records which key answered by its endpoint.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-17 08:56:30 +02:00
1 parent 688b205604
commit fe310576e7
14 files changed
+283 -15

No files matched your search

+11
View File
@@ -131,7 +131,18 @@ const defaultActivationWait = 45 * time.Second
// SetNotifier tells the accountant where to say a budget is spent. Nothing
// without a store, where there are no budgets.
func (r *Router) SetNotifier(n *notify.Notifier) {
r.notifier = n
if r.acct != nil {
r.acct.SetNotifier(n)
}
}
// keyRefused says that an outside provider answered 401 to one of a model's
// keys, by its endpoint, which numbers it: never the key.
func (r *Router) keyRefused(model, endpointID string) {
r.log.Warn("an outside provider refused a key", "model", model, "endpoint", endpointID)
r.notifier.Notify(notify.ExternalKeyRefused, endpointID,
"the provider of "+model+" answered 401 to the key of "+endpointID+
": revoked, expired or mistyped; replace its file on the control plane",
map[string]any{"model": model, "endpoint": endpointID})
}