An external model takes api_key_files, each a file and an optional weight, in place of api_key_file. Each key is an endpoint of its own (external:MODEL#1, #2), so the router's scoring, retries and breakers work per key. A key answering 429 is passed over while its breaker is open. A key answering 401 is passed over too, and external_key_refused is sent, naming the endpoint and never the key; with a single key the 401 is still the client's. A key whose file cannot be read is left out, and usage records which key answered by its endpoint. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
688b205604
commit
fe310576e7
14 files changed
+283
-15
No files matched your search
@@ -131,7 +131,18 @@ const defaultActivationWait = 45 * time.Second
|
||||
// SetNotifier tells the accountant where to say a budget is spent. Nothing
|
||||
// without a store, where there are no budgets.
|
||||
func (r *Router) SetNotifier(n *notify.Notifier) {
|
||||
r.notifier = n
|
||||
if r.acct != nil {
|
||||
r.acct.SetNotifier(n)
|
||||
}
|
||||
}
|
||||
|
||||
// keyRefused says that an outside provider answered 401 to one of a model's
|
||||
// keys, by its endpoint, which numbers it: never the key.
|
||||
func (r *Router) keyRefused(model, endpointID string) {
|
||||
r.log.Warn("an outside provider refused a key", "model", model, "endpoint", endpointID)
|
||||
r.notifier.Notify(notify.ExternalKeyRefused, endpointID,
|
||||
"the provider of "+model+" answered 401 to the key of "+endpointID+
|
||||
": revoked, expired or mistyped; replace its file on the control plane",
|
||||
map[string]any{"model": model, "endpoint": endpointID})
|
||||
}
|
||||
Reference in new issue
Block a user