A band may wait longer than the rest; a busy refusal says when to return
A knowledge-base import sends far more than the cluster can take at once, and every band waited the same 30 s. A band now has its own queue_ttl, so one for bulk work waits behind people instead of being turned away. And a request turned away because the cluster stayed busy — the queue full (503) or waited as long as its band may (504) — now carries Retry-After: 10. Without it llmgw-kb, which retries only what says when to, counted each such passage as failed; that is what turned parts of the metallurgy import away. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
9b7ceabd5b
commit
497e4b24a1
11 files changed
+113
-5
No files matched your search
+2
-1
@@ -547,7 +547,8 @@ its usual exception with the gateway's message in it. The ones worth knowing:
|
||||
| 401 | No key, a mistyped one, or one that was revoked or has expired. |
|
||||
| 403 | The key may not use that model. |
|
||||
| 429 | A rate limit or budget of the key, its project or its team was reached. `Retry-After` says when to try again, and is absent when waiting will not help — a spent monthly budget. |
|
||||
| 503 | No model of that name is served (the message lists the ones that are), the model is still loading, or every replica is busy and the request waited as long as it may. The SDKs retry by themselves; a name that is not served will not start to be. |
|
||||
| 503 | No model of that name is served (the message lists the ones that are), the model is still loading, or every replica is busy and the queue is full. The SDKs retry by themselves; a name that is not served will not start to be. |
|
||||
| 504 | Every replica was busy for as long as the key's band may wait. `Retry-After` says when to send it again. |
|
||||
|
||||
## Priority
|
||||
|
||||
|
||||
Reference in new issue
Block a user