A job waiting for the card says what holds it and about how long
CI / test (push) Failing after 6m16s
CI / test (push) Failing after 6m16s
A 3D job waiting for the picture model to give up the one card had an empty
progress; its client could not tell waiting from stuck. The control plane
says which model holds the cards and when it lets go, plus the waiting
model's last start; the 503 carries it as headers, and a job keeps it as
progress {state: waiting_gpu, held_by, eta_seconds}.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
cf2edfd995
commit
b1490dbf30
12 files changed
+252
-8
No files matched your search
+7
-1
@@ -2072,7 +2072,13 @@ curl -X POST http://gateway:8080/v1/3d/generations/job_…/cancel -H "Authorizat
|
||||
- **Status:** `queued`, `running`, `completed`, `failed` (with `error`),
|
||||
`cancelling`, `cancelled`, `expired` (not finished within 24 h).
|
||||
`progress` carries what the engine said of the work so far, once it says
|
||||
anything; `GET /v1/3d/generations` lists the caller's jobs.
|
||||
anything; `GET /v1/3d/generations` lists the caller's jobs. While the
|
||||
model waits for another on-demand model to give up the card,
|
||||
`progress` is `{"state":"waiting_gpu","held_by":"flux-schnell","eta_seconds":N}`:
|
||||
N is when the holder is stopped (idle and old enough to evict, or at its
|
||||
idle timeout) plus how long the model's last start took, and is left out
|
||||
when that is not known. A request made directly gets the same as a 503
|
||||
with `X-Llmgw-Waiting-For`, `X-Llmgw-Eta-Seconds` and `Retry-After`.
|
||||
- **Refused at the door:** no key, a model that is not there or not in the
|
||||
key's allowlist, a request the model cannot serve. Refused when served,
|
||||
and said as the job's error: a spent budget, the guardrails, a model
|
||||
|
||||
Reference in new issue
Block a user