A job waiting for the card says what holds it and about how long
CI / test (push) Failing after 6m16s

A 3D job waiting for the picture model to give up the one card had an empty
progress; its client could not tell waiting from stuck. The control plane
says which model holds the cards and when it lets go, plus the waiting
model's last start; the 503 carries it as headers, and a job keeps it as
progress {state: waiting_gpu, held_by, eta_seconds}.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-10 22:25:27 +02:00
1 parent cf2edfd995
commit b1490dbf30
12 files changed
+252 -8

No files matched your search

+7 -1
View File
@@ -2072,7 +2072,13 @@ curl -X POST http://gateway:8080/v1/3d/generations/job_…/cancel -H "Authorizat
- **Status:** `queued`, `running`, `completed`, `failed` (with `error`),
`cancelling`, `cancelled`, `expired` (not finished within 24 h).
`progress` carries what the engine said of the work so far, once it says
anything; `GET /v1/3d/generations` lists the caller's jobs.
anything; `GET /v1/3d/generations` lists the caller's jobs. While the
model waits for another on-demand model to give up the card,
`progress` is `{"state":"waiting_gpu","held_by":"flux-schnell","eta_seconds":N}`:
N is when the holder is stopped (idle and old enough to evict, or at its
idle timeout) plus how long the model's last start took, and is left out
when that is not known. A request made directly gets the same as a 503
with `X-Llmgw-Waiting-For`, `X-Llmgw-Eta-Seconds` and `Retry-After`.
- **Refused at the door:** no key, a model that is not there or not in the
key's allowlist, a request the model cannot serve. Refused when served,
and said as the job's error: a spent budget, the guardrails, a model