A 3D model is a job: queued with an id, asked after, its result a file

POST /v1/3d/generations answers 202 with a job; GET says its status,
progress and result file; cancel stops it in the engine's queue too. The
request is kept as sent and replayed by whichever gateway holds the lease
through the path a client's request takes, so it is charged and judged
like any request (tagged job=<id>), and a model still loading is waited
out for as long as the job has. Migration 0044. Nothing makes 3D models
yet; the door refuses every model until the ComfyUI adapter learns meshes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Fable 5.1 committed 2026-10-10 16:04:11 +02:00
1 parent baaaa4e019
commit b51cc3cf92
16 files changed
+1198 -10

No files matched your search

+44
View File
@@ -1672,6 +1672,7 @@ and accounting exactly as an API call does.
| `POST /v1/audio/transcriptions`, `/v1/audio/translations` | the multipart form passed through as it came, to an engine that serves speech to text: vLLM with Whisper |
| `POST /v1/moderations` | answered by the guardrails' judge model (`role: guard`): `flagged`, and the judge's own categories |
| `/v1/files`, `/v1/batches` | batches; see [Batches](#batches) |
| `/v1/3d/generations` | 3D models, as jobs asked after; see [Jobs](#jobs-3d-models) |
| `GET /v1/models` | the models this key may use |
## Watching it
@@ -2030,6 +2031,49 @@ Each line of the file is `{"custom_id": …, "method": "POST", "url":
or past their retention — a project under a guardrails policy with
`no_store` is refused it.
## Jobs: 3D models
A 3D model from a picture takes minutes on a card that first has to be
emptied of whatever else was on it. No client should hold a connection for
that, so it is a job: queued, answered with an id, asked after, its result
downloaded.
```
curl http://gateway:8080/v1/3d/generations -H "Authorization: Bearer $LLMGW_API_KEY" \
-d '{"model": "trellis2", "image": "<base64>"}'
# → 202 {"id": "job_…", "status": "queued", …}
curl http://gateway:8080/v1/3d/generations/job_… -H "Authorization: Bearer $LLMGW_API_KEY"
# → {"status": "completed", "output_file_id": "file-…", "content_type": "model/gltf-binary", "usage": {…}}
curl http://gateway:8080/v1/files/file-…/content -H "Authorization: Bearer $LLMGW_API_KEY" -o model.glb
curl -X POST http://gateway:8080/v1/3d/generations/job_…/cancel -H "Authorization: Bearer $LLMGW_API_KEY"
```
- **Status:** `queued`, `running`, `completed`, `failed` (with `error`),
`cancelling`, `cancelled`, `expired` (not finished within 24 h).
`progress` carries what the engine said of the work so far, once it says
anything; `GET /v1/3d/generations` lists the caller's jobs.
- **Refused at the door:** no key, a model that is not there or not in the
key's allowlist, a request the model cannot serve. Refused when served,
and said as the job's error: a spent budget, the guardrails, a model
that will not come up.
- **How it is served:** the request is kept as it was sent and replayed by
whichever gateway holds the job's lease, through the same path a
client's request takes — limits, admission in the key's band, guardrails,
accounting. It is one usage record under the job's id, tagged
`job=<id>`, charged like any request; `GET` shows the usage once it is
done. A model still loading, a rate limit or a full queue is waited out
for as long as the job has: nobody is holding a connection, which is the
point.
- **Surviving a restart:** a gateway that stops hands its jobs to another
within two minutes; a job it was running is run again from the start.
- **What it keeps:** the request as sent, and the result as a file owned
by the key's project, kept `router.batch.retain_days` (30) like a batch's
files; the job's record goes with it. A project under a `no_store`
policy is refused jobs, as it is batches.
No model makes 3D models yet: until the ComfyUI adapter learns meshes,
every request is refused at the door as "does not make 3D models".
## Conversations across models
Off everywhere until a project asks. llmgw otherwise stores no message content