A 3D model is a job, asked after, not a request held open

Section G of the backlog: a jobs table of its own with a leased worker,
progress read from ComfyUI's websocket by the gate, the ComfyUI adapter
widened from pictures to meshes with custom-node installs, and the pages.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Fable 5.1 committed 2026-10-10 14:57:13 +02:00
1 parent 9b7ceabd5b
commit 5ed18cc4c4
1 file changed
+87
+87
View File
@@ -1101,6 +1101,93 @@ at a third of its CPUs.
28 (MRR 0.738 → 0.789 → 0.838). Next: translating the question with the
chat model, and a reranker.
## G. Jobs: 3D generation through ComfyUI (2026-10-10)
A request for a 3D model (Hunyuan3D 2 first; TRELLIS later if its quality
is wanted) takes minutes on the one 24 GB card, and the card has to be
emptied of the chat model first. That is not a request to hold a socket for
— `internal/router/activate.go:46` already says so — it is a job: queued,
answered with an id, asked after, its result downloaded. The job machinery
is built first and tested without a GPU; the ComfyUI generalisation and the
custom-node install come after. Decided 2026-10-10: a `jobs` table of its
own rather than a one-line batch (the JSONL shape would be artificial and
has nowhere for progress), and progress is shown, not just a status.
### G1. Jobs: store, worker, endpoints (M)
- Store: `jobs` (id, owner key/principal, kind, model, status, input
(JSON), progress (JSON: queue position, percent, step), result file id,
error, created/started/finished, lease owner/until), migrations for
sqlite and postgres beside `internal/store/batches.go`, whose
claim/renew/release lease pattern is copied, not shared: a job is one
unit of work, a batch is many lines.
- Worker: `internal/router/jobworker.go` after `batchworker.go`. Any
gateway on the store claims a queued job, asks for the model with
`activator.Want` and waits as long as it takes (no `ActivationWait`
bound: nobody is holding a connection), runs it, stores the output
through the files store, writes usage and price, marks it done. A
gateway that dies mid-job loses its lease and another one picks it up;
the ComfyUI prompt id is in the job row so the second gateway asks
after the same work rather than queueing it again.
- Endpoints: `POST /v1/3d/generations` validates like any inference
request (key, allowlist, limits, admission, guardrails on the prompt),
writes the job, answers 202 with `{id, status: "queued"}`;
`GET /v1/3d/generations/{id}` returns status, progress and, when done,
`output_file_id` and the usage; `POST .../{id}/cancel`; the file comes
from `GET /v1/files/{id}/content`. Owner-scoped like batches. Results
are kept for the batch files' retention and deleted with them: a
departure from `images.go`'s "kept nowhere", because a mesh is a
download, not a JSON field; the retention is the bound.
- Done when: a job survives the gateway that took it being killed, a
cancelled job leaves ComfyUI's queue (`comfy.Client.cancel`), two
gateways never both run one job (test with the lease), and the usage
row carries the job's price.
### G2. Progress from ComfyUI (S, after G1)
The gate (`internal/comfy/gate.go`) passes seven paths and no websocket.
Rather than open `/ws`, the gate subscribes to ComfyUI's own `/ws` once,
with `golang.org/x/net/websocket` (already a dependency), under a client
id of its own, and rewrites the `client_id` of every `/prompt` it
forwards to that one. It keeps the last `executing`, `progress`
(value/max/node) and `execution_error` message per prompt id and serves
them at `GET /progress/{prompt_id}` on the gate; the router polls that
beside `/history` and `/queue` and writes queue position, percent and the
current node's class into the job's progress. The gate's buffer is
bounded (the last N prompts) and the websocket reconnects. Done when a
3D job shows "queued, 2 ahead" then "running 40%, step 12/30" in
`ctl jobs` and on the account page, and a job whose replica vanished
says so within one poll.
### G3. ComfyUI beyond pictures: 3D models and custom nodes (M, after G1)
- Catalog: a model makes `pictures` or `meshes` (`MakesImages` becomes an
output kind; `ServesComfyUI` stays). 3D weight folders in the gate's
`modelFolders`; the workflow's placeholders gain `image` (the input
picture, base64 in the request, written to the replica's `input/` by
the gate through a new `POST /upload/image` the gate allows — or a
base64-loading node, whichever Hunyuan3D's wrapper wants).
- `comfy.Client.fetch` reads every output key, not `images` only, and
fetches each file named by `/view`; `Image` becomes `Artifact` with its
content type (model/gltf-binary, application/octet-stream for .obj).
- Custom nodes: a model (or the engine config) names git repositories
and a pip requirements list; the agent's provision step clones them
into the replica's `custom_nodes/` (the gate's `--base-directory` is
where ComfyUI looks, not the install directory) and installs the
requirements into the engine's venv with the same PyTorch index and
lock discipline as A8. A replica with a missing node refuses to start
with the node's name in the error rather than ComfyUI's traceback.
- Pricing: `per_image` stays; `per_mesh` beside it in `accounting.Price`,
`Outcome.Meshes`, the usage sums and exports (as eaea671 did for
pictures).
- Done when: Hunyuan3D 2 on cuda01 with `scaling: {min: 0,
idle_timeout: 10m}` makes a .glb from a picture through G1's endpoint,
the card is free again after the idle timeout, and the same request
against flux-schnell is refused as "makes pictures, not meshes".
### G4. Portal and console (S, after G2)
A 3D page beside `/images` (upload a picture, watch progress, download
the .glb, a viewer with `<model-viewer>` from the allowed CDNs), `ctl
jobs list|get|cancel`, the console's Jobs view. The page sends down the
same path as the API, like `imagesSend` does.
## D. Facts worth knowing before starting
- Admin API: `127.0.0.1:9090`, no auth, `ctlClient` in `internal/cli/ctl.go`.