v0.14.3: changelog, README, docs for 3D models; a finished job says done
A finished job kept the last word heard from ComfyUI, the node running; it is now said done before the job is marked completed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
2f54780802
commit
7f320c5f65
6 files changed
+142
-10
No files matched your search
+23
-3
@@ -1168,7 +1168,12 @@ files endpoint still serves every file as an octet stream.
|
||||
gateways never both run one job (test with the lease), and the usage
|
||||
row carries the job's price.
|
||||
|
||||
### G2. Progress from ComfyUI (S, after G1)
|
||||
### G2. Progress from ComfyUI (S, after G1) — done 2026-10-10 (branch `jobs-3d`)
|
||||
Built as written: the gate's websocket under its own client id, the
|
||||
client_id of every /prompt rewritten, /progress/{prompt_id} on the gate,
|
||||
the router reading it and /queue into the job (only into a job of the same
|
||||
key). Left: a node that reports no steps (TripoSR's) shows its name and no
|
||||
percent; a vanished replica is said by the job failing, not by progress.
|
||||
The gate (`internal/comfy/gate.go`) passes seven paths and no websocket.
|
||||
Rather than open `/ws`, the gate subscribes to ComfyUI's own `/ws` once,
|
||||
with `golang.org/x/net/websocket` (already a dependency), under a client
|
||||
@@ -1183,7 +1188,19 @@ bounded (the last N prompts) and the websocket reconnects. Done when a
|
||||
`ctl jobs` and on the account page, and a job whose replica vanished
|
||||
says so within one poll.
|
||||
|
||||
### G3. ComfyUI beyond pictures: 3D models and custom nodes (M, after G1)
|
||||
### G3. ComfyUI beyond pictures: 3D models and custom nodes (M, after G1) — done 2026-10-10 (branch `jobs-3d`), with TripoSR
|
||||
Built: output kinds, image upload through the gate, every output list
|
||||
fetched, OBJ to GLB, licence in /v1/models, the job and the GLB, per_mesh
|
||||
and usage.meshes (migration 0045), custom nodes cloned at their commit by
|
||||
the gate with packages beside the replica (a node's named packages at its
|
||||
range, the rest at ComfyUI's versions). Not as written: the model. On the
|
||||
PoC's Turing card TRELLIS.2 cannot run — it needs Meta's gated DINOv3,
|
||||
its published background remover (RMBG-2.0) is non-commercial, and its
|
||||
sparse attention and Direct3D-S2's need flash-attention (Ampere and
|
||||
newer). TripoSR (MIT) proves the chain on cuda01: a .glb from a picture
|
||||
in about 30 s. Left: background removal before TripoSR (BiRefNet, MIT,
|
||||
as a node), TRELLIS.2 or Direct3D-S2 on an Ampere card with an ungated
|
||||
encoder, a missing node refusing the replica by name.
|
||||
- Catalog: a model makes `pictures` or `meshes` (`MakesImages` becomes an
|
||||
output kind; `ServesComfyUI` stays). 3D weight folders in the gate's
|
||||
`modelFolders`; the workflow's placeholders gain `image` (the input
|
||||
@@ -1226,7 +1243,10 @@ says so within one poll.
|
||||
the card is free again after the idle timeout, and the same request
|
||||
against flux-schnell is refused as "makes pictures, not meshes".
|
||||
|
||||
### G4. Portal and console (S, after G2)
|
||||
### G4. Portal and console (S, after G2) — done 2026-10-10 (branch `jobs-3d`)
|
||||
Built: /3d with model-viewer served from the gateway, progress, download
|
||||
and licence; `ctl 3d list|get|cancel` over /api/v1/client-jobs (ctl jobs
|
||||
was taken by the control plane's own work). Left: the console's Jobs view.
|
||||
A 3D page beside `/images` (upload a picture, watch progress, download
|
||||
the .glb, a viewer with `<model-viewer>` from the allowed CDNs), `ctl
|
||||
jobs list|get|cancel`, the console's Jobs view. The page sends down the
|
||||
|
||||
+37
-4
@@ -18,7 +18,7 @@ is only released once its **exit criterion** is met, not when its code lands.
|
||||
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
|
||||
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
||||
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI) |
|
||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs) |
|
||||
| v1.0.0 | 5 | Tenancy, prefix affinity, benchmarking | planned |
|
||||
|
||||
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
|
||||
@@ -26,6 +26,23 @@ APIs change between minors without a deprecation period.
|
||||
|
||||
## Unreleased
|
||||
|
||||
## v0.14.3 — 2026-10-10
|
||||
|
||||
3D models from a picture, as jobs: made by ComfyUI on the cluster's cards,
|
||||
with their progress, their licence written into the mesh, a price per mesh
|
||||
and a page to make them. Also: a profile's own weights, a band for bulk
|
||||
work, and fixes for control planes behind a balancer and agents that
|
||||
restart.
|
||||
|
||||
**Upgrading.**
|
||||
- The store gains tables and columns (migrations 0043–0045: notify_sent,
|
||||
jobs, usage meshes); nothing to do.
|
||||
- **Nodes** need this release for their engines' output to survive an
|
||||
agent restart and for ComfyUI's custom nodes and progress; engines
|
||||
started by an older agent keep its pipe until they are restarted.
|
||||
- A 3D model needs a model that makes them (see install.md, "A model that
|
||||
makes them"); none is added.
|
||||
|
||||
### Added
|
||||
|
||||
- **Jobs: 3D models asked after, not waited for.** `POST
|
||||
@@ -38,9 +55,25 @@ APIs change between minors without a deprecation period.
|
||||
it is charged like any request, tagged `job=<id>`, and a model still
|
||||
loading is waited out for as long as the job has (24 h) rather than
|
||||
refused. A job survives the gateway that took it; results are kept for
|
||||
`router.batch.retain_days`. Migration 0044. No model makes 3D models
|
||||
yet: every request is refused at the door until the ComfyUI adapter
|
||||
learns meshes (BACKLOG G3).
|
||||
`router.batch.retain_days`. Migration 0044.
|
||||
- **ComfyUI models that make meshes** (`comfyui.output: meshes`). The
|
||||
picture is uploaded to the replica's ComfyUI and the workflow's
|
||||
`{{image}}`; the mesh the workflow saves is the job's file, an OBJ turned
|
||||
into a GLB. TripoSR (MIT) is the tested model; on Ampere and newer cards
|
||||
stronger ones can be added the same way.
|
||||
- **Custom nodes** (`comfyui.nodes`): a repository at a full commit, put in
|
||||
each replica's own `custom_nodes` by the gate, with its packages
|
||||
installed beside the replica, never into ComfyUI's environment.
|
||||
- **A job's progress**: the gate listens to ComfyUI's websocket and the job
|
||||
says queued and how many ahead, then the node running, its step and
|
||||
percent.
|
||||
- **A model's `license`** (name, url, notice), in `/v1/models`, in the job,
|
||||
on the portal and in the GLB's `asset.extras`.
|
||||
- **`per_mesh`** in `auth.pricing`; usage rows, sums and exports count
|
||||
`meshes`.
|
||||
- **`/3d` on the portal**, beside `/images`: a picture chosen, the job
|
||||
followed, the mesh in a viewer with a download and its licence; and
|
||||
**`llmgw ctl 3d list|get|cancel`** for every owner's jobs.
|
||||
- **A profile's own `source`**: the weights it loads instead of the
|
||||
model's, so an embedding model can run on vLLM from its repository on a
|
||||
card and on llama.cpp from a GGUF on CPUs as one model. Placement prefers
|
||||
|
||||
@@ -40,7 +40,7 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md).
|
||||
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
|
||||
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
||||
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI) |
|
||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs) |
|
||||
| v1.0.0 | 5 | Tenancy, prefix affinity, benchmarking | planned |
|
||||
|
||||
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
|
||||
|
||||
+74
-2
@@ -2092,8 +2092,80 @@ curl -X POST http://gateway:8080/v1/3d/generations/job_…/cancel -H "Authorizat
|
||||
files; the job's record goes with it. A project under a `no_store`
|
||||
policy is refused jobs, as it is batches.
|
||||
|
||||
No model makes 3D models yet: until the ComfyUI adapter learns meshes,
|
||||
every request is refused at the door as "does not make 3D models".
|
||||
A request may also send `seed` and `resolution` (the model's own grid, a
|
||||
multiple of 32 from 64 to 2048). The picture should show one object on a
|
||||
plain background: the model makes a mesh of what it sees, background
|
||||
included.
|
||||
|
||||
### A model that makes them
|
||||
|
||||
A 3D model is made by a ComfyUI model whose `comfyui.output` is `meshes`
|
||||
(see [Pictures from ComfyUI](#pictures-from-comfyui) for engine `comfyui`).
|
||||
Its workflow is a JSON file in ComfyUI's API format: `{{image}}` is the
|
||||
uploaded picture, which a `LoadImage` node reads; `{{seed}}`,
|
||||
`{{resolution|256}}` and `{{checkpoint}}` are as for pictures. The node that
|
||||
saves the mesh may save a GLB or an OBJ; an OBJ leaves the gateway as a GLB,
|
||||
vertex colours kept. TripoSR (MIT) on any card of 8 GB:
|
||||
|
||||
```yaml
|
||||
- name: triposr
|
||||
source: hf:stabilityai/TripoSR/model.ckpt
|
||||
comfyui:
|
||||
workflow: /etc/llmgw/workflows/triposr.json # on every control plane
|
||||
output: meshes
|
||||
nodes:
|
||||
- repo: https://github.com/flowtyone/ComfyUI-Flowty-TripoSR
|
||||
commit: a0c94ac60a7cc062604f61aeeea6d0d493521de3
|
||||
requirements: [omegaconf, trimesh, scikit-image, imageio,
|
||||
"transformers>=4.50.3,<5", "huggingface-hub<1", tokenizers]
|
||||
license:
|
||||
name: MIT
|
||||
url: https://huggingface.co/stabilityai/TripoSR/blob/main/LICENSE
|
||||
notice: "Made with TripoSR (Stability AI and Tripo AI, MIT). ..."
|
||||
scaling: {min: 0, max: 1, idle_timeout: 10m}
|
||||
profiles:
|
||||
- {name: gpu, engine: comfyui, gpu_class: 6000-24g, tp: 1}
|
||||
```
|
||||
|
||||
```json
|
||||
{
|
||||
"1": {"class_type": "LoadImage", "inputs": {"image": "{{image}}"}},
|
||||
"2": {"class_type": "TripoSRModelLoader", "inputs": {"model": "{{checkpoint}}", "chunk_size": 8192}},
|
||||
"3": {"class_type": "TripoSRSampler", "inputs": {"model": ["2", 0], "reference_image": ["1", 0],
|
||||
"geometry_resolution": "{{resolution|256}}", "threshold": 25.0}},
|
||||
"4": {"class_type": "TripoSRViewer", "inputs": {"mesh": ["3", 0]}}
|
||||
}
|
||||
```
|
||||
|
||||
- **Custom nodes** are named by https repository and full commit (a branch
|
||||
is refused: it moves). The gate puts each in the replica's own
|
||||
`custom_nodes` and installs its `requirements` beside the replica, never
|
||||
into ComfyUI's environment. A package named there is installed at the
|
||||
range named; every other one at the version ComfyUI already runs. A
|
||||
package named is first on ComfyUI's path, so its range must suit ComfyUI
|
||||
as well. TripoSR's checkpoint loads with transformers 4, which ComfyUI's
|
||||
`>=4.50.3` allows.
|
||||
- **Progress:** the gate listens to ComfyUI's websocket and the router
|
||||
writes what it hears into the job: `{"state": "queued", "ahead": 2}`,
|
||||
then `{"state": "running", "node_class": "KSampler", "step": 12,
|
||||
"steps": 30, "percent": 40}` for a node that counts its steps, and
|
||||
`done` at the end.
|
||||
- **Licence:** a model's `license` is shown in `/v1/models`, in the job and
|
||||
on the portal, and written into the GLB's `asset.extras.license`, since
|
||||
the file travels without the API. Choose the model for its licence before
|
||||
its quality when what it makes is shipped. TRELLIS.2 needs Meta's gated
|
||||
DINOv3 and, as published, a background remover under a non-commercial
|
||||
licence; Hunyuan3D's licence excludes the EU, the UK and South Korea.
|
||||
- **Price:** `per_mesh` in `auth.pricing`, beside `per_image`; each usage
|
||||
row says how many meshes it made (`meshes`).
|
||||
- **Cards:** the strongest models today (TRELLIS.2, Direct3D-S2) need
|
||||
flash-attention, which runs on Ampere and newer. A Turing card such as
|
||||
the RTX 6000 runs TripoSR.
|
||||
|
||||
People make them at `/3d` on the portal: they choose a picture and watch
|
||||
the job, and the mesh is shown in a viewer (model-viewer, served from the
|
||||
gateway) with a download link and the licence. `llmgw ctl 3d list`, `get`
|
||||
and `cancel` show and stop every owner's jobs.
|
||||
|
||||
## Conversations across models
|
||||
|
||||
|
||||
@@ -202,6 +202,10 @@ func (r *Router) serveJob(ctx context.Context, j *store.Job) error {
|
||||
if err := r.store.PutFile(ctx, f, w.buf.Bytes()); err != nil {
|
||||
return err
|
||||
}
|
||||
// Said done while it may still be said: the last word heard was the
|
||||
// node running, and a finished job saying it is running reads as
|
||||
// one that stopped halfway.
|
||||
_ = r.store.SetJobProgress(ctx, j.ID, `{"state":"done","percent":100}`, "")
|
||||
j.Status, j.OutputFileID, j.ContentType, j.FinishedAt = store.JobCompleted, f.ID, ct, done
|
||||
return r.store.UpdateJob(ctx, *j)
|
||||
}
|
||||
|
||||
@@ -161,6 +161,9 @@ func TestA3DModelIsMadeFromAPictureWithItsLicenceInside(t *testing.T) {
|
||||
if got["status"] != "completed" || got["content_type"] != "model/gltf-binary" {
|
||||
t.Fatalf("job %v", got)
|
||||
}
|
||||
if p, _ := got["progress"].(map[string]any); p["state"] != "done" {
|
||||
t.Errorf("a finished job says %v", got["progress"])
|
||||
}
|
||||
if lic, _ := got["license"].(map[string]any); lic["name"] != "MIT" {
|
||||
t.Errorf("the job does not say the licence: %v", got["license"])
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user