flux and sf3d on one card handed it to each other, each starting ComfyUI
and loading its weights from nothing (BACKLOG H3, placement side). An
environment of two or more models now gets a hidden catalog entry,
comfyui-env-<name>: on its first member's profiles with the most memory
any member needs, on demand for the longest idle timeout, carrying every
member's weights, repositories and the environment's custom nodes. The
members get no replicas of their own; asking for one wants the entry.
The agent fetches each member's weights; the gate links all of them,
refusing two weights files of one name, links each member's Hugging Face
repository, and answers /v1/models for every member. The control plane
offers the replica under the members' names with each one's workflow and
checkpoint, which the router sends by the model asked for. A member's
sibling never evicts the replica serving it, and the members' load counts
as the entry's.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Stable Fast 3D's texture_baker and uv_unwrapper are C++/CUDA packages no
index has; on the PoC they were built and installed into ComfyUI's
environment by hand. nodes[].build names such directories, and the gate
builds each as a wheel with ComfyUI's Python, for the compute capability
torch reports for the replica's own card, keeps it on the node by the
node's commit, the architecture and the directory, and installs it
beside the replica with the node's requirements. A copy in ComfyUI's
environment does not pin it. A node with no CUDA toolkit says so.
comfyui.huggingface_repo shows the replica's weights to a node that
loads them by repository name, in the engines' Hugging Face cache, so a
gated repository needs no token on the node.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
LLMGWPrepareForMesh, llmgw's own ComfyUI node, written by the gate into
every replica's custom_nodes from the binary: TripoSR's own preparation
(tsr/utils.py) — the background taken away, by the picture's alpha where
it has one and by rembg (MIT; u2net, Apache 2.0) where it does not, the
object cropped to its bounds, set square at a ratio of the side (0.85),
on grey. A picture sent as it was had a white or black shell round the
object, whatever was behind it made into the mesh.
/v1/models listed only models with a replica up, so one that loads when
asked — triposr, flux-schnell — was invisible to the clients that would
have asked for it. Those are listed now, with loaded: false; every entry
says loaded.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
G2: the gate keeps a websocket open to its ComfyUI under a client id of its
own, puts that id on every workflow it lets through — ComfyUI tells only
the client that queued a workflow how it goes — and keeps the last word on
each: the node running, its step of how many. It answers at
/progress/{prompt_id}. While a job's request waits, the router reads the
queue (queued, n ahead) and the gate (running, node, step, percent) and
writes it into the job, which a client reads back. Only into a job of the
same key: a tag naming another's job writes nothing.
G3, the rest of the plumbing: comfyui.nodes names custom nodes by https
repository and full commit, with the PyPI packages they need. The gate puts
each in the replica's own custom_nodes at that commit and installs the
packages beside the replica, constrained to the versions ComfyUI's
environment already has, which it never changes. The catalog refuses a
branch for a commit and anything but package specs for requirements. An
OBJ a node saves leaves the gateway as a GLB, colours kept, so every mesh
can carry its licence.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
G3, the part that needs no model: a ComfyUI model's comfyui.output (or
external.output) says pictures or meshes. A request for a 3D model — a
job at /v1/3d/generations — carries the picture as base64; at the replica
it is uploaded to ComfyUI's input directory (the gate lets /upload/image
through), the workflow is filled in with its name, the seed and the
resolution, and the mesh the workflow saves comes back with its type,
which is the file the job keeps. comfy.Client reads every list of files an
output names, not "images" only, and types a file by its extension where
ComfyUI sends octet-stream.
A model's license (name, url, notice) is said in /v1/models and in the
job, and written into a GLB's asset.extras, since the file travels without
the API; a mesh it cannot be written into is refused. per_mesh in
auth.pricing, usage.meshes in the rows, the sums and the export
(migration 0045). A picture asked of a mesh model, or a mesh of a picture
model, is refused saying which it makes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ComfyUI wrote straight to the agent's pipe. An agent that restarts and
adopts the replica leaves that pipe with no reader, and ComfyUI's next
progress bar failed with a broken pipe, failing the workflow. It now
writes to a pipe of the gate's, which reads for as long as ComfyUI writes
and passes on what it can.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ComfyUI lists its base directory's custom_nodes before anything else and
exits when there is none.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A model whose profiles run engine comfyui is placed on a node's card like
any other, and makes pictures there. ComfyUI asks for no key, so the agent
starts it behind llmgw comfyui-gate: ComfyUI on the loopback, the gate on
the replica's port demanding the replica's key and letting through only
what the router asks — queueing, waiting, fetching, cancelling. The gate
answers the agent's /health and /v1/models, shows ComfyUI the fetched
weights by their own names, and gives each replica its own base directory.
ComfyUI runs as the gate's child, so its card memory is the replica's.
A node offers the engine when agent.yaml names its ComfyUI
(engines.comfyui.dir). The model's comfyui: block names the workflow and,
for a source that is a set of files, the checkpoint.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>