Docs: pictures from ComfyUI

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-09 21:26:31 +02:00
1 parent 79a503f824
commit 751533f853
1 file changed
+45
+45
View File
@@ -1270,6 +1270,51 @@ prices and budgets, rate limits, usage records, and key allowlists.
whose allowlist leaves the external model out cannot reach it, as a fallback or
by name. The cluster page and `ctl models` mark it `external`.
### Pictures from ComfyUI
A ComfyUI server is an external model with `api: comfyui`. Clients ask it for
pictures at OpenAI's `/v1/images/generations`; llmgw fills a ComfyUI workflow
with the request, queues it, waits, and sends the pictures back as `b64_json`:
```yaml
- name: flux-schnell
source: external
external:
url: http://10.0.0.147:8188 # ComfyUI, started with --listen
model: flux1-schnell-fp8.safetensors # the checkpoint, as ComfyUI lists it
api: comfyui
workflow: flux-schnell # or flux-dev, or a JSON file
```
`flux-schnell` (the default) and `flux-dev` are built in. They suit the
single-file FLUX checkpoints that ComfyUI loads with `CheckpointLoaderSimple`.
For any other model, export the workflow from ComfyUI with **Save (API)** and
name the file. In the file, put placeholders where the request's values go:
- `{{prompt}}` and `{{negative_prompt}}` are the prompts.
- `{{width}}` and `{{height}}` are the size.
- `{{n}}` is the batch size.
- `{{seed}}` is the seed.
- `{{steps|4}}` and `{{guidance|3.5}}` take the value after the `|` when the
request leaves them out.
- `{{checkpoint}}` is the `model` above.
A request is OpenAI's: `prompt`, `n` (at most 4), `size` (each side a multiple
of 16, from 256 to 2048) and `response_format: b64_json`. Beyond those, it may
send `seed`, `steps`, `negative_prompt` and `guidance`. The seed used comes
back in `x-llmgw-seed`. `url` is not offered, because pictures are kept
nowhere. ComfyUI also writes the workflow into each PNG; llmgw strips it
before the picture is sent.
Requests for pictures pass the same checks as any other request: keys,
allowlists, limits, admission, and the guardrails on the prompt. A client that
leaves takes its workflow off the ComfyUI queue. Usage is recorded, but for
now it is charged as the prompt's tokens; a per-picture price is not yet
available.
Start ComfyUI with `--disable-smart-memory` on a card that llmgw also places
models on. Without it, ComfyUI keeps the model on the card after each picture.
### Federating clusters
Another llmgw cluster — another site, another team's GPUs — can be a provider