Docs: pictures from ComfyUI
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
79a503f824
commit
751533f853
1 file changed
+45
@@ -1270,6 +1270,51 @@ prices and budgets, rate limits, usage records, and key allowlists.
|
||||
whose allowlist leaves the external model out cannot reach it, as a fallback or
|
||||
by name. The cluster page and `ctl models` mark it `external`.
|
||||
|
||||
### Pictures from ComfyUI
|
||||
|
||||
A ComfyUI server is an external model with `api: comfyui`. Clients ask it for
|
||||
pictures at OpenAI's `/v1/images/generations`; llmgw fills a ComfyUI workflow
|
||||
with the request, queues it, waits, and sends the pictures back as `b64_json`:
|
||||
|
||||
```yaml
|
||||
- name: flux-schnell
|
||||
source: external
|
||||
external:
|
||||
url: http://10.0.0.147:8188 # ComfyUI, started with --listen
|
||||
model: flux1-schnell-fp8.safetensors # the checkpoint, as ComfyUI lists it
|
||||
api: comfyui
|
||||
workflow: flux-schnell # or flux-dev, or a JSON file
|
||||
```
|
||||
|
||||
`flux-schnell` (the default) and `flux-dev` are built in. They suit the
|
||||
single-file FLUX checkpoints that ComfyUI loads with `CheckpointLoaderSimple`.
|
||||
For any other model, export the workflow from ComfyUI with **Save (API)** and
|
||||
name the file. In the file, put placeholders where the request's values go:
|
||||
|
||||
- `{{prompt}}` and `{{negative_prompt}}` are the prompts.
|
||||
- `{{width}}` and `{{height}}` are the size.
|
||||
- `{{n}}` is the batch size.
|
||||
- `{{seed}}` is the seed.
|
||||
- `{{steps|4}}` and `{{guidance|3.5}}` take the value after the `|` when the
|
||||
request leaves them out.
|
||||
- `{{checkpoint}}` is the `model` above.
|
||||
|
||||
A request is OpenAI's: `prompt`, `n` (at most 4), `size` (each side a multiple
|
||||
of 16, from 256 to 2048) and `response_format: b64_json`. Beyond those, it may
|
||||
send `seed`, `steps`, `negative_prompt` and `guidance`. The seed used comes
|
||||
back in `x-llmgw-seed`. `url` is not offered, because pictures are kept
|
||||
nowhere. ComfyUI also writes the workflow into each PNG; llmgw strips it
|
||||
before the picture is sent.
|
||||
|
||||
Requests for pictures pass the same checks as any other request: keys,
|
||||
allowlists, limits, admission, and the guardrails on the prompt. A client that
|
||||
leaves takes its workflow off the ComfyUI queue. Usage is recorded, but for
|
||||
now it is charged as the prompt's tokens; a per-picture price is not yet
|
||||
available.
|
||||
|
||||
Start ComfyUI with `--disable-smart-memory` on a card that llmgw also places
|
||||
models on. Without it, ComfyUI keeps the model on the card after each picture.
|
||||
|
||||
### Federating clusters
|
||||
|
||||
Another llmgw cluster — another site, another team's GPUs — can be a provider
|
||||
|
||||
Reference in new issue
Block a user