From 751533f853509d0f64a39d7cb13250396885b0c1 Mon Sep 17 00:00:00 2001 From: Ilker Manap Date: Fri, 9 Oct 2026 21:26:31 +0200 Subject: [PATCH] Docs: pictures from ComfyUI Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/install.md | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 45 insertions(+) diff --git a/docs/install.md b/docs/install.md index 69a6c34..8a72c5c 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1270,6 +1270,51 @@ prices and budgets, rate limits, usage records, and key allowlists. whose allowlist leaves the external model out cannot reach it, as a fallback or by name. The cluster page and `ctl models` mark it `external`. +### Pictures from ComfyUI + +A ComfyUI server is an external model with `api: comfyui`. Clients ask it for +pictures at OpenAI's `/v1/images/generations`; llmgw fills a ComfyUI workflow +with the request, queues it, waits, and sends the pictures back as `b64_json`: + +```yaml +- name: flux-schnell + source: external + external: + url: http://10.0.0.147:8188 # ComfyUI, started with --listen + model: flux1-schnell-fp8.safetensors # the checkpoint, as ComfyUI lists it + api: comfyui + workflow: flux-schnell # or flux-dev, or a JSON file +``` + +`flux-schnell` (the default) and `flux-dev` are built in. They suit the +single-file FLUX checkpoints that ComfyUI loads with `CheckpointLoaderSimple`. +For any other model, export the workflow from ComfyUI with **Save (API)** and +name the file. In the file, put placeholders where the request's values go: + +- `{{prompt}}` and `{{negative_prompt}}` are the prompts. +- `{{width}}` and `{{height}}` are the size. +- `{{n}}` is the batch size. +- `{{seed}}` is the seed. +- `{{steps|4}}` and `{{guidance|3.5}}` take the value after the `|` when the + request leaves them out. +- `{{checkpoint}}` is the `model` above. + +A request is OpenAI's: `prompt`, `n` (at most 4), `size` (each side a multiple +of 16, from 256 to 2048) and `response_format: b64_json`. Beyond those, it may +send `seed`, `steps`, `negative_prompt` and `guidance`. The seed used comes +back in `x-llmgw-seed`. `url` is not offered, because pictures are kept +nowhere. ComfyUI also writes the workflow into each PNG; llmgw strips it +before the picture is sent. + +Requests for pictures pass the same checks as any other request: keys, +allowlists, limits, admission, and the guardrails on the prompt. A client that +leaves takes its workflow off the ComfyUI queue. Usage is recorded, but for +now it is charged as the prompt's tokens; a per-picture price is not yet +available. + +Start ComfyUI with `--disable-smart-memory` on a card that llmgw also places +models on. Without it, ComfyUI keeps the model on the card after each picture. + ### Federating clusters Another llmgw cluster — another site, another team's GPUs — can be a provider