Pictures, 3D models, a vision model and workflows as tools at /mcp

An agent driving Blender made its concepts, meshes and render reviews
through a tool server of the game's own, which called this gateway; the
gateway's MCP server offered only the memory notes. It now offers
generate_image, generate_mesh, review_images, run_workflow, job_status and
gateway_status beside them, each only where a model for it is here.

Each tool makes its request of the client API as the caller, through the
handler Handler built, key check and all: allowlists, limits, licences
left out, budgets and accounting apply as to any request, and a job is
the caller's. What a tool makes is kept as the caller's file and answered
by id and download URL, since a server over HTTP cannot write to the
caller's disk; a picture comes with a 512 px preview, and whole where no
file can be kept. A picture to work from is a file id, a data URL or
base64; one sent to the vision model is shrunk to 1280 px.

A mesh takes minutes. A client that takes a stream is answered as
server-sent events, with notifications/progress for its token while the
job is queued, waiting for the GPU and running, and waits up to ten
minutes; one that does not waits 50 seconds and gets the job id, which
job_status asks after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-14 13:11:09 +02:00
1 parent 7c53380e68
commit 55d5e13534
9 files changed
+1314 -10

No files matched your search

+1 -1
View File
@@ -82,7 +82,7 @@
"/mcp": {
"post": {
"operationId": "postMcp",
"summary": "The gateway's MCP server: the memory notes as tools",
"summary": "The gateway's MCP server: the memory notes, and pictures, 3D models, vision and workflows, as tools",
"tags": [
"mcp"
],
+36
View File
@@ -534,6 +534,42 @@ curl https://llm.example.com/mcp \
-d '{"jsonrpc": "2.0", "id": 1, "method": "tools/list"}'
```
**Pictures, 3D models, vision and workflows.** Where the gateway has the
models for them, the same server offers more tools — only those it has a
model for:
| Tool | Does |
|---|---|
| `generate_image` | A picture from a prompt. `style` adds what a picture for a mesh wants: `concept` (one object, three-quarter view, grey background), `side` (an exact side view), `texture` (a seamless tile) or `none`. Answers the PNG's `file_id`, `url` and `seed`, with a 512 px preview to look at. |
| `generate_mesh` | A GLB from a picture — `image` is a `file-…` id (a `generate_image` result, say), a data URL or base64 — or from a `prompt`, for which a concept picture is made first. `seed`, `resolution`, `texture_resolution`, and any other input the model's workflow declares in `inputs`. Answers the GLB's `file_id`, `url` and licence. |
| `review_images` | Asks a vision model about one to four pictures (each shrunk to 1280 px on its long side); `json_schema` makes it answer that JSON. |
| `run_workflow` | Runs a ComfyUI workflow from the inputs its catalog entry declares, and answers its result file. |
| `job_status` | Where a mesh or a workflow run stands, and its file when done; `wait_seconds` waits for it. |
| `gateway_status` | The picture, 3D, vision and workflow models, whether each is loaded on a GPU now, a workflow's inputs, and your last jobs. |
Each tool makes its request of the gateway under your key, so your
allowlist, limits, left-out licences and budget apply, and it is charged
and listed like any other request. What it makes is kept as your file for
as long as job results are: fetch it from `url` with the same key (`curl -H
"Authorization: Bearer $LLMGW_API_KEY" -o rock.glb <url>`). A tool server
over HTTP cannot write to your disk, so an agent that needs the file
locally — to import it in Blender — fetches it, or goes through a small
local bridge that does.
A mesh takes from half a minute to a few minutes. A client that takes the
answer as a stream (`Accept: text/event-stream`, as Claude Code does) is
answered that way: `notifications/progress` for the call's
`progressToken` while it runs — queued, waiting for the GPU, the node and
its percent — then the result, and the call waits up to ten minutes. One
answered in one piece waits 50 seconds; a job not done by then is answered
with its `job_id`, to ask after with `job_status`. `wait_seconds` (up to
900) sets either.
```
claude mcp add --transport http llmgw https://llm.example.com/mcp \
--header "Authorization: Bearer $LLMGW_API_KEY"
```
### Licences you leave out
A model's licence is in `/v1/models` (`license`: its `id`, `name`, whether