Commit Graph
572 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 5a4b022dcd A lost store machine is given up on in seconds, and a new leader waits for its judge
Measured on three VMs: cutting the store primary's power cost about 45 s of
hanging requests. New connections tried the dead member first with no
connect timeout, and Patroni kept its 30 s leader key. Cutting the leader of
both cost a further 17 s of 503s while the new leader knew no judge.

The store dials each member for 3 s and closes connections to a machine that
is gone within about 10 s (keepalive, TCP_USER_TIMEOUT). Patroni runs with
ttl 20, loop_wait 5 and retry_timeout 5, applied in place to existing
clusters through its API. Judges that all fail at once are asked again
every 250 ms while the policy's budget lasts, and reported once.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 20:33:13 +02:00
Ilker ManapandClaude Opus 5.5 eeaffd764d A floating address keeps to its own network, and Caddy follows the leader
Found by a three-machine install on fresh VMs. The second standby's run
asked for the floating address's interface with `ip route get <vip>`.
The machine already held the address, so that answered lo, and keepalived
would not start. The interface now comes from the route to the address's
network. Every cluster's VRRP router id was 51, so the PoC's two control
planes and the new three on one LAN each logged the other's adverts as an
invalid password; the id now comes from the address.

The Caddyfile, deploy/ha and install.md send clients to the control plane
whose /leader is 200, not /ready: /ready is 503 on a leader with no model
yet and would take the portal away with it. Setup no longer hides a key
file's path as if it were a password, and the starter's text no longer
says it runs elsewhere when this machine is a node.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 17:35:10 +02:00
Ilker ManapandClaude Opus 5.5 94401e21e6 Setup lets every part be chosen, each with its settings; three machines each run the gateway
The setup screens list the parts of an install: control plane, store, model
mirror, this machine as a node, starter model, pictures and 3D, the judge,
firewall, notifications, limits, on-demand turns and an offline bundle. Each
says what it is or what is missing without it. Space turns a part on or off;
Enter opens its settings in a dialog that explains it. A plan page lists what
is installed and what is left out, and the same install as a command. Every
setting is a server init flag too, written into server.yaml when init writes
it.

With three machines, the two database machines become control planes too,
standing by (server add's steps), with an optional floating address. The
mirror's management API answers all three control planes, so a standby that
comes to lead can still fetch models.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 11:22:36 +02:00
Ilker Manap 733e20924c Merge remote-tracking branch 'origin/mcp-wait' into sd-dedupe 2026-10-15 07:53:22 +02:00
Ilker ManapandClaude Opus 5.5 fc60481e40 Every control plane lists the others alike to Prometheus
140 named 150 a store member, 150 named itself a control plane: Prometheus
asking both scraped 150 twice and counted its every series twice, and asked
of 140 alone while 150 led it lost every node. Each control plane now lists
all of them as control planes before the store's members, so two answers
agree and a target is scraped once. The shipped prometheus.yml says to
scrape and ask every control plane.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 07:53:22 +02:00
Ilker ManapandClaude Opus 5.5 edeeb9f8e1 MCP takes pictures of real size, by id too, and waits for a cold model
The game's bridge found three things on the PoC. /mcp held its body to
64 KiB — the edit meant for it had raised the memory notes' limit instead —
so a 1024 px concept sent as base64 failed as "expected one JSON-RPC
request object". The notes are back to 64 KiB, /mcp takes 64 MB, and a
call past it is a 413 saying so.

/v1/files took batch files only, so a picture could not be sent once and
named after. purpose vision and user_data are kept too, as long as job
results, and generate_mesh hands a workflow model the id as it is.

generate_image and review_images failed at once while their model was
cold. They wait out an answer that says when to come back — loading,
another model holding the GPU, a full queue — for as long as the call may,
saying who holds the GPU and about how long; a refusal that says no
Retry-After, a spent budget, is said at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 07:24:59 +02:00
Ilker ManapandClaude Opus 5.5 c22528c26e A model that does not fit its card is not added to fail; weights in use are not removed
Apertus 70B with a 65536-token pool, and then 8B with 131072, on a 24 GB
card loaded their weights and failed to make their context on every start,
the 8B taking the card from the 70B each time only to fail. ctl model serve
now shrinks a pool the card cannot hold beside the weights (KV cache from
the GGUF's shape, 90% of the card, a gigabyte for the engine), saying so,
and refuses weights that do not fit at all; the portal's serve form refuses
such a pool and says the largest that fits.

The mirror page removed the 70B's weights while the catalog still had the
model, whose replicas went on loading the node's copy with nothing said.
Removing weights a catalog model loads is refused, naming the model.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-15 00:22:01 +02:00
Ilker Manap bda988b78f Merge remote-tracking branch 'origin/setup-webhooks' into shared-replica
# Conflicts:
#	CHANGELOG.md
2026-10-14 14:26:27 +02:00
Ilker Manap a7cb3aac3c Merge remote-tracking branch 'origin/mcp-media' into shared-replica
# Conflicts:
#	CHANGELOG.md
2026-10-14 14:01:24 +02:00
Ilker ManapandClaude Opus 5.5 d1e91868ee Setup asks where alerts go, and sends each a test
notify.webhooks existed and setup never asked for one, so a new site
learnt of a parked model or a lost node from its users. `llmgw server
init --webhook URL` (repeatable) and a Notification webhooks field on
both setup screens take them now. Slack's address, Mattermost's and
Rocket.Chat's /hooks/ and Discord's …/slack are sent Slack's text, any
other the event as JSON; slack: or json: in front says which.

Once the control plane runs, they are added to the cluster's settings
beside the webhooks already there, none twice, and setup waits for the
control planes to start again on the change before sending each a test,
saying which answered. A webhook's path is its secret, so it is shown by
host only. A failure is said, not fatal: the cluster serves without its
alerts, and ctl notify test tries again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:55:48 +02:00
Ilker ManapandClaude Opus 5.5 aa531b1053 ComfyUI models of one environment share one replica
flux and sf3d on one card handed it to each other, each starting ComfyUI
and loading its weights from nothing (BACKLOG H3, placement side). An
environment of two or more models now gets a hidden catalog entry,
comfyui-env-<name>: on its first member's profiles with the most memory
any member needs, on demand for the longest idle timeout, carrying every
member's weights, repositories and the environment's custom nodes. The
members get no replicas of their own; asking for one wants the entry.

The agent fetches each member's weights; the gate links all of them,
refusing two weights files of one name, links each member's Hugging Face
repository, and answers /v1/models for every member. The control plane
offers the replica under the members' names with each one's workflow and
checkpoint, which the router sends by the model asked for. A member's
sibling never evicts the replica serving it, and the members' load counts
as the entry's.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 13:55:34 +02:00
Ilker ManapandClaude Opus 5.5 55d5e13534 Pictures, 3D models, a vision model and workflows as tools at /mcp
An agent driving Blender made its concepts, meshes and render reviews
through a tool server of the game's own, which called this gateway; the
gateway's MCP server offered only the memory notes. It now offers
generate_image, generate_mesh, review_images, run_workflow, job_status and
gateway_status beside them, each only where a model for it is here.

Each tool makes its request of the client API as the caller, through the
handler Handler built, key check and all: allowlists, limits, licences
left out, budgets and accounting apply as to any request, and a job is
the caller's. What a tool makes is kept as the caller's file and answered
by id and download URL, since a server over HTTP cannot write to the
caller's disk; a picture comes with a 512 px preview, and whole where no
file can be kept. A picture to work from is a file id, a data URL or
base64; one sent to the vision model is shrunk to 1280 px.

A mesh takes minutes. A client that takes a stream is answered as
server-sent events, with notifications/progress for its token while the
job is queued, waiting for the GPU and running, and waits up to ten
minutes; one that does not waits 50 seconds and gets the job id, which
job_status asks after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:11:09 +02:00
Ilker ManapandClaude Opus 5.5 7c53380e68 Extended thinking on /v1/messages: a model's reasoning as thinking blocks
Claude Code and the Anthropic SDKs asking for extended thinking got the
answer alone: a reasoning model's reasoning_content was dropped. Asked for,
it comes back as a thinking block before the text — streamed as
thinking_delta events and closed with a signature before the text begins —
and the model is told to think through its chat template
(enable_thinking); disabled tells it not to. Not asked for, it is left out
as before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 11:05:02 +02:00
Ilker ManapandClaude Opus 5.5 0460f52dd2 Jobs run side by side, one per model at a time
One job at a time, a 3D job waiting for the card held up another model's
job that could have run (seen on the PoC, BACKLOG H7). The leader's worker
claims jobs while fewer than eight run, passing over the models that have
one running, and starts each in its own goroutine; one ending wakes it for
the next. Lease and leader-only rules are as they were.

BACKLOG: H5–H7 marked, H8 (presets) and H9 (what the PoC taught) written.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 10:06:20 +02:00
Ilker ManapandClaude Opus 5.5 efffdb66b7 An embedding model is served for embedding, with many slots
ctl model serve served one as a chat model: no --embedding, four slots, a
512-token batch, embeddings unset, and a chat smoke test it could not
pass. A GGUF that pools (<arch>.pooling_type), or --embeddings, gets
--embedding, 32 slots of 1024 tokens unless sized otherwise, a batch that
holds a whole passage, the embeddings capability, and one vector asked for
as its smoke test (canaries and updates too). An import of a hundred
thousand chunks had gone at the pace of four slots with the card a third
busy.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 09:19:57 +02:00
Ilker ManapandClaude Opus 5.5 f5eb6963af An upgrade that leaves nodes behind says which
A site's nodes stayed a release behind after its control plane was
upgraded, with nothing printed to say so. ctl upgrade checks every node's
version once the waves are done, fails naming any not on the release, and
says when the control plane has no nodes registered. llmgw upgrade on a
control plane, which looks like the cluster's upgrade and is not, says how
many nodes it left behind and the command that upgrades them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 09:01:09 +02:00
Ilker ManapandClaude Opus 5.5 1e7086e13e How long an on-demand model holds a card is a live setting
On one card shared by flux and sf3d, a mesh asked for after a picture
waited out the picture model's five minutes of protection and two of idle.
on_demand.evict_after_idle and on_demand.min_lifetime set those for the
cluster, in force on the leader as they change; unset keeps 2m and 5m.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:58:38 +02:00
Ilker ManapandClaude Opus 5.5 a9a2360f44 pages.js is loaded by its versioned address
Five pages named /_static/pages.js outright, cached for a day, so a
browser kept the old script after an upgrade — the one holding refreshes
off while a form is filled in among them. They take it through asset, as
the rest do, whose address changes with its contents.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:15:44 +02:00
Ilker ManapandClaude Opus 5.5 e78075876b A form being filled in is not refreshed away
The cluster and node pages swap their live part every few seconds, and the
forms inside it went with it: what was typed, an open choice, the focus.
Filling one in raced the clock. The swap now waits while a field in the
live part has the focus or a form in it has been typed into and not sent;
sending it, or leaving the page, ends that. The mirror page's reload
during a download goes through the same check instead of http-equiv.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 08:13:08 +02:00
Ilker ManapandClaude Opus 5.5 2eb2a6d8fe A mesh carries its whole licence, not only its name
GLBs carried the licence's name, URL and notice in asset.extras.license.
A pipeline that sorts assets by licence needs what a machine matches on:
the id, whether commercial use is allowed, the regions it excludes, the
revenue cap and the attribution. The whole licence is written, as
/v1/models gives it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 01:12:47 +02:00
Ilker ManapandClaude Opus 5.5 cfc636e328 A mesh's base colour can be brought to the picture it was made from
Stable Fast 3D takes the light out of the picture it sees, and a dark
basalt (median 0.17) came back a light stone dimly lit (0.67). The save
node takes albedo_match, the picture and its mask, and scales the base
colour texture in linear light by the ratio of the object's medians, so
marks keep their contrast; texels nothing maps to are left alone. On the
game team's rock it is 0.18.

A workflow's {{name|false}} default now fills a BOOLEAN input with a
boolean rather than the word.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-14 00:51:06 +02:00
Ilker ManapandClaude Opus 5.5 848db08469 The cluster page and the console say which picture and 3D models are here, and add them
A site whose catalog had no flux-schnell or triposr showed no Images or 3D
page, and nothing said they were one click away. The cluster page now has
a Pictures and 3D models card under the models: each preset, what it
makes, the classes of card here it fits, whether and by whom it was
added, and why the leader has not added it by itself; an Add button, which
says first when it would install ComfyUI on the GPU nodes and restart the
control planes; and the switch for presets.auto. The console's models
screen shows the same under its table: A adds one, asking first in the
same case, and P turns automatic adding on or off.

`ctl model add PRESET` and `ctl model presets` go through the presets API,
so the control plane keeps who added one, and `ctl model presets auto
on|off` flips the setting; against a control plane from before, add sends
the entry as v0.14.4 did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 20:48:40 +02:00
Ilker ManapandClaude Opus 5.5 f42af27da5 The leader adds the picture and 3D presets a node fits, unless told not to
A customer's portal had no Images or 3D pages: nobody had written a
definition for either, on a cluster whose V100 runs both. The leading
control plane now adds flux-schnell and triposr, on demand, once a node
has a card that fits one, the cluster installs ComfyUI, and the catalog
makes nothing of that kind. presets.auto (a live setting, on by default)
turns it off; a preset somebody removes is recorded as declined and not
added back until added again by hand. Each addition is in the audit log.

GET /api/v1/presets says how each preset stands, the ComfyUI the cluster
installs, and why one is not added; POST /api/v1/presets/{name} adds one
(setting engines.comfyui where there is none); PUT /api/v1/presets/auto
switches it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 20:48:40 +02:00
Ilker ManapandClaude Opus 5.5 40b8948d7d A picture or 3D model is added by its name: llmgw ctl model add triposr
flux-schnell and triposr come with llmgw as presets: their workflow,
custom nodes, packages and licence as they were found and tried on the
PoC, and a profile that says the card it needs rather than a class, so the
entry places on any site's cards. TripoSR's workflow is built in, with the
LLMGWPrepareForMesh node that keeps a picture's background out of the
mesh, which the copy in the document had left out.

`ctl model add PRESET` adds one, and has the nodes install ComfyUI when
the cluster's settings name none; `ctl model presets` lists them; the
console's add-model dialog offers them beside a definition file. A
ComfyUI build is now also checked to keep its order on a Turing card, so
a node's installed cu13 is not replaced by the cu126 build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 10:21:28 +02:00
Ilker ManapandClaude Opus 5.5 52898fbb80 A V100 gets a ComfyUI it can run, and an open driver is read by its version
PyTorch's CUDA 12.8 and 12.9 builds start at Turing since torch 2.8, as
CUDA 13 does, so on a V100 both ComfyUI builds were installed, 10 GB each,
and failed on the card with "no kernel image". The cu129 build now needs
7.5, and a third, on PyTorch's CUDA 12.6 torch, carries Volta and Pascal.

The open kernel module, NVIDIA's default since 560, writes "Open Kernel
Module for x86_64  580.82.07" in /proc/driver/nvidia/version, and the word
after "Module" was taken as the driver: "for". With the driver unknown,
every build was tried whatever driver it needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 10:21:27 +02:00
Ilker ManapandClaude Opus 5.5 fa6e68c8e8 A model put first is not evicted, idle, for a lower class
The VLM is to come first on the one card. Its class makes it take the card
from flux, triposr or sf3d when asked for, but idle eviction ignored class:
a 3D job two minutes into a lull would have taken the card from it. Idle
eviction offers only replicas of no higher class than the model wanting
room, and the wait such a model is told runs to the holder's idle timeout.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-13 10:15:14 +02:00
Ilker ManapandClaude Opus 5.5 967df77e53 The agent hands the gate the repository a node loads its weights by
sf3d's first jobs failed with a 401 from stabilityai/stable-fast-3d. The
weights were on the node, and the gate links them into the Hugging Face
cache when told the repository — but the agent passed on only the custom
nodes, so nothing was linked and the node went to the network for a gated
repository.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 23:28:22 +02:00
Ilker ManapandClaude Opus 5.5 3c626cdc96 A model removed from the catalog is no longer wanted
Removed and added again to give it a licence, qwen3-vl-8b came back loading
by itself: its name was still on the wanted list from before. A model that
leaves the catalog leaves the list too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 20:50:57 +02:00
Ilker ManapandClaude Opus 5.5 3858114ada A picture in a prompt counts as what a model makes of it, not its base64
A 1280x720 PNG sent to qwen3-vl-8b was estimated at 138 thousand prompt
tokens and refused against its 8k context before reaching the engine, which
makes about a thousand of it. Parts are counted by kind: text by its text,
a picture or a sound as a thousand tokens.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 20:45:06 +02:00
Ilker ManapandClaude Opus 5.5 ac4360b3fd A custom node's compiled packages are built for the card that runs them
Stable Fast 3D's texture_baker and uv_unwrapper are C++/CUDA packages no
index has; on the PoC they were built and installed into ComfyUI's
environment by hand. nodes[].build names such directories, and the gate
builds each as a wheel with ComfyUI's Python, for the compute capability
torch reports for the replica's own card, keeps it on the node by the
node's commit, the architecture and the directory, and installs it
beside the replica with the node's requirements. A copy in ComfyUI's
environment does not pin it. A node with no CUDA toolkit says so.

comfyui.huggingface_repo shows the replica's weights to a node that
loads them by repository name, in the engines' Hugging Face cache, so a
gated repository needs no token on the node.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 17:23:05 +02:00
Ilker ManapandClaude Opus 5.5 587d8d5059 A licence left out by its name stays out once its entry gives an id
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 16:28:40 +02:00
Ilker ManapandClaude Opus 5.5 4bbac0833d A person leaves licences out, and no key of theirs reaches a model under one
A game sold in the EU cannot ship what a licence excluding the EU made. On
the profile page a person leaves out licences one by one, or by rule:
every licence forbidding commercial use, every licence granting nothing in
a region. The rules cover models added later. The choice reaches every
principal through the resolved limits, so permits() leaves such models out
of /v1/models, requests, jobs, fallbacks and the portal's pickers, and a
request for one is refused 403 licence_excluded saying which choice did.

Kept in user_excluded_licences (migration 0046), copied with the store and
gone with the person.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 16:10:22 +02:00
Ilker ManapandClaude Opus 5.5 a313b3e0da The reference documents a profile's gpu need and its own comfyui block
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 16:10:16 +02:00
Ilker ManapandClaude Opus 5.5 777a9223a7 A profile may name the card it needs rather than its class, and fit any site
A catalog entry meant for many sites cannot know their GPU class names:
the PoC has a Quadro RTX 6000, a customer an A100, another an L40S. A
profile may now say gpu: {min_compute_capability, min_memory_gb} in
place of gpu_class, and stands for every class in the cluster that meets
it, named <profile>@<class>, at its own priority; placement still sees
exact classes. ProfileByName and EngineProfile resolve such a name to
the profile on that class, so every caller that maps a replica back to
its profile works unchanged. A ComfyUI profile may carry its own
workflow, checkpoint and nodes, and with its own source an fp8 workflow
and weights go to cards with fp8 and fp16 ones to Turing.

A profile by need is left out of placement until the cluster's cards are
known: with no class it would have read as a CPU profile.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 16:02:32 +02:00
Ilker ManapandClaude Opus 5.5 b6bfc33ef7 Any ComfyUI workflow runs as a job, from the inputs its catalog entry declares
A model's catalog entry now says what its workflow takes and makes:
comfyui.inputs, each a placeholder of its name with a type (text, int,
float, bool, choice, image, mask, audio, video, file), bounds, choices
and a default; and comfyui.output, which gains video, audio and files
beside pictures and meshes. POST /v1/workflows/runs with {model, inputs}
checks a request against them before a job is written — a required input
left out, an unknown one, a number out of bounds, a song sent as a
picture, another owner's file id are each refused with what was wrong —
and the job worker replays it to a replica, where the files go into
ComfyUI's input directory, the workflow is filled and run, and the first
file of the model's output kind is the job's result. /v1/models lists
each ComfyUI model's workflow, so a client can build its form.

A PNG result carries the licence in an iTXt chunk, as a GLB does in
asset.extras. A file whose server named no type is typed by its name
before its bytes, so a WAV is audio/wav, not audio/wave.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 14:28:15 +02:00
Ilker ManapandClaude Opus 5.5 b43e66c95e ComfyUI models that agree on their nodes share an environment; a licence says what a machine can match
A ComfyUI model may name an environment: the models of one environment
agree on every custom node they both name and on every package range,
so one replica can carry all of them and ComfyUI moves between their
weights itself rather than the card being handed over with a cold start
each time. The catalog refuses an environment that cannot share a
Python, naming both models; Catalog.Environments gives each one's
models, the union of their nodes and their weight sources, for
placement to build on.

A licence gains an id (SPDX where there is one), whether its output may
be used commercially, the regions it excludes, a revenue cap and the
attribution it asks for, so a person's or a site's choice of licences
can be matched without reading prose.

BACKLOG gains section H: ComfyUI as a platform.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 09:19:50 +02:00
Ilker ManapandClaude Opus 5.5 0639edfd3b An adopted on-demand replica counts as used when adopted
After a control plane restart flux, adopted and asked for by nobody, had no
last use and so was never idle enough to evict: a 3D job waited out its
whole fifteen minutes with no estimate. It counts as used at adoption, as it
is already counted as wanted then; a replica in use is reported by the
router within seconds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-11 08:55:33 +02:00
Ilker ManapandClaude Opus 5.5 95c61daf1b Jobs are served by the leader, and a moment of fitting nowhere is waited out
A 3D job leased by the standby control plane, whose router has no endpoints
and which a job replayed in the process never leaves, was told "loading"
for seven minutes while the leader served the jobs behind it. Jobs and
batches are claimed only while leading.

Five jobs asked for while cuda01's agent restarted failed for good as
"cannot be placed": re-registering, the node had no engines for a few
seconds. A job asks again for five minutes before that is its end.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 23:53:33 +02:00
Ilker ManapandClaude Opus 5.5 9b8ec89247 A mesh node's result is a file the gate can fetch
Stable Fast 3D's own save node hands back the GLB as base64 in its UI
output, which the gate does not read. LLMGWSaveMesh writes each mesh to
output/ as a GLB and names it, as a picture's save node does. The
prepare node also gives the object's mask now, which SF3D's sampler
takes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 22:29:34 +02:00
Ilker ManapandClaude Opus 5.5 b1490dbf30 A job waiting for the card says what holds it and about how long
A 3D job waiting for the picture model to give up the one card had an empty
progress; its client could not tell waiting from stuck. The control plane
says which model holds the cards and when it lets go, plus the waiting
model's last start; the 503 carries it as headers, and a job keeps it as
progress {state: waiting_gpu, held_by, eta_seconds}.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 22:25:27 +02:00
Ilker ManapandClaude Opus 5.5 cf2edfd995 A model evicted for being idle is no longer wanted
flux evicted to make room for triposr was placed again two seconds later:
still on the wanted list, and first in the catalog, it took back the card
it had just given up, and the 3D job went on waiting. Its last replica
evicted, it leaves the list until it is asked for again.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 21:11:40 +02:00
Ilker ManapandClaude Opus 5.5 d0f7b1fc4f A model waiting for an on-demand one to step aside is waiting, not unplaceable
A 3D job asked for straight after a picture, on a cluster with one card,
failed at once with "cannot be placed": flux held the card, too freshly
started to evict. The cards coming back when an on-demand model goes idle
makes it a wait, so the request is told to retry and the job waits.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 20:59:59 +02:00
Ilker ManapandClaude Opus 5.5 b743b11c54 A picture is made ready for a 3D model; a model asleep is listed asleep
LLMGWPrepareForMesh, llmgw's own ComfyUI node, written by the gate into
every replica's custom_nodes from the binary: TripoSR's own preparation
(tsr/utils.py) — the background taken away, by the picture's alpha where
it has one and by rembg (MIT; u2net, Apache 2.0) where it does not, the
object cropped to its bounds, set square at a ratio of the side (0.85),
on grey. A picture sent as it was had a white or black shell round the
object, whatever was behind it made into the mesh.

/v1/models listed only models with a replica up, so one that loads when
asked — triposr, flux-schnell — was invisible to the clients that would
have asked for it. Those are listed now, with loaded: false; every entry
says loaded.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 20:43:20 +02:00
Ilker ManapandClaude Opus 5.5 fd3c7bda32 The client API described in OpenAPI, and both APIs in Swagger UI
/v1/openapi.json is the API clients use — OpenAI's endpoints, Anthropic's,
and what llmgw adds — in OpenAPI 3.1, generated from internal/clientapi's
route table by the generator the admin API's document comes from, now
told each document's title, server, token and error. A test reads the
router's mux.HandleFunc calls and refuses a route described and not
mounted, or mounted and not described. Asked for before there is a key,
so it needs none.

/docs/api shows it in Swagger UI 5.33.1 (Apache 2.0, served from the
gateway as the page's policy requires): Authorize takes a key, Try it out
sends to this gateway. Staff are offered the admin API's document too,
from /docs/api/admin.json, which nobody else is given.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 20:22:48 +02:00
Ilker ManapandClaude Opus 5.5 7f320c5f65 v0.14.3: changelog, README, docs for 3D models; a finished job says done
A finished job kept the last word heard from ComfyUI, the node running;
it is now said done before the job is marked completed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 19:22:58 +02:00
Ilker ManapandClaude Opus 5.5 2f54780802 G4: a page that makes 3D models, and ctl 3d
/3d beside /images, while some model makes meshes: a picture chosen, the
job asked for through the same path an API client's takes, its progress
shown (waiting for the card, n ahead; making it, the node and percent),
and the mesh in a viewer — model-viewer 4.4.0 (Apache 2.0, with three.js
and lit), served from the gateway since the page's policy loads scripts
from nowhere else — with a download and the model's licence. The mesh is
served to its owner from /3d/jobs/{id}/file; another project's person is
told there is no such job.

A job made from the portal acted for a key that does not exist: the
worker now rebuilds a session's principal from the account that made it,
while it is active and still in the project the job is charged to.

ctl 3d list|get|cancel, over /api/v1/client-jobs (/api/v1/jobs is the
control plane's own work, which ctl jobs shows).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 19:16:20 +02:00
Ilker ManapandClaude Opus 5.5 dfc2f0bf15 A custom node's own range decides the packages it names
TripoSR's checkpoint loads with transformers 4, and ComfyUI runs 5: of a
package a node names, its range decides, and of everything else what
ComfyUI runs.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:54:15 +02:00
Ilker ManapandClaude Opus 5.5 4b68328e3e A mesh's resolution is the model's own: a multiple of 32 from 64 to 2048
TripoSR takes 256 by default; 512, 1024 and 1536 were TRELLIS.2's alone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:47:52 +02:00
Ilker ManapandClaude Opus 5.5 88781cfcfe A job says how far along it is; a ComfyUI model brings its custom nodes
G2: the gate keeps a websocket open to its ComfyUI under a client id of its
own, puts that id on every workflow it lets through — ComfyUI tells only
the client that queued a workflow how it goes — and keeps the last word on
each: the node running, its step of how many. It answers at
/progress/{prompt_id}. While a job's request waits, the router reads the
queue (queued, n ahead) and the gate (running, node, step, percent) and
writes it into the job, which a client reads back. Only into a job of the
same key: a tag naming another's job writes nothing.

G3, the rest of the plumbing: comfyui.nodes names custom nodes by https
repository and full commit, with the PyPI packages they need. The gate puts
each in the replica's own custom_nodes at that commit and installs the
packages beside the replica, constrained to the versions ComfyUI's
environment already has, which it never changes. The catalog refuses a
branch for a commit and anything but package specs for requirements. An
OBJ a node saves leaves the gateway as a GLB, colours kept, so every mesh
can carry its licence.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:46:44 +02:00
Ilker ManapandClaude Opus 5.5 c87e413360 A ComfyUI model may make meshes: from a picture, with its licence inside
G3, the part that needs no model: a ComfyUI model's comfyui.output (or
external.output) says pictures or meshes. A request for a 3D model — a
job at /v1/3d/generations — carries the picture as base64; at the replica
it is uploaded to ComfyUI's input directory (the gate lets /upload/image
through), the workflow is filled in with its name, the seed and the
resolution, and the mesh the workflow saves comes back with its type,
which is the file the job keeps. comfy.Client reads every list of files an
output names, not "images" only, and types a file by its extension where
ComfyUI sends octet-stream.

A model's license (name, url, notice) is said in /v1/models and in the
job, and written into a GLB's asset.extras, since the file travels without
the API; a mesh it cannot be written into is refused. per_mesh in
auth.pricing, usage.meshes in the rows, the sums and the export
(migration 0045). A picture asked of a mesh model, or a mesh of a picture
model, is refused saying which it makes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-10 18:30:52 +02:00