Jobs run side by side, one per model at a time

One job at a time, a 3D job waiting for the card held up another model's
job that could have run (seen on the PoC, BACKLOG H7). The leader's worker
claims jobs while fewer than eight run, passing over the models that have
one running, and starts each in its own goroutine; one ending wakes it for
the next. Lease and leader-only rules are as they were.

BACKLOG: H5–H7 marked, H8 (presets) and H9 (what the PoC taught) written.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-14 10:06:20 +02:00
1 parent efffdb66b7
commit 0460f52dd2
7 files changed
+131 -8

No files matched your search

+4
View File
@@ -10413,6 +10413,10 @@
"type": "boolean",
"description": "Tools says the model makes tool calls the engine returns as calls: it is written into the catalog, where clients and editors read it. Check a running replica with llmgw ctl tool-check before saying so."
},
"embeddings": {
"type": "boolean",
"description": "Embeddings serves it as an embedding model, which its GGUF header says by itself for most: --embedding, many slots, a batch that holds a whole passage, /v1/embeddings."
},
"cpu_cores": {
"type": "integer",
"description": "CPUCores is what a replica computes on; zero takes half the smallest node it fits on."