A profile may name the card it needs rather than its class, and fit any site
CI / test (push) Failing after 6m19s

A catalog entry meant for many sites cannot know their GPU class names:
the PoC has a Quadro RTX 6000, a customer an A100, another an L40S. A
profile may now say gpu: {min_compute_capability, min_memory_gb} in
place of gpu_class, and stands for every class in the cluster that meets
it, named <profile>@<class>, at its own priority; placement still sees
exact classes. ProfileByName and EngineProfile resolve such a name to
the profile on that class, so every caller that maps a replica back to
its profile works unchanged. A ComfyUI profile may carry its own
workflow, checkpoint and nodes, and with its own source an fp8 workflow
and weights go to cards with fp8 and fp16 ones to Turing.

A profile by need is left out of placement until the cluster's cards are
known: with no class it would have read as a CPU profile.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-11 16:02:32 +02:00
1 parent b6bfc33ef7
commit 777a9223a7
11 files changed
+419 -14

No files matched your search

+17
View File
@@ -264,6 +264,8 @@ rather than repeated.
| `priority` | `int` | [Profile](#profile) |
| `startup_timeout` | `time.Duration` | [Profile](#profile) |
| `kv_events` | `bool` | [Profile](#profile) |
| `gpu` | `*GPUNeed` | [Profile](#profile) |
| `comfyui` | `*ProfileComfyUI` | [Profile](#profile) |
| `activation_wait` | `time.Duration` | [Router](#router) |
| `scrape_interval` | `time.Duration` | [Router](#router) |
| `scrape_timeout` | `time.Duration` | [Router](#router) |
@@ -2306,6 +2308,21 @@ is started with sha256_cbor block hashing and a fixed PYTHONHASHSEED,
which the router's hashing assumes; arguments overriding either leave
the index empty, and the agent says so in its log.
### `gpu`
`*GPUNeed`
GPU says what card the profile needs, in place of naming its class:
it stands for every class in the cluster that meets it. A catalog
entry written once then places sensibly on any site's cards.
### `comfyui`
`*ProfileComfyUI`
ComfyUI is this profile's own workflow, checkpoint and nodes, over the
model's: an fp8 workflow where the card has fp8, fp16 where it has not.
## Router
### `activation_wait`