The reference documents a profile's gpu need and its own comfyui block
CI / test (push) Failing after 8m7s
CI / test (push) Failing after 8m7s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
777a9223a7
commit
a313b3e0da
3 files changed
+48
No files matched your search
@@ -119,6 +119,8 @@ rather than repeated.
|
||||
| `name` | `string` | [Federation](#federation) |
|
||||
| `interval` | `time.Duration` | [Federation](#federation) |
|
||||
| `enabled` | `bool` | [FirewallSettings](#firewallsettings) |
|
||||
| `min_compute_capability` | `float64` | [GPUNeed](#gpuneed) |
|
||||
| `min_memory_gb` | `int` | [GPUNeed](#gpuneed) |
|
||||
| `format` | `string` | [GuardModel](#guardmodel) |
|
||||
| `categories` | `[]string` | [GuardModel](#guardmodel) |
|
||||
| `name` | `string` | [Input](#input) |
|
||||
@@ -266,6 +268,9 @@ rather than repeated.
|
||||
| `kv_events` | `bool` | [Profile](#profile) |
|
||||
| `gpu` | `*GPUNeed` | [Profile](#profile) |
|
||||
| `comfyui` | `*ProfileComfyUI` | [Profile](#profile) |
|
||||
| `workflow` | `string` | [ProfileComfyUI](#profilecomfyui) |
|
||||
| `checkpoint` | `string` | [ProfileComfyUI](#profilecomfyui) |
|
||||
| `nodes` | `[]CustomNode` | [ProfileComfyUI](#profilecomfyui) |
|
||||
| `activation_wait` | `time.Duration` | [Router](#router) |
|
||||
| `scrape_interval` | `time.Duration` | [Router](#router) |
|
||||
| `scrape_timeout` | `time.Duration` | [Router](#router) |
|
||||
@@ -1202,6 +1207,21 @@ site whose own firewall does that job; the rules are kept, and true
|
||||
puts them back. Changed in the cluster's settings, it is in force on
|
||||
every node and control plane within seconds.
|
||||
|
||||
## GPUNeed
|
||||
|
||||
### `min_compute_capability`
|
||||
|
||||
`float64`
|
||||
|
||||
MinComputeCapability is the oldest architecture that runs it: 7.5 is
|
||||
Turing, 8.0 Ampere (bf16, flash-attention 2), 8.9 Ada (fp8), 9.0 Hopper.
|
||||
|
||||
### `min_memory_gb`
|
||||
|
||||
`int`
|
||||
|
||||
MinMemoryGB is the card's own memory it needs.
|
||||
|
||||
## GuardModel
|
||||
|
||||
### `format`
|
||||
@@ -2316,6 +2336,8 @@ GPU says what card the profile needs, in place of naming its class:
|
||||
it stands for every class in the cluster that meets it. A catalog
|
||||
entry written once then places sensibly on any site's cards.
|
||||
|
||||
Takes a [GPUNeed](#gpuneed) block.
|
||||
|
||||
### `comfyui`
|
||||
|
||||
`*ProfileComfyUI`
|
||||
@@ -2323,6 +2345,22 @@ entry written once then places sensibly on any site's cards.
|
||||
ComfyUI is this profile's own workflow, checkpoint and nodes, over the
|
||||
model's: an fp8 workflow where the card has fp8, fp16 where it has not.
|
||||
|
||||
Takes a [ProfileComfyUI](#profilecomfyui) block.
|
||||
|
||||
## ProfileComfyUI
|
||||
|
||||
### `workflow`
|
||||
|
||||
`string`
|
||||
|
||||
### `checkpoint`
|
||||
|
||||
`string`
|
||||
|
||||
### `nodes`
|
||||
|
||||
`[]CustomNode`
|
||||
|
||||
## Router
|
||||
|
||||
### `activation_wait`
|
||||
|
||||
@@ -148,6 +148,10 @@ var Reference = []Block{
|
||||
{Package: "config", Name: "FirewallSettings", Title: "FirewallSettings", Doc: "", Fields: []Field{
|
||||
{Key: "enabled", Type: "bool", Doc: "Enabled keeps llmgw's internal ports — engines, node exporter, the\nagents' stream, the mirror, the store — to the cluster's networks on\nevery machine. False removes llmgw's nftables table everywhere, for a\nsite whose own firewall does that job; the rules are kept, and true\nputs them back. Changed in the cluster's settings, it is in force on\nevery node and control plane within seconds.", Nested: ""},
|
||||
}},
|
||||
{Package: "catalog", Name: "GPUNeed", Title: "GPUNeed", Doc: "", Fields: []Field{
|
||||
{Key: "min_compute_capability", Type: "float64", Doc: "MinComputeCapability is the oldest architecture that runs it: 7.5 is\nTuring, 8.0 Ampere (bf16, flash-attention 2), 8.9 Ada (fp8), 9.0 Hopper.", Nested: ""},
|
||||
{Key: "min_memory_gb", Type: "int", Doc: "MinMemoryGB is the card's own memory it needs.", Nested: ""},
|
||||
}},
|
||||
{Package: "catalog", Name: "GuardModel", Title: "GuardModel", Doc: "", Fields: []Field{
|
||||
{Key: "format", Type: "string", Doc: "Format is qwen3guard (Qwen3Guard-Gen: \"Safety: Safe\", \"Unsafe\" or\n\"Controversial\", and the categories), llamaguard3 (Meta's Llama Guard\n3: \"safe\", or \"unsafe\" and the categories), shieldgemma (Google's\nShieldGemma: Yes or No), or json ({\"safe\": false, \"categories\":\n[\"...\"]}).", Nested: ""},
|
||||
{Key: "categories", Type: "[]string", Doc: "Categories the model is asked about, for shieldgemma — each a policy\nsentence — and the names reported for llamaguard3's S1, S2, ….", Nested: ""},
|
||||
@@ -337,6 +341,11 @@ var Reference = []Block{
|
||||
{Key: "gpu", Type: "*GPUNeed", Doc: "GPU says what card the profile needs, in place of naming its class:\nit stands for every class in the cluster that meets it. A catalog\nentry written once then places sensibly on any site's cards.", Nested: "catalog.GPUNeed"},
|
||||
{Key: "comfyui", Type: "*ProfileComfyUI", Doc: "ComfyUI is this profile's own workflow, checkpoint and nodes, over the\nmodel's: an fp8 workflow where the card has fp8, fp16 where it has not.", Nested: "catalog.ProfileComfyUI"},
|
||||
}},
|
||||
{Package: "catalog", Name: "ProfileComfyUI", Title: "ProfileComfyUI", Doc: "", Fields: []Field{
|
||||
{Key: "workflow", Type: "string", Doc: "", Nested: ""},
|
||||
{Key: "checkpoint", Type: "string", Doc: "", Nested: ""},
|
||||
{Key: "nodes", Type: "[]CustomNode", Doc: "", Nested: ""},
|
||||
}},
|
||||
{Package: "config", Name: "Router", Title: "Router", Doc: "", Fields: []Field{
|
||||
{Key: "activation_wait", Type: "time.Duration", Doc: "ActivationWait is how long a request waits for an on-demand model to\nload before the client is told to come back. Zero takes the default.\n\nThe one real decision in on-demand loading. Refusing outright makes the\nfeature useless, since the first request for every model would fail;\nwaiting without a bound turns a request into a job, and a client that\nbudgeted thirty seconds ends up holding a connection for ten minutes.", Nested: ""},
|
||||
{Key: "scrape_interval", Type: "time.Duration", Doc: "", Nested: ""},
|
||||
|
||||
@@ -27,6 +27,7 @@ var Sources = []string{
|
||||
"internal/tlsconf/tlsconf.go",
|
||||
"internal/control/catalog/catalog.go",
|
||||
"internal/control/catalog/workflow.go",
|
||||
"internal/control/catalog/hardware.go",
|
||||
"internal/router/admission/controller.go",
|
||||
"internal/accounting/cost.go",
|
||||
"internal/guard/policy.go",
|
||||
|
||||
Reference in new issue
Block a user