ComfyUI models that agree on their nodes share an environment; a licence says what a machine can match
CI / test (push) Failing after 6m21s
CI / test (push) Failing after 6m21s
A ComfyUI model may name an environment: the models of one environment agree on every custom node they both name and on every package range, so one replica can carry all of them and ComfyUI moves between their weights itself rather than the card being handed over with a cold start each time. The catalog refuses an environment that cannot share a Python, naming both models; Catalog.Environments gives each one's models, the union of their nodes and their weight sources, for placement to build on. A licence gains an id (SPDX where there is one), whether its output may be used commercially, the regions it excludes, a revenue cap and the attribution it asks for, so a person's or a site's choice of licences can be matched without reading prose. BACKLOG gains section H: ComfyUI as a platform. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
0639edfd3b
commit
b43e66c95e
7 files changed
+549
No files matched your search
+127
@@ -1270,6 +1270,133 @@ the .glb, a viewer with `<model-viewer>` from the allowed CDNs), `ctl
|
||||
jobs list|get|cancel`, the console's Jobs view. The page sends down the
|
||||
same path as the API, like `imagesSend` does.
|
||||
|
||||
## H. ComfyUI as a platform (written 2026-10-11)
|
||||
|
||||
G made one model at a time work: flux for pictures, TripoSR and SF3D for
|
||||
meshes, each with its own endpoint, its own request shape and its own
|
||||
replica. The user's direction (2026-10-11): llmgw is a generic platform,
|
||||
and should serve whatever ComfyUI can run — pictures, edits, video,
|
||||
audio, meshes, upscaling, segmentation — as data in the catalog rather
|
||||
than code per model. Two decisions shape everything below:
|
||||
|
||||
- **The licence is metadata, not a filter.** llmgw does not leave a
|
||||
model out because of its licence. The licence travels with the model
|
||||
and with every file it makes; what a site will not serve is that
|
||||
site's policy (H6). The game studio's EU constraint is one such policy,
|
||||
not a decision the code makes for everyone.
|
||||
- **The install fits the hardware.** The PoC has one Turing card; other
|
||||
sites will have Ampere, Ada and Hopper. A catalog entry must be
|
||||
writable once and placed sensibly anywhere: fp16 on Turing, bf16 or
|
||||
fp8 where the card has them, flash-attention models only where it
|
||||
runs, CUDA extensions built for the card that runs them (H2, H4).
|
||||
|
||||
The first set of models proves the structure at its corners rather than
|
||||
by licence: Qwen-Image-Edit (several weight files, two pictures in),
|
||||
Wan 2.2 small (video out, a long job), ACE-Step (audio out), SF3D (a
|
||||
package not on PyPI, gated weights), Hunyuan3D (a licence that excludes
|
||||
the EU, which H6 must be able to say no to).
|
||||
|
||||
### H1. A workflow says what it takes and makes (M)
|
||||
- Catalog: `comfyui.inputs` — a list of `{name, type, required, default,
|
||||
min, max, choices, max_mb, doc}`, type one of text, int, float, bool,
|
||||
choice, image, mask, audio, video, file. Each is a placeholder of the
|
||||
same name in the workflow. The template's own placeholders are checked
|
||||
against the list when the catalog is built: one the workflow uses that
|
||||
no input or built-in names is refused there, not at the first request.
|
||||
- `comfyui.output` gains `video`, `audio` and `files` beside `pictures`
|
||||
and `meshes`; `files` takes whatever the workflow saves. `comfy.Image`
|
||||
becomes `comfy.File` and `typeByName` knows mp4, webm, gif, wav, flac,
|
||||
mp3, ogg.
|
||||
- Endpoint: `POST /v1/workflows/runs` with `{model, inputs: {...}}`, a
|
||||
job like G1's (`GET /v1/workflows/runs[/{id}]`, `.../cancel`). A file
|
||||
input is base64, a data URL, or a `file-…` id from `/v1/files` for
|
||||
what is too big to send inline (video, long audio). Files go into the
|
||||
replica's input directory through the gate's `/upload/image`, which
|
||||
ComfyUI uses for any file. `/v1/images/generations` and
|
||||
`/v1/3d/generations` stay as they are, as fixed shapes over the same
|
||||
machinery.
|
||||
- The models list (`/v1/models`) gives each ComfyUI model its inputs and
|
||||
output kind, so a client can build a form without reading the catalog.
|
||||
- Done when: the existing flux, triposr and sf3d entries work unchanged,
|
||||
and one model with a picture and a text input (Qwen-Image-Edit) runs
|
||||
through `/v1/workflows/runs` with a 400 for a missing required input
|
||||
before any job is written.
|
||||
|
||||
### H2. Profiles that fit the hardware (M)
|
||||
- A profile may say `gpu: {min_compute_capability, min_memory_gb}` in
|
||||
place of `gpu_class`. When the reconciler places a model, such a
|
||||
profile stands for every GPU class in the cluster that meets it, at
|
||||
the profile's priority: the catalog function that turns profiles into
|
||||
placement's view takes the cluster's classes (class, compute
|
||||
capability, memory) and returns one concrete profile per class that
|
||||
fits, named `<profile>@<class>`. Placement itself is unchanged: it
|
||||
still sees exact classes, and a tensor-parallel group still never
|
||||
mixes them.
|
||||
- A ComfyUI profile may carry its own `comfyui` block (workflow, nodes,
|
||||
checkpoint) over the model's: the fp8 workflow on compute 8.9 and up,
|
||||
the fp16 one on 7.5. A Turing card is not an error, it is the profile
|
||||
that fits it.
|
||||
- Done when: one catalog entry with an "ampere+" profile (min 8.0) and a
|
||||
"turing" fallback places the fallback on cuda01, and the same entry,
|
||||
in a test with an A100 in the inventory, places the other.
|
||||
|
||||
### H3. Environments: which models can share a replica (S, the catalog side)
|
||||
- `comfyui.environment: <name>` puts a model in a named environment.
|
||||
Models in one environment must agree on every custom node they both
|
||||
name (same repository, same commit) and their requirements must
|
||||
resolve together; the catalog refuses an environment that cannot, with
|
||||
both models' names. Models with no environment are each their own, as
|
||||
today.
|
||||
- `Catalog.Environments()` returns each environment with its models and
|
||||
the union of their nodes: what one replica of it must install.
|
||||
- Placement and eviction of a replica that serves every model of its
|
||||
environment (so flux, SF3D and Qwen-Image-Edit stop handing the card to
|
||||
each other with a 30 s cold start each time) is the other session's
|
||||
work (llmgw-b7), built on this.
|
||||
|
||||
### H4. Custom nodes with packages that are built, not downloaded (S)
|
||||
- `nodes[].build: [subdir, ...]`: directories of the node's repository
|
||||
that are Python packages with C++/CUDA extensions (SF3D's texture_baker
|
||||
and uv_unwrapper). The gate builds each as a wheel with the engine's
|
||||
Python, `TORCH_CUDA_ARCH_LIST` set from the compute capability of the
|
||||
card it runs on, and installs it into the replica's own `site` like
|
||||
any other requirement. Built wheels are kept under the engine's home by
|
||||
node commit, subdirectory and architecture, so the second replica on
|
||||
the same kind of card does not build again.
|
||||
- A node needing nvcc on a node without it refuses the replica with that
|
||||
said, not with a compiler's traceback.
|
||||
- Done when: SF3D runs on cuda01 with nothing installed into the engine's
|
||||
venv by hand (the hand install of 2026-10-10 removed first).
|
||||
|
||||
### H5. Licence on every output (S)
|
||||
- `License` gains `id` (SPDX where there is one, else a lowercase name:
|
||||
stability-community, tencent-hunyuan-community), `commercial` (unset is
|
||||
not said), `excluded_regions` (ISO 3166 codes, `EU` for the union),
|
||||
`revenue_cap_usd` and `attribution` ("Powered by Stability AI"). All
|
||||
optional but the name. Done 2026-10-11 in the catalog, so H6 can match
|
||||
on `id`.
|
||||
- Written into the file where the format has a place for it: glTF
|
||||
`asset.extras.license` (as now), PNG an `iTXt` chunk named `license`
|
||||
(the workflow's own text chunks are still stripped), MP4/WebM and the
|
||||
audio formats in their metadata where ffmpeg is at hand on the router,
|
||||
otherwise beside it in the job's answer only — said in the docs.
|
||||
|
||||
### H6. Licence choices (S) — the other session's (llmgw-b7), at the user's request
|
||||
The user asked (2026-10-11) for the choice to be a person's: on their
|
||||
profile they pick the licences they will not use, and their keys and
|
||||
portal session neither see those models in `/v1/models` nor may call
|
||||
them (403, saying which licence). Matched on `license.id`, and on
|
||||
`commercial` / `excluded_regions` where a choice is put that way. Its
|
||||
store table is migration 0046. A site-wide or team-wide default can sit
|
||||
on top of it later.
|
||||
|
||||
### H7. A job worker per model, not one for all (S)
|
||||
The worker runs one job at a time: a job waiting for the card holds up a
|
||||
job for another model that could run now (seen on the PoC, 2026-10-10).
|
||||
Jobs are claimed and run concurrently up to a limit per model (or per
|
||||
environment, after H3) and a cluster-wide cap, with the lease and
|
||||
leader-only rules of G1 kept.
|
||||
|
||||
## D. Facts worth knowing before starting
|
||||
|
||||
- Admin API: `127.0.0.1:9090`, no auth, `ctlClient` in `internal/cli/ctl.go`.
|
||||
|
||||
@@ -8782,6 +8782,10 @@
|
||||
"$ref": "#/components/schemas/catalog.CustomNode"
|
||||
},
|
||||
"description": "Nodes are custom nodes the workflow needs, put in each replica's own custom_nodes at the commit named."
|
||||
},
|
||||
"environment": {
|
||||
"type": "string",
|
||||
"description": "Environment names the models that may share one ComfyUI replica: every model of an environment agrees on the custom nodes they both name, so one replica can carry all of them and ComfyUI moves between their weights itself. Empty is an environment of the model alone."
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -8879,6 +8883,10 @@
|
||||
"type": "object",
|
||||
"description": "License is the terms what a model makes comes under: said beside every result, and written into a mesh, which travels without the API.",
|
||||
"properties": {
|
||||
"id": {
|
||||
"type": "string",
|
||||
"description": "ID names the licence for a machine: an SPDX identifier where there is one (MIT, Apache-2.0, CC-BY-NC-4.0), else a lowercase name of the licence's own (stability-community, tencent-hunyuan-community). What a person's or a site's choice of licences matches on. Name is for people."
|
||||
},
|
||||
"name": {
|
||||
"type": "string",
|
||||
"description": "Name is the licence's own: MIT, Apache-2.0."
|
||||
@@ -8889,6 +8897,26 @@
|
||||
"notice": {
|
||||
"type": "string",
|
||||
"description": "Notice is what must accompany what was made: attribution, terms on the output, what the weights were trained on."
|
||||
},
|
||||
"commercial": {
|
||||
"type": "boolean",
|
||||
"description": "Commercial says whether what is made may be used commercially: nil is not said, which is not the same as yes."
|
||||
},
|
||||
"excluded_regions": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
},
|
||||
"description": "ExcludedRegions are where the licence grants nothing: ISO 3166 codes, and EU for the union (Hunyuan's: EU, UK, KR)."
|
||||
},
|
||||
"revenue_cap_usd": {
|
||||
"type": "integer",
|
||||
"format": "int64",
|
||||
"description": "RevenueCapUSD is the yearly revenue above which the licence asks for another agreement (Stability's community licence: 1000000). Zero is none."
|
||||
},
|
||||
"attribution": {
|
||||
"type": "string",
|
||||
"description": "Attribution is what a product using the output must show: \"Powered by Stability AI\"."
|
||||
}
|
||||
}
|
||||
},
|
||||
|
||||
@@ -66,6 +66,7 @@ rather than repeated.
|
||||
| `checkpoint` | `string` | [ComfyUI](#comfyui) |
|
||||
| `output` | `string` | [ComfyUI](#comfyui) |
|
||||
| `nodes` | `[]CustomNode` | [ComfyUI](#comfyui) |
|
||||
| `environment` | `string` | [ComfyUI](#comfyui) |
|
||||
| `dir` | `string` | [ComfyUIConfig](#comfyuiconfig) |
|
||||
| `python` | `string` | [ComfyUIConfig](#comfyuiconfig) |
|
||||
| `interval` | `time.Duration` | [Control](#control) |
|
||||
@@ -133,9 +134,14 @@ rather than repeated.
|
||||
| `notice` | `string` | [Landing](#landing) |
|
||||
| `language` | `string` | [Landing](#landing) |
|
||||
| `docs_sign_in` | `bool` | [Landing](#landing) |
|
||||
| `id` | `string` | [License](#license) |
|
||||
| `name` | `string` | [License](#license) |
|
||||
| `url` | `string` | [License](#license) |
|
||||
| `notice` | `string` | [License](#license) |
|
||||
| `commercial` | `*bool` | [License](#license) |
|
||||
| `excluded_regions` | `[]string` | [License](#license) |
|
||||
| `revenue_cap_usd` | `int64` | [License](#license) |
|
||||
| `attribution` | `string` | [License](#license) |
|
||||
| `bin` | `string` | [LlamaCppConfig](#llamacppconfig) |
|
||||
| `parallel` | `int` | [LlamaCppConfig](#llamacppconfig) |
|
||||
| `threads` | `int` | [LlamaCppConfig](#llamacppconfig) |
|
||||
@@ -793,6 +799,15 @@ at /v1/images/generations, or meshes, asked for as a job at
|
||||
Nodes are custom nodes the workflow needs, put in each replica's own
|
||||
custom_nodes at the commit named.
|
||||
|
||||
### `environment`
|
||||
|
||||
`string`
|
||||
|
||||
Environment names the models that may share one ComfyUI replica: every
|
||||
model of an environment agrees on the custom nodes they both name, so
|
||||
one replica can carry all of them and ComfyUI moves between their
|
||||
weights itself. Empty is an environment of the model alone.
|
||||
|
||||
## ComfyUIConfig
|
||||
|
||||
### `dir`
|
||||
@@ -1299,6 +1314,15 @@ to sign in, and without accounts the portal needs an API key.
|
||||
|
||||
## License
|
||||
|
||||
### `id`
|
||||
|
||||
`string`
|
||||
|
||||
ID names the licence for a machine: an SPDX identifier where there is
|
||||
one (MIT, Apache-2.0, CC-BY-NC-4.0), else a lowercase name of the
|
||||
licence's own (stability-community, tencent-hunyuan-community). What a
|
||||
person's or a site's choice of licences matches on. Name is for people.
|
||||
|
||||
### `name`
|
||||
|
||||
`string`
|
||||
@@ -1316,6 +1340,35 @@ Name is the licence's own: MIT, Apache-2.0.
|
||||
Notice is what must accompany what was made: attribution, terms on
|
||||
the output, what the weights were trained on.
|
||||
|
||||
### `commercial`
|
||||
|
||||
`*bool`
|
||||
|
||||
Commercial says whether what is made may be used commercially: nil
|
||||
is not said, which is not the same as yes.
|
||||
|
||||
### `excluded_regions`
|
||||
|
||||
`[]string`
|
||||
|
||||
ExcludedRegions are where the licence grants nothing: ISO 3166 codes,
|
||||
and EU for the union (Hunyuan's: EU, UK, KR).
|
||||
|
||||
### `revenue_cap_usd`
|
||||
|
||||
`int64`
|
||||
|
||||
RevenueCapUSD is the yearly revenue above which the licence asks for
|
||||
another agreement (Stability's community licence: 1000000). Zero is
|
||||
none.
|
||||
|
||||
### `attribution`
|
||||
|
||||
`string`
|
||||
|
||||
Attribution is what a product using the output must show: "Powered by
|
||||
Stability AI".
|
||||
|
||||
## LlamaCppConfig
|
||||
|
||||
### `bin`
|
||||
|
||||
@@ -518,12 +518,53 @@ func (m Model) MakesMeshes() bool {
|
||||
// License is the terms what a model makes comes under: said beside every
|
||||
// result, and written into a mesh, which travels without the API.
|
||||
type License struct {
|
||||
// ID names the licence for a machine: an SPDX identifier where there is
|
||||
// one (MIT, Apache-2.0, CC-BY-NC-4.0), else a lowercase name of the
|
||||
// licence's own (stability-community, tencent-hunyuan-community). What a
|
||||
// person's or a site's choice of licences matches on. Name is for people.
|
||||
ID string `json:"id,omitempty" koanf:"id"`
|
||||
// Name is the licence's own: MIT, Apache-2.0.
|
||||
Name string `json:"name" koanf:"name"`
|
||||
URL string `json:"url,omitempty" koanf:"url"`
|
||||
// Notice is what must accompany what was made: attribution, terms on
|
||||
// the output, what the weights were trained on.
|
||||
Notice string `json:"notice,omitempty" koanf:"notice"`
|
||||
// Commercial says whether what is made may be used commercially: nil
|
||||
// is not said, which is not the same as yes.
|
||||
Commercial *bool `json:"commercial,omitempty" koanf:"commercial"`
|
||||
// ExcludedRegions are where the licence grants nothing: ISO 3166 codes,
|
||||
// and EU for the union (Hunyuan's: EU, UK, KR).
|
||||
ExcludedRegions []string `json:"excluded_regions,omitempty" koanf:"excluded_regions"`
|
||||
// RevenueCapUSD is the yearly revenue above which the licence asks for
|
||||
// another agreement (Stability's community licence: 1000000). Zero is
|
||||
// none.
|
||||
RevenueCapUSD int64 `json:"revenue_cap_usd,omitempty" koanf:"revenue_cap_usd"`
|
||||
// Attribution is what a product using the output must show: "Powered by
|
||||
// Stability AI".
|
||||
Attribution string `json:"attribution,omitempty" koanf:"attribution"`
|
||||
}
|
||||
|
||||
var (
|
||||
licenseID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9.+-]{0,63}$`)
|
||||
regionID = regexp.MustCompile(`^[A-Z]{2}$`)
|
||||
)
|
||||
|
||||
func (l License) validate(model string) error {
|
||||
if strings.TrimSpace(l.Name) == "" {
|
||||
return fmt.Errorf("model %q: license.name is required: a licence nobody can name is none", model)
|
||||
}
|
||||
if l.ID != "" && !licenseID.MatchString(l.ID) {
|
||||
return fmt.Errorf("model %q: license.id %q is not an SPDX-style identifier (letters, digits, '.', '+', '-')", model, l.ID)
|
||||
}
|
||||
for _, r := range l.ExcludedRegions {
|
||||
if !regionID.MatchString(r) {
|
||||
return fmt.Errorf("model %q: license.excluded_regions %q is not an ISO 3166 code in capitals (EU for the union)", model, r)
|
||||
}
|
||||
}
|
||||
if l.RevenueCapUSD < 0 {
|
||||
return fmt.Errorf("model %q: license.revenue_cap_usd is negative", model)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// EngineComfyUI is the engine that runs ComfyUI on a node.
|
||||
@@ -555,6 +596,11 @@ type ComfyUI struct {
|
||||
// Nodes are custom nodes the workflow needs, put in each replica's own
|
||||
// custom_nodes at the commit named.
|
||||
Nodes []CustomNode `json:"nodes,omitempty" koanf:"nodes"`
|
||||
// Environment names the models that may share one ComfyUI replica: every
|
||||
// model of an environment agrees on the custom nodes they both name, so
|
||||
// one replica can carry all of them and ComfyUI moves between their
|
||||
// weights itself. Empty is an environment of the model alone.
|
||||
Environment string `json:"environment,omitempty" koanf:"environment"`
|
||||
}
|
||||
|
||||
// CustomNode is a ComfyUI node from a git repository.
|
||||
@@ -763,6 +809,9 @@ func New(models []Model) (*Catalog, error) {
|
||||
}
|
||||
canaryOf[target.Name] = name
|
||||
}
|
||||
if _, err := c.environments(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
@@ -897,6 +946,11 @@ func (m Model) validate() error {
|
||||
if m.Source == "" {
|
||||
return fmt.Errorf("model %q: source is required", m.Name)
|
||||
}
|
||||
if m.License != nil {
|
||||
if err := m.License.validate(m.Name); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
// A page, not weights: a node fetches the HTML as one file and the
|
||||
// engine fails at start, on a path that looks right. For vLLM it was
|
||||
// read as the repository id when the model was made; only a source
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
package catalog
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Environments: the ComfyUI models that may share one replica.
|
||||
//
|
||||
// Every ComfyUI model used to run in a replica of its own, so two models on
|
||||
// one card took turns with a cold start each time: flux, then a mesh model,
|
||||
// then flux again, each loading ComfyUI and its weights from nothing. ComfyUI
|
||||
// itself moves between models' weights in one process; what keeps two models
|
||||
// out of one process is their custom nodes, which share one Python. An
|
||||
// environment is a set of models whose nodes can: the catalog checks it, and
|
||||
// placement may then run one replica for all of them.
|
||||
|
||||
// Environment is one environment's models and what a replica of it carries.
|
||||
type Environment struct {
|
||||
Name string `json:"name"`
|
||||
// Models are the catalog models in it, in catalog order.
|
||||
Models []string `json:"models"`
|
||||
// Nodes are every custom node any of them names, once each.
|
||||
Nodes []CustomNode `json:"nodes,omitempty"`
|
||||
// Sources are every weights source its models load, once each: what a
|
||||
// replica of it must have on its node.
|
||||
Sources []string `json:"sources,omitempty"`
|
||||
}
|
||||
|
||||
// ComfyEnvironment is the environment a ComfyUI model is in, or "" for one in
|
||||
// none, which is an environment of its own.
|
||||
func (m Model) ComfyEnvironment() string {
|
||||
if m.ComfyUI == nil {
|
||||
return ""
|
||||
}
|
||||
return m.ComfyUI.Environment
|
||||
}
|
||||
|
||||
var envName = regexp.MustCompile(`^[a-z0-9][a-z0-9._-]{0,62}$`)
|
||||
|
||||
// Environments are the catalog's named environments, by name.
|
||||
func (c *Catalog) Environments() []Environment {
|
||||
envs, _ := c.environments()
|
||||
return envs
|
||||
}
|
||||
|
||||
// EnvironmentOf is the environment a model is in, when it is in a named one.
|
||||
func (c *Catalog) EnvironmentOf(model string) (Environment, bool) {
|
||||
m, ok := c.Resolve(model)
|
||||
if !ok || m.ComfyEnvironment() == "" {
|
||||
return Environment{}, false
|
||||
}
|
||||
for _, e := range c.Environments() {
|
||||
if e.Name == m.ComfyEnvironment() {
|
||||
return e, true
|
||||
}
|
||||
}
|
||||
return Environment{}, false
|
||||
}
|
||||
|
||||
// environments builds the named environments and refuses one whose models
|
||||
// cannot share a replica: two commits of one node, or two ranges of one
|
||||
// package, of which one Python can hold only one.
|
||||
func (c *Catalog) environments() ([]Environment, error) {
|
||||
byName := map[string]*Environment{}
|
||||
var order []string
|
||||
// Who brought each node and each package into each environment, to name
|
||||
// both models when two disagree.
|
||||
nodeFrom := map[string]map[string]string{} // env -> repo -> model
|
||||
nodeAt := map[string]map[string]string{} // env -> repo -> commit
|
||||
pkgFrom := map[string]map[string]string{} // env -> package -> model
|
||||
pkgSpec := map[string]map[string]string{} // env -> package -> spec
|
||||
for _, name := range c.order {
|
||||
m := c.models[name]
|
||||
env := m.ComfyEnvironment()
|
||||
if env == "" {
|
||||
continue
|
||||
}
|
||||
if !envName.MatchString(env) {
|
||||
return nil, fmt.Errorf("model %q: comfyui.environment %q is not a name of lowercase letters, digits, '.', '_' and '-'",
|
||||
name, env)
|
||||
}
|
||||
if !m.ServesComfyUI() || m.External != nil {
|
||||
return nil, fmt.Errorf("model %q: comfyui.environment is for a model the cluster's ComfyUI serves", name)
|
||||
}
|
||||
e := byName[env]
|
||||
if e == nil {
|
||||
e = &Environment{Name: env}
|
||||
byName[env] = e
|
||||
order = append(order, env)
|
||||
nodeFrom[env], nodeAt[env] = map[string]string{}, map[string]string{}
|
||||
pkgFrom[env], pkgSpec[env] = map[string]string{}, map[string]string{}
|
||||
}
|
||||
e.Models = append(e.Models, name)
|
||||
for _, src := range m.WeightSources() {
|
||||
if !slices.Contains(e.Sources, src) {
|
||||
e.Sources = append(e.Sources, src)
|
||||
}
|
||||
}
|
||||
for _, n := range m.ComfyUI.Nodes {
|
||||
repo := strings.TrimSuffix(strings.TrimRight(n.Repo, "/"), ".git")
|
||||
if at, seen := nodeAt[env][repo]; seen {
|
||||
if at != n.Commit {
|
||||
return nil, fmt.Errorf("environment %q: models %q and %q name custom node %s at different commits (%s, %s): "+
|
||||
"one replica holds one of them", env, nodeFrom[env][repo], name, n.Repo, short(at), short(n.Commit))
|
||||
}
|
||||
} else {
|
||||
nodeAt[env][repo], nodeFrom[env][repo] = n.Commit, name
|
||||
e.Nodes = append(e.Nodes, CustomNode{Repo: n.Repo, Commit: n.Commit})
|
||||
}
|
||||
for _, r := range n.Requirements {
|
||||
pkg, spec := requirementName(r), normalizeSpec(r)
|
||||
if prev, seen := pkgSpec[env][pkg]; seen {
|
||||
if prev != spec {
|
||||
return nil, fmt.Errorf("environment %q: models %q and %q ask for %s as %q and %q: "+
|
||||
"one Python holds one version; name the same range in both",
|
||||
env, pkgFrom[env][pkg], name, pkg, prev, spec)
|
||||
}
|
||||
continue
|
||||
}
|
||||
pkgSpec[env][pkg], pkgFrom[env][pkg] = spec, name
|
||||
// The requirement goes with the node that first named it:
|
||||
// what the replica installs is the union, and each once.
|
||||
i := slices.IndexFunc(e.Nodes, func(x CustomNode) bool {
|
||||
return strings.TrimSuffix(strings.TrimRight(x.Repo, "/"), ".git") == repo
|
||||
})
|
||||
e.Nodes[i].Requirements = append(e.Nodes[i].Requirements, strings.TrimSpace(r))
|
||||
}
|
||||
}
|
||||
}
|
||||
out := make([]Environment, 0, len(order))
|
||||
for _, n := range order {
|
||||
out = append(out, *byName[n])
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// requirementName is a requirement's package, as PyPI compares names.
|
||||
func requirementName(spec string) string {
|
||||
s := strings.TrimSpace(spec)
|
||||
if i := strings.IndexAny(s, "<>=!~;[ "); i >= 0 {
|
||||
s = s[:i]
|
||||
}
|
||||
return strings.NewReplacer("_", "-", ".", "-").Replace(strings.ToLower(s))
|
||||
}
|
||||
|
||||
// normalizeSpec is a requirement without its spaces and with its package
|
||||
// named as PyPI compares it, so "transformers >=4.50, <5" and
|
||||
// "Transformers>=4.50,<5" are one range.
|
||||
func normalizeSpec(spec string) string {
|
||||
s := strings.Join(strings.Fields(spec), "")
|
||||
name := requirementName(s)
|
||||
i := strings.IndexAny(s, "<>=!~;[")
|
||||
if i < 0 {
|
||||
return name
|
||||
}
|
||||
return name + s[i:]
|
||||
}
|
||||
|
||||
func short(commit string) string {
|
||||
if len(commit) > 12 {
|
||||
return commit[:12]
|
||||
}
|
||||
return commit
|
||||
}
|
||||
@@ -0,0 +1,114 @@
|
||||
package catalog
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
const (
|
||||
commitA = "0123456789abcdef0123456789abcdef01234567"
|
||||
commitB = "fedcba9876543210fedcba9876543210fedcba98"
|
||||
)
|
||||
|
||||
func inEnv(name, env, source string, nodes ...CustomNode) Model {
|
||||
m := comfyModel(source, &ComfyUI{Workflow: "w.json", Output: OutputMeshes, Environment: env, Nodes: nodes})
|
||||
m.Name = name
|
||||
return m
|
||||
}
|
||||
|
||||
func TestModelsInOneEnvironmentShareTheirNodesAndWeights(t *testing.T) {
|
||||
sf3d := CustomNode{Repo: "https://github.com/Stability-AI/stable-fast-3d", Commit: commitA,
|
||||
Requirements: []string{"trimesh", "transformers>=4.42,<5"}}
|
||||
tripo := CustomNode{Repo: "https://github.com/flowtyone/ComfyUI-Flowty-TripoSR", Commit: commitB,
|
||||
Requirements: []string{"Transformers >= 4.42, <5", "rembg"}}
|
||||
flux := comfyModel("/w/flux.safetensors", &ComfyUI{Environment: "shared"})
|
||||
cat, err := New([]Model{
|
||||
flux,
|
||||
inEnv("sf3d", "shared", "/w/sf3d/model.safetensors", sf3d),
|
||||
inEnv("triposr", "shared", "/w/tripo/model.ckpt", tripo),
|
||||
inEnv("alone", "", "/w/alone.ckpt"),
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
envs := cat.Environments()
|
||||
if len(envs) != 1 {
|
||||
t.Fatalf("environments %+v", envs)
|
||||
}
|
||||
e := envs[0]
|
||||
if strings.Join(e.Models, ",") != "flux,sf3d,triposr" {
|
||||
t.Errorf("models %v", e.Models)
|
||||
}
|
||||
if len(e.Sources) != 3 || len(e.Nodes) != 2 {
|
||||
t.Errorf("sources %v nodes %+v", e.Sources, e.Nodes)
|
||||
}
|
||||
// transformers once, with the node that named it first; rembg with its own.
|
||||
if got := strings.Join(e.Nodes[0].Requirements, " "); got != "trimesh transformers>=4.42,<5" {
|
||||
t.Errorf("first node's requirements %q", got)
|
||||
}
|
||||
if got := strings.Join(e.Nodes[1].Requirements, " "); got != "rembg" {
|
||||
t.Errorf("second node's requirements %q", got)
|
||||
}
|
||||
if got, ok := cat.EnvironmentOf("sf3d"); !ok || got.Name != "shared" {
|
||||
t.Errorf("sf3d's environment %+v %v", got, ok)
|
||||
}
|
||||
if _, ok := cat.EnvironmentOf("alone"); ok {
|
||||
t.Error("a model with no environment is in one")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnEnvironmentWhoseModelsDisagreeIsRefused(t *testing.T) {
|
||||
node := func(commit string, reqs ...string) CustomNode {
|
||||
return CustomNode{Repo: "https://github.com/a/node", Commit: commit, Requirements: reqs}
|
||||
}
|
||||
for name, c := range map[string]struct {
|
||||
models []Model
|
||||
want string
|
||||
}{
|
||||
"two commits of one node": {[]Model{
|
||||
inEnv("a", "e", "/w/a", node(commitA)), inEnv("b", "e", "/w/b", node(commitB))}, "different commits"},
|
||||
"two ranges of one package": {[]Model{
|
||||
inEnv("a", "e", "/w/a", node(commitA, "numpy<2")),
|
||||
inEnv("b", "e", "/w/b", CustomNode{Repo: "https://github.com/b/other", Commit: commitB,
|
||||
Requirements: []string{"numpy>=2"}})}, `"a" and "b" ask for numpy`},
|
||||
"a bad name": {[]Model{inEnv("a", "Shared Env", "/w/a")}, "comfyui.environment"},
|
||||
} {
|
||||
if _, err := New(c.models); err == nil || !strings.Contains(err.Error(), c.want) {
|
||||
t.Errorf("%s: %v", name, err)
|
||||
}
|
||||
}
|
||||
// The same node at the same commit, the same range written two ways: fine.
|
||||
if _, err := New([]Model{
|
||||
inEnv("a", "e", "/w/a", node(commitA, "numpy >= 2")),
|
||||
inEnv("b", "e", "/w/b", node(commitA, "NumPy>=2")),
|
||||
}); err != nil {
|
||||
t.Errorf("agreeing models refused: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestALicenceIsSaidSoAMachineCanMatchIt(t *testing.T) {
|
||||
yes := true
|
||||
ok := License{ID: "tencent-hunyuan-community", Name: "Tencent Hunyuan Community License", Commercial: &yes,
|
||||
ExcludedRegions: []string{"EU", "GB", "KR"}}
|
||||
m := comfyModel("/w/f.safetensors", nil)
|
||||
m.License = &ok
|
||||
if _, err := New([]Model{m}); err != nil {
|
||||
t.Fatalf("a full licence refused: %v", err)
|
||||
}
|
||||
for name, c := range map[string]struct {
|
||||
l License
|
||||
want string
|
||||
}{
|
||||
"no name": {License{ID: "MIT"}, "license.name"},
|
||||
"a sentence id": {License{ID: "the MIT one", Name: "MIT"}, "license.id"},
|
||||
"a lowercase eu": {License{Name: "x", ExcludedRegions: []string{"eu"}}, "excluded_regions"},
|
||||
"a negative cap": {License{Name: "x", RevenueCapUSD: -1}, "revenue_cap_usd"},
|
||||
} {
|
||||
m := comfyModel("/w/f.safetensors", nil)
|
||||
l := c.l
|
||||
m.License = &l
|
||||
if _, err := New([]Model{m}); err == nil || !strings.Contains(err.Error(), c.want) {
|
||||
t.Errorf("%s: %v", name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -72,6 +72,7 @@ var Reference = []Block{
|
||||
{Key: "checkpoint", Type: "string", Doc: "Checkpoint is the file the workflow's {{checkpoint}} names: the\nsource's own name unless set, which a source that is a set of files\nmust.", Nested: ""},
|
||||
{Key: "output", Type: "string", Doc: "Output is what the workflow makes: pictures (the default), asked for\nat /v1/images/generations, or meshes, asked for as a job at\n/v1/3d/generations from a picture.", Nested: ""},
|
||||
{Key: "nodes", Type: "[]CustomNode", Doc: "Nodes are custom nodes the workflow needs, put in each replica's own\ncustom_nodes at the commit named.", Nested: ""},
|
||||
{Key: "environment", Type: "string", Doc: "Environment names the models that may share one ComfyUI replica: every\nmodel of an environment agrees on the custom nodes they both name, so\none replica can carry all of them and ComfyUI moves between their\nweights itself. Empty is an environment of the model alone.", Nested: ""},
|
||||
}},
|
||||
{Package: "engines", Name: "ComfyUIConfig", Title: "ComfyUIConfig", Doc: "", Fields: []Field{
|
||||
{Key: "dir", Type: "string", Doc: "Dir is ComfyUI's directory, where main.py is. Readable by the user\nengines run as.", Nested: ""},
|
||||
@@ -171,9 +172,14 @@ var Reference = []Block{
|
||||
{Key: "docs_sign_in", Type: "bool", Doc: "DocsSignIn serves the docs portal at /docs only to someone signed in.\nOff, the portal is public like this page: its client guide is for the\nperson who has no account yet, and the operator documents describe\nllmgw in general — no address, model or key of this installation —\nwhich the project's repository publishes anyway. On, a visitor is sent\nto sign in, and without accounts the portal needs an API key.", Nested: ""},
|
||||
}},
|
||||
{Package: "catalog", Name: "License", Title: "License", Doc: "", Fields: []Field{
|
||||
{Key: "id", Type: "string", Doc: "ID names the licence for a machine: an SPDX identifier where there is\none (MIT, Apache-2.0, CC-BY-NC-4.0), else a lowercase name of the\nlicence's own (stability-community, tencent-hunyuan-community). What a\nperson's or a site's choice of licences matches on. Name is for people.", Nested: ""},
|
||||
{Key: "name", Type: "string", Doc: "Name is the licence's own: MIT, Apache-2.0.", Nested: ""},
|
||||
{Key: "url", Type: "string", Doc: "", Nested: ""},
|
||||
{Key: "notice", Type: "string", Doc: "Notice is what must accompany what was made: attribution, terms on\nthe output, what the weights were trained on.", Nested: ""},
|
||||
{Key: "commercial", Type: "*bool", Doc: "Commercial says whether what is made may be used commercially: nil\nis not said, which is not the same as yes.", Nested: ""},
|
||||
{Key: "excluded_regions", Type: "[]string", Doc: "ExcludedRegions are where the licence grants nothing: ISO 3166 codes,\nand EU for the union (Hunyuan's: EU, UK, KR).", Nested: ""},
|
||||
{Key: "revenue_cap_usd", Type: "int64", Doc: "RevenueCapUSD is the yearly revenue above which the licence asks for\nanother agreement (Stability's community licence: 1000000). Zero is\nnone.", Nested: ""},
|
||||
{Key: "attribution", Type: "string", Doc: "Attribution is what a product using the output must show: \"Powered by\nStability AI\".", Nested: ""},
|
||||
}},
|
||||
{Package: "engines", Name: "LlamaCppConfig", Title: "LlamaCppConfig", Doc: "", Fields: []Field{
|
||||
{Key: "bin", Type: "string", Doc: "Bin is the llama-server binary. It is the server, not the CLI: the\ngateway talks OpenAI over HTTP and needs the one that serves it.\nEmpty means the agent installs llama.cpp itself, at the version the\ncontrol plane asks for. Setting it opts out of that.", Nested: ""},
|
||||
|
||||
Reference in new issue
Block a user