A picture or 3D model is added by its name: llmgw ctl model add triposr

flux-schnell and triposr come with llmgw as presets: their workflow,
custom nodes, packages and licence as they were found and tried on the
PoC, and a profile that says the card it needs rather than a class, so the
entry places on any site's cards. TripoSR's workflow is built in, with the
LLMGWPrepareForMesh node that keeps a picture's background out of the
mesh, which the copy in the document had left out.

`ctl model add PRESET` adds one, and has the nodes install ComfyUI when
the cluster's settings name none; `ctl model presets` lists them; the
console's add-model dialog offers them beside a definition file. A
ComfyUI build is now also checked to keep its order on a Turing card, so
a node's installed cu13 is not replaced by the cu126 build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-13 10:21:28 +02:00
1 parent 52898fbb80
commit 40b8948d7d
12 files changed
+537 -36

No files matched your search

+37 -7
View File
@@ -1300,6 +1300,22 @@ waits, and sends the pictures back as `b64_json`.
**On the cluster's own cards.** Use engine `comfyui`. The model is placed on a
GPU node like any other, started when asked for, and stopped when unused.
**The short way: a preset.** Some models come with llmgw, with their
workflow, custom nodes, packages and licence already found and tried:
```
llmgw ctl model presets
llmgw ctl model add flux-schnell # pictures, Apache 2.0, a card of 24 GB, Volta or newer
llmgw ctl model add triposr # 3D models from a picture, MIT, a card of 8 GB
```
A preset's profile says the card it needs, not a class, so it places on
whatever cards the site has. When the cluster's settings name no ComfyUI,
adding one sets `engines.comfyui` to the version this release installs, and
every node with an NVIDIA GPU installs it; a cluster that names a version,
or `""` for none, keeps it. In the console it is **A** on the models screen.
The rest of this section is for a model of your own.
The node must offer the engine. Ask for it in the cluster's settings, and every
node with an NVIDIA GPU installs it:
@@ -1312,8 +1328,10 @@ digest llmgw holds for that version. GitHub publishes none for a tag's archive,
so a version llmgw does not know is refused. The agent then builds a Python
environment beside the source, under `/opt/llmgw/engines/comfyui/`. Its
dependencies are resolved as PyPI stood the day after the release. torch comes
from PyPI (CUDA 13), or from PyTorch's CUDA 12.9 build where the driver is older
than 580. The install is checked on the node's GPU, and takes about 10 GB.
from PyPI (CUDA 13), from PyTorch's CUDA 12.9 build where the driver is older
than 580, or from its CUDA 12.6 build on cards older than Turing (a V100):
torch's CUDA 12.8 and newer builds have no kernels for them. The install is
checked on the node's GPU, and takes about 10 GB.
`engine_overrides` can keep it off nodes that should not have it.
Alternatively, install ComfyUI by hand. You need a directory with `main.py` and
@@ -2114,38 +2132,50 @@ Its workflow is a JSON file in ComfyUI's API format: `{{image}}` is the
uploaded picture, which a `LoadImage` node reads; `{{seed}}`,
`{{resolution|256}}` and `{{checkpoint}}` are as for pictures. The node that
saves the mesh may save a GLB or an OBJ; an OBJ leaves the gateway as a GLB,
vertex colours kept. TripoSR (MIT) on any card of 8 GB:
vertex colours kept.
TripoSR (MIT), on any card of 8 GB, comes with llmgw: `llmgw ctl model add
triposr`. What that adds is this entry, and its workflow is built in:
```yaml
- name: triposr
source: hf:stabilityai/TripoSR/model.ckpt
comfyui:
workflow: /etc/llmgw/workflows/triposr.json # on every control plane
workflow: triposr # built in; a model of your own names a JSON file on every control plane
output: meshes
nodes:
- repo: https://github.com/flowtyone/ComfyUI-Flowty-TripoSR
commit: a0c94ac60a7cc062604f61aeeea6d0d493521de3
requirements: [omegaconf, trimesh, scikit-image, imageio,
"transformers>=4.50.3,<5", "huggingface-hub<1", tokenizers]
"transformers>=4.50.3,<5", "huggingface-hub<1", tokenizers,
rembg, onnxruntime]
license:
id: MIT
name: MIT
commercial: true
url: https://huggingface.co/stabilityai/TripoSR/blob/main/LICENSE
notice: "Made with TripoSR (Stability AI and Tripo AI, MIT). ..."
scaling: {min: 0, max: 1, idle_timeout: 10m}
profiles:
- {name: gpu, engine: comfyui, gpu_class: 6000-24g, tp: 1}
- {name: gpu, engine: comfyui, tp: 1, gpu: {min_compute_capability: 7.0, min_memory_gb: 8}}
```
```json
{
"1": {"class_type": "LoadImage", "inputs": {"image": "{{image}}"}},
"5": {"class_type": "LLMGWPrepareForMesh", "inputs": {"image": ["1", 0], "mask": ["1", 1],
"foreground_ratio": 0.85, "background": 0.5, "remove_background": "auto", "rembg_model": "u2net"}},
"2": {"class_type": "TripoSRModelLoader", "inputs": {"model": "{{checkpoint}}", "chunk_size": 8192}},
"3": {"class_type": "TripoSRSampler", "inputs": {"model": ["2", 0], "reference_image": ["1", 0],
"3": {"class_type": "TripoSRSampler", "inputs": {"model": ["2", 0], "reference_image": ["5", 0],
"geometry_resolution": "{{resolution|256}}", "threshold": 25.0}},
"4": {"class_type": "TripoSRViewer", "inputs": {"mesh": ["3", 0]}}
}
```
`LLMGWPrepareForMesh` comes with the gate: it cuts the object out — by the
picture's alpha, or by rembg where it has none — and centres it on a plain
background, without which the background becomes geometry.
- **Custom nodes** are named by https repository and full commit (a branch
is refused: it moves). The gate puts each in the replica's own
`custom_nodes` and installs its `requirements` beside the replica, never
+26
View File
@@ -149,3 +149,29 @@ func TestARequirementsFileIsReadForPackagesAndNothingElse(t *testing.T) {
}
}
}
// A card every build runs on keeps the order it had: a Turing node's
// installed cu13 is still the first found, and is not installed again
// because a build for older cards was added after it.
func TestComfyUIBuildsAreTriedBestFirst(t *testing.T) {
variants := func(h discovery.Host) string {
builds, err := comfyuiCandidates("0.39.2", "", h)
if err != nil {
return err.Error()
}
var v []string
for _, b := range builds {
v = append(v, b.Variant)
}
return strings.Join(v, ",")
}
volta := turingNode
volta.MinComputeCapability = 7.0
older := turingNode
older.NVIDIADriver = "550.163.01"
for h, want := range map[*discovery.Host]string{&turingNode: "cu13,cu129,cu126", &volta: "cu126", &older: "cu129,cu126"} {
if got := variants(*h); got != want {
t.Errorf("cc %.1f driver %s: %s, want %s", h.MinComputeCapability, h.NVIDIADriver, got, want)
}
}
}
+12
View File
@@ -592,6 +592,18 @@ var comfyuiReleases = map[string]struct {
"0.39.2": {"1d655e22cf546700fc1d6d2536e27701695c5832a51f63bb16c41866ac09be0b", "2026-10-07T21:58:07Z"},
}
// ComfyUIVersion is the newest ComfyUI this agent installs: what a cluster
// is set to when a model first needs it.
func ComfyUIVersion() string {
newest := ""
for v := range comfyuiReleases {
if newest == "" || versionBefore(newest, v) {
newest = v
}
}
return newest
}
// ComfyUIRepo is where ComfyUI's source is published.
const ComfyUIRepo = "Comfy-Org/ComfyUI"
+52 -22
View File
@@ -32,7 +32,7 @@ func newModelCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
Use: "model",
Short: "Add or remove models while the cluster is running",
}
c.AddCommand(newModelAddCmd(client), newModelRemoveCmd(client), newModelPlanCmd(client),
c.AddCommand(newModelAddCmd(client), newModelPresetsCmd(), newModelRemoveCmd(client), newModelPlanCmd(client),
newResourcesCmd(client), newClassCmd(client), newOwnerCmd(client),
newTargetCmd(client, asJSON), newAdaptersCmd(client),
newModelServeCmd(client), newModelUpdatesCmd(client, asJSON), newModelUpdateCmd(client),
@@ -46,34 +46,42 @@ func newModelAddCmd(client func() *ctlClient) *cobra.Command {
var path string
var skipCheck bool
c := &cobra.Command{
Use: "add -f <file.yaml>",
Short: "Add a model from a definition file",
Long: "The definition is the same YAML a model takes in server.yaml, so an\n" +
Use: "add PRESET | -f <file.yaml>",
Short: "Add a model that comes with llmgw, or one from a definition file",
Long: "A preset is a model that comes with llmgw, its workflow, nodes and licence\n" +
"already found and tried: " + strings.Join(catalog.PresetNames(), ", ") + " (llmgw ctl model presets).\n" +
"Adding a ComfyUI one also has the nodes install ComfyUI, when the cluster\n" +
"has not been told otherwise.\n\n" +
"A definition is the same YAML a model takes in server.yaml, so an\n" +
"entry can be moved between the two by copying it.\n\n" +
"The catalog is rebuilt and validated before anything is written, so a\n" +
"definition that collides with an existing name, or names a fallback that\n" +
"does not exist, is refused rather than stored and then ignored on every\n" +
"start. Once accepted it takes effect on the reconciler's next pass — no\n" +
"restart — and survives one.",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
if path == "" {
return fmt.Errorf("--file is required: the model definition, as YAML")
}
// Read by koanf, the same parser and the same tags that read
// server.yaml — so an entry can be moved between the two by copying
// it, and a field that works in one cannot silently mean nothing in
// the other.
k := koanf.New(".")
if err := k.Load(file.Provider(path), yaml.Parser()); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
Example: " llmgw ctl model add triposr\n" +
" llmgw ctl model add -f qwen.yaml",
Args: cobra.MaximumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
var m catalog.Model
if err := k.UnmarshalWithConf("", &m, koanf.UnmarshalConf{Tag: "koanf"}); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
if m.Name == "" {
return fmt.Errorf("%s: the definition has no name", path)
switch {
case len(args) == 1 && path != "":
return fmt.Errorf("a preset or a file, not both")
case len(args) == 1:
p, ok := catalog.PresetNamed(args[0])
if !ok {
return fmt.Errorf("no preset %q: there are %s; any other model is added from a file (-f)",
args[0], strings.Join(catalog.PresetNames(), ", "))
}
m = p.Model
case path == "":
return fmt.Errorf("a preset (%s) or --file, the model definition as YAML",
strings.Join(catalog.PresetNames(), ", "))
default:
var err error
if m, err = readModelFile(path); err != nil {
return err
}
}
// Before the catalog is touched, because a source that does not
@@ -107,6 +115,9 @@ func newModelAddCmd(client func() *ctlClient) *cobra.Command {
fmt.Fprintf(cmd.OutOrStdout(),
"%s added. It will be placed on the next reconcile pass; watch it with:\n"+
" llmgw ctl models\n", m.Name)
if m.ServesComfyUI() && !m.IsExternal() {
return ensureComfyUI(cmd, client, m.Name)
}
return nil
},
}
@@ -116,6 +127,25 @@ func newModelAddCmd(client func() *ctlClient) *cobra.Command {
return c
}
// readModelFile reads a model's definition. By koanf, the same parser and
// the same tags that read server.yaml — so an entry can be moved between the
// two by copying it, and a field that works in one cannot silently mean
// nothing in the other.
func readModelFile(path string) (catalog.Model, error) {
var m catalog.Model
k := koanf.New(".")
if err := k.Load(file.Provider(path), yaml.Parser()); err != nil {
return m, fmt.Errorf("%s: %w", path, err)
}
if err := k.UnmarshalWithConf("", &m, koanf.UnmarshalConf{Tag: "koanf"}); err != nil {
return m, fmt.Errorf("%s: %w", path, err)
}
if m.Name == "" {
return m, fmt.Errorf("%s: the definition has no name", path)
}
return m, nil
}
func newModelRemoveCmd(client func() *ctlClient) *cobra.Command {
return &cobra.Command{
Use: "remove <name>",
+83
View File
@@ -0,0 +1,83 @@
package cli
import (
"fmt"
"github.com/spf13/cobra"
"gitea.mynodes.xyz/ilker/llmgw/internal/agent/provision"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
"gitea.mynodes.xyz/ilker/llmgw/internal/settings"
)
func newModelPresetsCmd() *cobra.Command {
return &cobra.Command{
Use: "presets",
Short: "The models that come with llmgw, added by name",
Long: "Each is a catalog entry whose workflow, custom nodes, packages and licence\n" +
"were found and tried, and whose profile says the card it needs rather than\n" +
"a class, so it places on any site's cards. Add one with\n" +
"`llmgw ctl model add NAME`.",
Args: cobra.NoArgs,
RunE: func(cmd *cobra.Command, _ []string) error {
w := tabTo(cmd.OutOrStdout())
fmt.Fprintln(w, "NAME\tMAKES\tCARD\tLICENCE\tWHAT IT IS FOR")
for _, p := range catalog.Presets() {
fmt.Fprintf(w, "%s\t%s\t%s\t%s\t%s\n", p.Model.Name, p.Makes, presetCard(p.Model),
presetLicence(p.Model), p.Model.Description)
}
return w.Flush()
},
}
}
func presetCard(m catalog.Model) string {
if len(m.Profiles) == 0 || m.Profiles[0].GPU == nil {
return "—"
}
return m.Profiles[0].GPU.String()
}
func presetLicence(m catalog.Model) string {
if m.License == nil {
return "—"
}
l := m.License.Name
if c := m.License.Commercial; c != nil && !*c {
l += ", not commercial"
}
return l
}
// ensureComfyUI has the nodes install ComfyUI for a model that needs it,
// unless the cluster names a version already, or none on purpose.
func ensureComfyUI(cmd *cobra.Command, client func() *ctlClient, model string) error {
out := cmd.OutOrStdout()
path := []string{"engines", catalog.EngineComfyUI}
var cur adminapi.SettingsView
if err := client().get(cmd.Context(), "/api/v1/settings", &cur); err != nil {
return err
}
doc, err := settings.FromBytes("settings", []byte(cur.Body))
if err != nil {
return err
}
version := provision.ComfyUIVersion()
if n := doc.Node(path); n != nil {
if n.Value == "" {
fmt.Fprintf(out, "engines.comfyui is set to none, so the nodes install no ComfyUI for it: set a version "+
"(llmgw ctl config set engines.comfyui %s) or name one installed by hand in agent.yaml\n", version)
}
return nil
}
fmt.Fprintf(out, "Nodes with an NVIDIA GPU install ComfyUI %s for it, about 10 GB each:\n", version)
if err := doc.SetYAML(path, version); err != nil {
return err
}
b, err := doc.Bytes()
if err != nil {
return err
}
return putSettings(cmd, client, cur, b, "ComfyUI for "+model)
}
+107
View File
@@ -0,0 +1,107 @@
package cli
import (
"bytes"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
)
// presetAdmin is an admin API holding settings, which records the models
// added and the settings put.
type presetAdmin struct {
mu sync.Mutex
settings string
added []catalog.Model
put []string
}
func (a *presetAdmin) client(t *testing.T) func() *ctlClient {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
a.mu.Lock()
defer a.mu.Unlock()
body, _ := io.ReadAll(r.Body)
w.Header().Set("Content-Type", "application/json")
switch {
case r.Method == http.MethodPost && r.URL.Path == "/api/v1/models":
var m catalog.Model
_ = json.Unmarshal(body, &m)
a.added = append(a.added, m)
_ = json.NewEncoder(w).Encode(map[string]string{"name": m.Name})
case r.Method == http.MethodGet && r.URL.Path == "/api/v1/settings":
_ = json.NewEncoder(w).Encode(adminapi.SettingsView{Version: 1, Body: a.settings})
case r.Method == http.MethodPut && r.URL.Path == "/api/v1/settings":
var req adminapi.SettingsRequest
_ = json.Unmarshal(body, &req)
a.put = append(a.put, req.Body)
a.settings = req.Body
_ = json.NewEncoder(w).Encode(adminapi.SettingsView{Version: 2, Body: req.Body})
default:
http.NotFound(w, r)
}
}))
t.Cleanup(srv.Close)
return func() *ctlClient { return newCtlClient(strings.TrimPrefix(srv.URL, "http://"), "") }
}
func runModelAdd(t *testing.T, client func() *ctlClient, args ...string) (string, error) {
t.Helper()
cmd := newModelAddCmd(client)
var out bytes.Buffer
cmd.SetOut(&out)
cmd.SetErr(&out)
cmd.SetArgs(append(args, "--no-source-check"))
err := cmd.ExecuteContext(t.Context())
return out.String(), err
}
func TestAPresetIsAddedByNameAndHasTheNodesInstallComfyUI(t *testing.T) {
a := &presetAdmin{settings: "router:\n retries: 2\n"}
out, err := runModelAdd(t, a.client(t), "triposr")
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
if len(a.added) != 1 || a.added[0].Name != "triposr" || a.added[0].ComfyUI.Workflow != "triposr" {
t.Fatalf("added %+v", a.added)
}
if len(a.put) != 1 || !strings.Contains(a.put[0], "comfyui: 0.39.2") || !strings.Contains(a.put[0], "retries: 2") {
t.Fatalf("settings put %q", a.put)
}
if !strings.Contains(out, "install ComfyUI 0.39.2") {
t.Errorf("not said:\n%s", out)
}
}
func TestAClusterThatNamesItsComfyUIKeepsIt(t *testing.T) {
for _, settings := range []string{"engines:\n comfyui: 0.38.0\n", "engines:\n comfyui: \"\"\n"} {
a := &presetAdmin{settings: settings}
out, err := runModelAdd(t, a.client(t), "flux-schnell")
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
if len(a.put) != 0 {
t.Errorf("%q: settings changed to %q", settings, a.put)
}
if none := strings.Contains(settings, `""`); none != strings.Contains(out, "set to none") {
t.Errorf("%q: %s", settings, out)
}
}
}
func TestAnUnknownPresetNamesTheOnesThereAre(t *testing.T) {
a := &presetAdmin{}
_, err := runModelAdd(t, a.client(t), "sdxl")
if err == nil || !strings.Contains(err.Error(), "flux-schnell, triposr") || len(a.added) != 0 {
t.Errorf("%v, added %v", err, a.added)
}
if _, err := runModelAdd(t, a.client(t), "triposr", "-f", "x.yaml"); err == nil {
t.Error("a preset and a file were both taken")
}
}
+1 -1
View File
@@ -1237,7 +1237,7 @@ func (m consoleModel) panel() (title, body, keys string) {
strings.Join(md.Profiles, ","), upd, warn})
}
if m.inPanel {
keys = "↑↓ row · Enter actions · A add from a file · ← back · Q leave"
keys = "↑↓ row · Enter actions · A add a model · ← back · Q leave"
}
return "Models", "\n" + table([]string{"MODEL", "READY", "STATUS", "CLASS", "PROFILES", "UPDATE", ""}, rows, m.row, pw, m.inPanel), keys
case cReplicas:
+25 -6
View File
@@ -12,6 +12,7 @@ import (
tea "github.com/charmbracelet/bubbletea"
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
)
@@ -253,19 +254,37 @@ func (m consoleModel) mirrorPanel(pw int) (string, string, string) {
return "Mirror", head + table([]string{"MODEL", "VERSION", "FILE", "STATE", "SIZE", "USED BY"}, rows, m.row, pw, m.inPanel), keys
}
// addModelForm adds a model from a definition file on this machine: the
// same YAML a model takes in server.yaml.
// fromFile is the add-model choice that reads a definition file rather
// than naming a preset.
const fromFile = "from a file"
// addModelForm adds a model that comes with llmgw, or one from a definition
// file on this machine: the same YAML a model takes in server.yaml.
func (m *consoleModel) addModelForm() {
server := "--server=" + m.client.base
var note strings.Builder
note.WriteString("A model that comes with llmgw, tried and with its licence:\n")
for _, p := range catalog.Presets() {
fmt.Fprintf(&note, " %s: %s, %s, %s\n", p.Model.Name, p.Makes, presetCard(p.Model), presetLicence(p.Model))
}
note.WriteString("A ComfyUI one has the nodes install ComfyUI if the cluster has none.\n" +
"Or a definition file: the YAML a model takes in server.yaml's models.")
m.openForm(&formDialog{title: "Add a model",
note: "From a definition file on this machine: the YAML a model takes in\nserver.yaml's models. Serve a model, under Do, needs none.",
note: note.String(),
fields: []formField{
textF("file", "", "like /root/qwen.yaml"),
choiceF("model", append(catalog.PresetNames(), fromFile), catalog.PresetNames()[0]),
textF("file", "", "for "+fromFile+": like /root/qwen.yaml"),
},
submit: func(m *consoleModel, v []string) (tea.Cmd, error) {
if _, err := os.Stat(v[0]); err != nil {
if v[0] != fromFile {
return m.runCLI("Add "+v[0], "ctl", server, "model", "add", v[0]), nil
}
if v[1] == "" {
return nil, fmt.Errorf("a file, or a model that comes with llmgw")
}
if _, err := os.Stat(v[1]); err != nil {
return nil, err
}
return m.runCLI("Add a model", "ctl", server, "model", "add", "-f", v[0]), nil
return m.runCLI("Add a model", "ctl", server, "model", "add", "-f", v[1]), nil
}})
}
+27
View File
@@ -15,6 +15,9 @@ import (
func TestTheBuiltInWorkflowsFillIn(t *testing.T) {
for _, name := range Builtins() {
if raw, _ := builtin.ReadFile("workflows/" + name + ".json"); strings.Contains(string(raw), "{{image}}") {
continue // made from a picture: TestTheBuiltInMeshWorkflowFillsIn
}
tpl, err := Load(name)
if err != nil {
t.Fatalf("%s: %v", name, err)
@@ -37,6 +40,30 @@ func TestTheBuiltInWorkflowsFillIn(t *testing.T) {
}
}
func TestTheBuiltInMeshWorkflowFillsIn(t *testing.T) {
tpl, err := Load("triposr")
if err != nil {
t.Fatal(err)
}
g, err := tpl.Fill(Params{Checkpoint: "model.ckpt", Image: "llmgw-1.png"})
if err != nil {
t.Fatal(err)
}
b, _ := json.Marshal(g)
s := string(b)
// The picture is cut out and centred before TripoSR sees it: without
// that, its background becomes geometry.
for _, want := range []string{`"image":"llmgw-1.png"`, `"model":"model.ckpt"`, `"geometry_resolution":256`,
`"class_type":"LLMGWPrepareForMesh"`, `"reference_image":["5",0]`} {
if !strings.Contains(s, want) {
t.Errorf("%s not in %s", want, s)
}
}
if strings.Contains(s, "{{") {
t.Errorf("a placeholder was left: %s", s)
}
}
func TestAStepCountGivenReplacesTheDefault(t *testing.T) {
tpl, _ := Load("flux-schnell")
g, _ := tpl.Fill(Params{Checkpoint: "c", Prompt: "p", Width: 512, Height: 512})
+9
View File
@@ -0,0 +1,9 @@
{
"1": {"class_type": "LoadImage", "inputs": {"image": "{{image}}"}},
"5": {"class_type": "LLMGWPrepareForMesh", "inputs": {"image": ["1", 0], "mask": ["1", 1],
"foreground_ratio": 0.85, "background": 0.5, "remove_background": "auto", "rembg_model": "u2net"}},
"2": {"class_type": "TripoSRModelLoader", "inputs": {"model": "{{checkpoint}}", "chunk_size": 8192}},
"3": {"class_type": "TripoSRSampler", "inputs": {"model": ["2", 0], "reference_image": ["5", 0],
"geometry_resolution": "{{resolution|256}}", "threshold": 25.0}},
"4": {"class_type": "TripoSRViewer", "inputs": {"mesh": ["3", 0]}}
}
+99
View File
@@ -0,0 +1,99 @@
package catalog
import (
"slices"
"time"
)
// Preset is a catalog entry that comes with llmgw: a model whose workflow,
// custom nodes, packages and licence somebody has already found and tried,
// so that adding it is naming it.
//
// A ComfyUI model cannot be worked out from its repository the way a GGUF
// can: which workflow runs it, which node at which commit, which packages at
// which ranges, and under which licence what it makes is shipped are learned
// by running it. Written in a document, they were copied by hand on every
// site, and a line left out — the node that cuts the picture out — gave
// meshes with their background as geometry.
type Preset struct {
// Makes is what it makes: pictures or meshes.
Makes string
Model Model
}
// Presets are the entries that come with llmgw, by name. Each profile says
// the card it needs rather than naming a class, so the entry places on any
// site's cards.
func Presets() []Preset {
yes := true
return []Preset{
{
Makes: OutputPictures,
Model: Model{
Name: "flux-schnell",
Description: "FLUX.1 [schnell]: pictures from a prompt, in four steps",
Source: "hf:Comfy-Org/flux1-schnell/flux1-schnell-fp8.safetensors",
ComfyUI: &ComfyUI{Workflow: "flux-schnell"},
License: &License{
ID: "Apache-2.0", Name: "Apache-2.0", Commercial: &yes,
URL: "https://huggingface.co/black-forest-labs/FLUX.1-schnell",
Notice: "Made with FLUX.1 [schnell] (Black Forest Labs, Apache 2.0).",
},
Scaling: Scaling{Min: 0, Max: 1, IdleTimeout: 15 * time.Minute},
// The checkpoint is 17 GB with its text encoders; a card with
// no fp8 has its weights upcast as they load.
Profiles: []Profile{{Name: "gpu", Engine: EngineComfyUI, TP: 1,
GPU: &GPUNeed{MinComputeCapability: 7.0, MinMemoryGB: 24}}},
},
},
{
Makes: OutputMeshes,
Model: Model{
Name: "triposr",
Description: "TripoSR: a 3D model (GLB) from a picture of one object on a plain background, as a job",
Source: "hf:stabilityai/TripoSR/model.ckpt",
ComfyUI: &ComfyUI{
Workflow: "triposr",
Output: OutputMeshes,
Nodes: []CustomNode{{
Repo: "https://github.com/flowtyone/ComfyUI-Flowty-TripoSR",
Commit: "a0c94ac60a7cc062604f61aeeea6d0d493521de3",
// transformers 5 renames the checkpoint's keys; rembg
// and onnxruntime cut out a picture with no alpha.
Requirements: []string{"omegaconf", "trimesh", "scikit-image", "imageio",
"transformers>=4.50.3,<5", "huggingface-hub<1", "tokenizers", "rembg", "onnxruntime"},
}},
},
License: &License{
ID: "MIT", Name: "MIT", Commercial: &yes,
URL: "https://huggingface.co/stabilityai/TripoSR/blob/main/LICENSE",
Notice: "Made with TripoSR (Stability AI and Tripo AI, MIT). Its image encoder is " +
"facebook/dino-vitb16 (Apache 2.0). TripoSR was trained on a filtered subset of " +
"Objaverse, some of whose objects carry their own licences.",
},
Scaling: Scaling{Min: 0, Max: 1, IdleTimeout: 10 * time.Minute},
Profiles: []Profile{{Name: "gpu", Engine: EngineComfyUI, TP: 1,
GPU: &GPUNeed{MinComputeCapability: 7.0, MinMemoryGB: 8}}},
},
},
}
}
// PresetNamed is the preset of that name.
func PresetNamed(name string) (Preset, bool) {
ps := Presets()
i := slices.IndexFunc(ps, func(p Preset) bool { return p.Model.Name == name })
if i < 0 {
return Preset{}, false
}
return ps[i], true
}
// PresetNames are the presets' names, in order.
func PresetNames() []string {
var out []string
for _, p := range Presets() {
out = append(out, p.Model.Name)
}
return out
}
+59
View File
@@ -0,0 +1,59 @@
package catalog
import (
"testing"
"gitea.mynodes.xyz/ilker/llmgw/internal/comfy"
)
func TestEveryPresetIsACatalogEntryThatLoads(t *testing.T) {
var models []Model
for _, p := range Presets() {
models = append(models, p.Model)
}
cat, err := New(models)
if err != nil {
t.Fatal(err)
}
for _, p := range Presets() {
m, ok := cat.Resolve(p.Model.Name)
if !ok {
t.Fatalf("%s is not in the catalog", p.Model.Name)
}
if p.Makes == OutputMeshes && !m.MakesMeshes() || p.Makes == OutputPictures && !m.MakesImages() {
t.Errorf("%s does not make %s", m.Name, p.Makes)
}
// A shipped entry has its workflow inside llmgw, not in a file a
// site would have to write.
w, ckpt := m.ComfyWorkflow()
if _, err := comfy.Load(w); err != nil {
t.Errorf("%s: %v", m.Name, err)
}
if ckpt == "" {
t.Errorf("%s has no checkpoint", m.Name)
}
if m.License == nil || m.License.ID == "" || m.License.Commercial == nil {
t.Errorf("%s does not say its licence for a machine to match", m.Name)
}
for _, pr := range m.Profiles {
if pr.GPUClass != "" || pr.GPU == nil {
t.Errorf("%s/%s names a class: a preset fits any site's cards", m.Name, pr.Name)
}
}
}
}
func TestAPresetIsFoundByName(t *testing.T) {
if p, ok := PresetNamed("triposr"); !ok || p.Model.ComfyUI.Workflow != "triposr" {
t.Errorf("triposr: %+v %v", p, ok)
}
if _, ok := PresetNamed("sdxl"); ok {
t.Error("a preset that is not there was found")
}
// Each call is a fresh copy: a caller changing one changes nobody's.
a, _ := PresetNamed("triposr")
a.Model.ComfyUI.Nodes[0].Requirements[0] = "x"
if b, _ := PresetNamed("triposr"); b.Model.ComfyUI.Nodes[0].Requirements[0] == "x" {
t.Error("a preset was changed by a caller")
}
}