ComfyUI models of one environment share one replica
flux and sf3d on one card handed it to each other, each starting ComfyUI and loading its weights from nothing (BACKLOG H3, placement side). An environment of two or more models now gets a hidden catalog entry, comfyui-env-<name>: on its first member's profiles with the most memory any member needs, on demand for the longest idle timeout, carrying every member's weights, repositories and the environment's custom nodes. The members get no replicas of their own; asking for one wants the entry. The agent fetches each member's weights; the gate links all of them, refusing two weights files of one name, links each member's Hugging Face repository, and answers /v1/models for every member. The control plane offers the replica under the members' names with each one's workflow and checkpoint, which the router sends by the model asked for. A member's sibling never evicts the replica serving it, and the members' load counts as the entry's. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
7c53380e68
commit
aa531b1053
22 files changed
+614
-18
No files matched your search
@@ -9292,6 +9292,10 @@
|
||||
"origin": {
|
||||
"$ref": "#/components/schemas/catalog.Origin",
|
||||
"description": "Origin is the registry repository the weights came from, at which commit, and each file's checksum there. Written when a model is served from the registry; for one that was not, worked out from the mirror's record of where it fetched the file. It is what lets the control plane say that the repository has published different weights since, and move the model to them."
|
||||
},
|
||||
"shared": {
|
||||
"$ref": "#/components/schemas/catalog.SharedComfy",
|
||||
"description": "Shared is set on an environment's hidden entry only: the members its replica answers for. Made by the catalog, never written."
|
||||
}
|
||||
}
|
||||
},
|
||||
@@ -9545,6 +9549,51 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"catalog.SharedComfy": {
|
||||
"type": "object",
|
||||
"description": "SharedComfy is what an environment's entry carries of its members.",
|
||||
"properties": {
|
||||
"environment": {
|
||||
"type": "string"
|
||||
},
|
||||
"members": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/catalog.SharedMember"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"catalog.SharedMember": {
|
||||
"type": "object",
|
||||
"description": "SharedMember is one model a shared replica answers for.",
|
||||
"properties": {
|
||||
"name": {
|
||||
"type": "string"
|
||||
},
|
||||
"aliases": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"source": {
|
||||
"type": "string",
|
||||
"description": "Source is the member's weights, which the replica's node fetches."
|
||||
},
|
||||
"hf_repo": {
|
||||
"type": "string",
|
||||
"description": "HFRepo is the repository its node loads the weights by name from."
|
||||
},
|
||||
"workflow": {
|
||||
"type": "string",
|
||||
"description": "Workflow and Checkpoint are what a request for it is sent."
|
||||
},
|
||||
"checkpoint": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"eval.Live": {
|
||||
"type": "object",
|
||||
"description": "Live is a run as it stands: one running, with how many of its cases are done, or one finished.",
|
||||
|
||||
Reference in new issue
Block a user