Judged by the nodes as they are now, through modelplan.Options, the planner that serving a model uses: GGUF at Q4_K_M from the parameter count, safetensors at the size the registry's dtype breakdown gives (packed AWQ, GPTQ and MXFP4 at four bits), FP8 only where a card has compute capability 8.9, MLX safetensors never. A picture model is judged against one card. A cluster with cards asks the registry for safetensors too, a hundred at a time; one without asks for GGUF only. The page says what it judged by: the largest card, its capability, the most RAM on one node. On a repository's page a file set no node holds is marked with what it needs, against what the best node offers, and has no Fetch button. Tried against the PoC's seven nodes and Hugging Face's trending hundred: 63 shown, the rest too big for one 24 GB Turing card or FP8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
e84631ce3a
commit
2a45068461
11 files changed
+458
-18
No files matched your search
@@ -1829,9 +1829,12 @@ func (a *API) setOrigin(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
// PoolNodes is the fleet's ready nodes as a plan sees them: their cards,
|
||||
// which of them are free, the fabric they share, and their memory.
|
||||
func (a *API) PoolNodes() []modelplan.PoolNode {
|
||||
func (a *API) PoolNodes() []modelplan.PoolNode { return PoolNodesOf(a.Nodes()) }
|
||||
|
||||
// PoolNodesOf is nodes as a plan sees them, the ready ones only.
|
||||
func PoolNodesOf(nodes []NodeView) []modelplan.PoolNode {
|
||||
var out []modelplan.PoolNode
|
||||
for _, n := range a.Nodes() {
|
||||
for _, n := range nodes {
|
||||
if n.State != "ready" {
|
||||
continue
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user