MIG: a card in MIG mode is placed as its instances, each a GPU of class <card>-mig-<profile>

Discovery reports a card in MIG mode as its instances instead of the whole
card, on both paths: nvidia-smi (-L names them, -q -x gives their memory,
ids and processes) and NVML (GetMigMode, GetMaxMigDeviceCount,
GetMigDeviceHandleByIndex, behind an interface so it is tested with a fake).
Each instance has its MIG UUID, its own memory, and a class from the card's
and the profile: a100-80g-mig-3g.40gb. Placement numbers instances
1000 + 100 x card + instance and every view shows them as card/migN.

A replica on MIG runs on exactly one instance: a process opens one MIG
device and instances have no NVLink between them, so tp, pp, gpus or nodes
above one on a MIG class are refused in catalog validation and again in
placement, and the planner offers one instance or none. The agent gives
the engine CUDA_VISIBLE_DEVICES=MIG-<uuid> (no Vulkan device variable on
MIG). Memory and processes are per instance; heat, power and wear the
card's; utilisation is left out rather than reported as zero. A card in MIG
mode with nothing carved stays out of placement with the reason, and
llmgw doctor reports each card's mode and instances and warns of the states
in which nothing can run. ctl nodes, ctl status, the console and the plan
count devices by class. docs/install.md has a MIG section.

Written against fakes of nvidia-smi and NVML output; not yet run on MIG
hardware.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 00:01:14 +02:00
1 parent e81f1cba4c
commit 0fd55a862b
39 files changed
+2587 -327

No files matched your search

+24 -2
View File
@@ -14,6 +14,7 @@ import (
"github.com/spf13/cobra"
"gitea.mynodes.xyz/ilker/llmgw/internal/agent/discovery"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
)
@@ -212,7 +213,7 @@ func newNodesCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
// at when a model will not place and the hardware looks
// fine.
join(n.Engines),
join(n.Classes), join(n.Fabrics), n.Replicas, n.LastSeenAgo)
classesColumn(n), join(n.Fabrics), n.Replicas, n.LastSeenAgo)
}
_ = w.Flush()
@@ -410,7 +411,9 @@ func newReplicasCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
for _, r := range reps {
var nodes []string
for _, s := range r.Shards {
nodes = append(nodes, fmt.Sprintf("%s:%v", s.Node, s.GPUs))
// "[0/mig2]" for a MIG instance, not the 1002 it is
// numbered by.
nodes = append(nodes, fmt.Sprintf("%s:%v", s.Node, discovery.DeviceLabels(s.GPUs)))
}
// The state says stale rather than a column of its own: most
// replicas are not, a column would be empty for all of them,
@@ -540,6 +543,25 @@ func emitJSON(v any) error {
return enc.Encode(v)
}
// classesColumn is a node's GPU classes, with how many of each when it has
// more than one or MIG instances among them: a card cut into two 3g.40gb
// instances beside a whole card is "a100-80g-mig-3g.40gb×2,a100-80g×1", where
// the GPUS column's 3 alone would read as three of something.
func classesColumn(n adminapi.NodeView) string {
counted := len(n.ClassGPUs) > 1
for _, c := range n.ClassGPUs {
counted = counted || c.MIG
}
if !counted {
return join(n.Classes)
}
parts := make([]string, len(n.ClassGPUs))
for i, c := range n.ClassGPUs {
parts[i] = fmt.Sprintf("%s×%d", c.Class, c.Total)
}
return strings.Join(parts, ",")
}
func join(v []string) string {
if len(v) == 0 {
return "-"