MIG: a card in MIG mode is placed as its instances, each a GPU of class <card>-mig-<profile>
Discovery reports a card in MIG mode as its instances instead of the whole card, on both paths: nvidia-smi (-L names them, -q -x gives their memory, ids and processes) and NVML (GetMigMode, GetMaxMigDeviceCount, GetMigDeviceHandleByIndex, behind an interface so it is tested with a fake). Each instance has its MIG UUID, its own memory, and a class from the card's and the profile: a100-80g-mig-3g.40gb. Placement numbers instances 1000 + 100 x card + instance and every view shows them as card/migN. A replica on MIG runs on exactly one instance: a process opens one MIG device and instances have no NVLink between them, so tp, pp, gpus or nodes above one on a MIG class are refused in catalog validation and again in placement, and the planner offers one instance or none. The agent gives the engine CUDA_VISIBLE_DEVICES=MIG-<uuid> (no Vulkan device variable on MIG). Memory and processes are per instance; heat, power and wear the card's; utilisation is left out rather than reported as zero. A card in MIG mode with nothing carved stays out of placement with the reason, and llmgw doctor reports each card's mode and instances and warns of the states in which nothing can run. ctl nodes, ctl status, the console and the plan count devices by class. docs/install.md has a MIG section. Written against fakes of nvidia-smi and NVML output; not yet run on MIG hardware. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
e81f1cba4c
commit
0fd55a862b
39 files changed
+2587
-327
No files matched your search
+302
-182
File diff suppressed because it is too large.
Load diff
@@ -252,6 +252,25 @@ message GPU {
|
||||
string cluster_uuid = 19;
|
||||
string clique_id = 20;
|
||||
string fabric_bandwidth = 21;
|
||||
// mig is set when this is a MIG instance rather than a whole card. index
|
||||
// is then 1000 + 100 × card + device, uuid the instance's (MIG-…), and
|
||||
// memory the instance's share; utilization_percent stays zero, because no
|
||||
// driver measures it per instance.
|
||||
MIGDevice mig = 22;
|
||||
}
|
||||
|
||||
// MIGDevice is which card a MIG instance is carved from, and how.
|
||||
message MIGDevice {
|
||||
int32 parent_index = 1;
|
||||
string parent_uuid = 2;
|
||||
// parent_memory_total_mb is the whole card's: the class is named from it.
|
||||
int32 parent_memory_total_mb = 3;
|
||||
// device is the instance's number on its card, as nvidia-smi -L lists it.
|
||||
int32 device = 4;
|
||||
// profile is NVIDIA's name for the slice, like 3g.40gb.
|
||||
string profile = 5;
|
||||
int32 gpu_instance = 6;
|
||||
int32 compute_instance = 7;
|
||||
}
|
||||
|
||||
message FabricAttachment {
|
||||
|
||||
Reference in new issue
Block a user