placement.Plan(sites, request, weights) is a pure function - no I/O, no cloud - which is what makes the decision reproducible, unit-testable against synthetic multi-site fixtures, and usable as a dry-run endpoint later. hpcod plan can run against live sites or a saved discover -json snapshot. Filter then score. Filtering eliminates the impossible and records why for each site; scoring ranks the survivors with capability_fit weighted highest, because scarce hardware must be kept for requests that need it. A plain CPU job placed on the only InfiniBand site is a scheduling failure even though it fits, and the tests assert exactly that. The refusals carry the design's central distinction. A site is told apart as "none present", "present but unconfirmed - the fabric has not been verified by a node agent", and "present but not passthrough-safe (shared IOMMU group)". Those are three different problems needing three different responses: buy hardware, run the agent, fix the topology. Reporting them all as "no capacity" would waste an operator's afternoon. Requests state usable memory; the portal adds the in-RAM rootfs reserve on top, since a tenant allocated exactly what it asked for would OOM under its own job. Packing prefers hosts already in use by the tenant so whole hypervisors stay free for later requests. Verified against real hv01 inventory, where the request for 4 GPUs and InfiniBand is refused with "gpu: need 4, only 1 usable" and "infiniband: none present" - correct, because the ConnectX-4 Lx cards are RoCE and the curated map knows it. Also: an invalid lease now names the field instead of surfacing a bare "invalid duration", which a test caught.
27 lines
449 B
YAML
27 lines
449 B
YAML
# A tenant cluster that needs GPUs and a real fabric.
|
|
name: genomics-week12
|
|
lease: 14d
|
|
|
|
head:
|
|
cores: 4
|
|
memory_gb: 16
|
|
disk_gb: 200
|
|
|
|
compute:
|
|
count: 4
|
|
cores: 8
|
|
memory_gb: 32 # usable; the in-RAM rootfs reserve is added on top
|
|
require:
|
|
- capability: gpu
|
|
count: 1
|
|
vendor: nvidia
|
|
- capability: infiniband
|
|
count: 1
|
|
prefer:
|
|
- capability: nvme
|
|
|
|
stack:
|
|
os: rocky9
|
|
scheduler: slurm
|
|
identity: freeipa
|