Files
werewulf/scripts/B30-cluster-inventory.py
ilker 49ba7ed874 hpcod: separate the cluster from the tenant that owns it
One tenant may run several HPC clusters at the same time. The code did not
allow for that: a single tenant_id drove the bridge, the subnet, the MAC
prefix, the VM names, the vendor-data filename and the public hostname. A
tenant asking for a second cluster while the first was running would have
been handed the same bridge, the same subnet and the same MAC prefix - and
Warewulf matches nodes by MAC, so the second cluster would have broken the
first rather than failing cleanly.

Everything that must not collide is now keyed on cluster_id, and tenant_id
survives as what it always meant: who owns this. Bridges are vmbrC###,
hosts c###-head and c###-node#, the portal is c###.hpc.example.org, and the
Slurm ClusterName is the cluster's - which matters for the accounting work
next, because records attributed to a shared name cannot be separated
afterwards.

The bound of 254 therefore applies to clusters running at one time, not to
tenants. It comes from the addressing and nothing else: cluster_id is the
third octet of 10.64.x.0/24 and one byte of the MAC. That is written down
in 20.5a rather than left to be rediscovered.

The registry keeps its shape but not its name: /etc/hpcod/clusters.yaml
with a clusters: key, /etc/hpcod/clusters/ for the pinned certificates,
-clusters and -cluster-domain on the portal, and the two scripts renamed to
match. A half-finished rename would have left the most-read file still
calling a cluster a tenant, which is the confusion this commit exists to
remove.

Verified on a cold provision: terraform 37s, self-configuration 43s, nodes
61s, 141s total; nodes named c002-node[1-2], ClusterName c002, bridge
vmbrC002, and the portal answering 401 at c002.hpc.example.org.

Not addressed here, and the more binding limit at the scale this is meant
for: a cluster cannot span Proxmox nodes. Its bridge is bridge-ports none,
a local L2 island, so two hypervisors would give two segments and PXE would
not cross. Proxmox SDN is the way out and is already enabled on hv01;
docs/20 anticipated it with vlan_tag in the allocation record.
2026-09-20 20:26:49 +02:00

62 lines
2.2 KiB
Python
Executable File

#!/usr/bin/env python3
"""Turn the cluster module's ansible_inventory output into an Ansible inventory.
Terraform knows the addresses and MACs it assigned; Ansible needs them in its
own shape. Deriving the inventory rather than hand-writing it is what keeps the
two from drifting - Warewulf matches nodes by MAC, so a stale MAC here means a
node PXE boots and is never recognised, with nothing in the logs to say why.
Usage: B30-cluster-inventory.py <tofu-output-json> <cluster-id> <out.yml> [ssh-key]
"""
import json
import sys
def main() -> int:
src, cluster_id, dest = sys.argv[1], int(sys.argv[2]), sys.argv[3]
# Named explicitly rather than left to the provisioner's default identity:
# the service account has no real home directory, so whether SSH finds a
# key at all depends on $HOME being set in whatever invoked Ansible.
key = sys.argv[4] if len(sys.argv) > 4 else "/var/lib/hpcod/.ssh/id_ed25519"
with open(src) as fh:
inv = json.load(fh)
prefix = inv["cluster_cidr"].rsplit(".", 1)[0]
name = f"c{cluster_id:03d}"
lines = [
"---",
"# Generated by B30-cluster-inventory.py - do not edit by hand.",
"all:",
" children:",
" head:",
" hosts:",
f" {name}-head:",
f" ansible_host: {inv['head_mgmt_ip']}",
" ansible_user: ilker",
f" ansible_ssh_private_key_file: {key}",
" ansible_become: true",
" vars:",
f" cluster_id: {cluster_id}",
f" tenant_id: {inv.get('tenant_id', cluster_id)}",
f' head_mgmt_ip: "{inv["head_mgmt_ip"]}"',
f' cluster_cidr: "{inv["cluster_cidr"]}"',
f' head_cluster_ip: "{inv["head_ip"]}"',
f' dhcp_range_start: "{prefix}.100"',
f' dhcp_range_end: "{prefix}.199"',
" compute_nodes:",
]
for node in inv["nodes"]:
lines.append(
f' - {{ name: {node["name"]}, mac: "{node["mac"]}", ip: "{node["ip"]}" }}'
)
with open(dest, "w") as fh:
fh.write("\n".join(lines) + "\n")
print(f"wrote {dest} ({len(inv['nodes'])} compute nodes)")
return 0
if __name__ == "__main__":
sys.exit(main())