One tenant may run several HPC clusters at the same time. The code did not allow for that: a single tenant_id drove the bridge, the subnet, the MAC prefix, the VM names, the vendor-data filename and the public hostname. A tenant asking for a second cluster while the first was running would have been handed the same bridge, the same subnet and the same MAC prefix - and Warewulf matches nodes by MAC, so the second cluster would have broken the first rather than failing cleanly. Everything that must not collide is now keyed on cluster_id, and tenant_id survives as what it always meant: who owns this. Bridges are vmbrC###, hosts c###-head and c###-node#, the portal is c###.hpc.example.org, and the Slurm ClusterName is the cluster's - which matters for the accounting work next, because records attributed to a shared name cannot be separated afterwards. The bound of 254 therefore applies to clusters running at one time, not to tenants. It comes from the addressing and nothing else: cluster_id is the third octet of 10.64.x.0/24 and one byte of the MAC. That is written down in 20.5a rather than left to be rediscovered. The registry keeps its shape but not its name: /etc/hpcod/clusters.yaml with a clusters: key, /etc/hpcod/clusters/ for the pinned certificates, -clusters and -cluster-domain on the portal, and the two scripts renamed to match. A half-finished rename would have left the most-read file still calling a cluster a tenant, which is the confusion this commit exists to remove. Verified on a cold provision: terraform 37s, self-configuration 43s, nodes 61s, 141s total; nodes named c002-node[1-2], ClusterName c002, bridge vmbrC002, and the portal answering 401 at c002.hpc.example.org. Not addressed here, and the more binding limit at the scale this is meant for: a cluster cannot span Proxmox nodes. Its bridge is bridge-ports none, a local L2 island, so two hypervisors would give two segments and PXE would not cross. Proxmox SDN is the way out and is already enabled on hv01; docs/20 anticipated it with vlan_tag in the allocation record.
62 lines
2.2 KiB
Python
Executable File
62 lines
2.2 KiB
Python
Executable File
#!/usr/bin/env python3
|
|
"""Turn the cluster module's ansible_inventory output into an Ansible inventory.
|
|
|
|
Terraform knows the addresses and MACs it assigned; Ansible needs them in its
|
|
own shape. Deriving the inventory rather than hand-writing it is what keeps the
|
|
two from drifting - Warewulf matches nodes by MAC, so a stale MAC here means a
|
|
node PXE boots and is never recognised, with nothing in the logs to say why.
|
|
|
|
Usage: B30-cluster-inventory.py <tofu-output-json> <cluster-id> <out.yml> [ssh-key]
|
|
"""
|
|
import json
|
|
import sys
|
|
|
|
|
|
def main() -> int:
|
|
src, cluster_id, dest = sys.argv[1], int(sys.argv[2]), sys.argv[3]
|
|
# Named explicitly rather than left to the provisioner's default identity:
|
|
# the service account has no real home directory, so whether SSH finds a
|
|
# key at all depends on $HOME being set in whatever invoked Ansible.
|
|
key = sys.argv[4] if len(sys.argv) > 4 else "/var/lib/hpcod/.ssh/id_ed25519"
|
|
with open(src) as fh:
|
|
inv = json.load(fh)
|
|
|
|
prefix = inv["cluster_cidr"].rsplit(".", 1)[0]
|
|
name = f"c{cluster_id:03d}"
|
|
|
|
lines = [
|
|
"---",
|
|
"# Generated by B30-cluster-inventory.py - do not edit by hand.",
|
|
"all:",
|
|
" children:",
|
|
" head:",
|
|
" hosts:",
|
|
f" {name}-head:",
|
|
f" ansible_host: {inv['head_mgmt_ip']}",
|
|
" ansible_user: ilker",
|
|
f" ansible_ssh_private_key_file: {key}",
|
|
" ansible_become: true",
|
|
" vars:",
|
|
f" cluster_id: {cluster_id}",
|
|
f" tenant_id: {inv.get('tenant_id', cluster_id)}",
|
|
f' head_mgmt_ip: "{inv["head_mgmt_ip"]}"',
|
|
f' cluster_cidr: "{inv["cluster_cidr"]}"',
|
|
f' head_cluster_ip: "{inv["head_ip"]}"',
|
|
f' dhcp_range_start: "{prefix}.100"',
|
|
f' dhcp_range_end: "{prefix}.199"',
|
|
" compute_nodes:",
|
|
]
|
|
for node in inv["nodes"]:
|
|
lines.append(
|
|
f' - {{ name: {node["name"]}, mac: "{node["mac"]}", ip: "{node["ip"]}" }}'
|
|
)
|
|
|
|
with open(dest, "w") as fh:
|
|
fh.write("\n".join(lines) + "\n")
|
|
print(f"wrote {dest} ({len(inv['nodes'])} compute nodes)")
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
sys.exit(main())
|