Files
ilkerandClaude Opus 5.5 3dfcbf1ffc
ci / derle (push) Successful in 25s
docs: installing with hpcod setup
DEPLOY.md now follows the installer as it is: the full-screen TUI, its
order (where clusters are built, Proxmox, the site, review), the menu on a
host already installed, the steps in the order they run, and screenshots
from a real install on a simulated Proxmox. What you need before starting
says that a simulated Proxmox, or hpcod's own DNS and CA, can stand in for
a Proxmox, public DNS and a certificate.

The READMEs no longer describe the portal as read-only, or as something
run by hand on the Caddy host; the Ansible README says the roles reach the
head node inside the template and run there on first boot, and where the
site's values come from.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 18:43:42 +01:00
..
2026-11-16 18:43:42 +01:00

Ansible — tenant configuration

Phase 4: turns the VMs Terraform created into a working HPC cluster. These roles are ports of scripts/, which were verified by hand first — the comments explaining why came with them deliberately, because each one records a failure that cost real time to find.

Where this runs

On the head node, by itself, on first boot. hpcod setup writes these roles to /var/lib/hpcod/ansible from inside the binary. The head node template is built from them, carrying a copy (selfconfig). Each cluster's head then runs site.yml locally when it first boots, from an inventory Terraform writes into its cloud-init. Nobody runs these by hand in the normal course of things.

The site's values are not in this tree. They are the portal domain, the node and desktop images, and the registry. hpcod setup generates them into group_vars/all/site.yml, which is not in the repository. Both playbooks refuse to run without it. A change to them reaches new clusters only once the template is rebuilt (hpcod setup -rebuild-template).

To run a play by hand, on the control plane host:

cd /var/lib/hpcod/ansible && sudo -u hpcod ANSIBLE_CONFIG=$PWD/ansible.cfg \
    ansible-playbook -i inventory/tenant.yml site.yml

# once the compute nodes are powered on and have had time to PXE boot
sudo -u hpcod ANSIBLE_CONFIG=$PWD/ansible.cfg ansible-playbook -i inventory/tenant.yml site.yml --tags cluster

Ansible's scratch directory is set in ansible.cfg because the service account has no home; without it Ansible dies at startup trying to create ~/.ansible/tmp.

Role order is not arbitrary

slurm runs before warewulf_nodes because the hpc overlay it creates has to exist before the final overlay build. Reverse them and the nodes boot without slurm.conf or the MUNGE key, and come up unable to join.

What the comments are protecting

Role Trap
warewulf cloud-init's forced-command wrapper in root's authorized_keys gets copied to every node and locks root out of all of them
warewulf_nodes --netdev must be the node's real interface name; "default" matches nothing and the node never leaves its DHCP lease
warewulf_nodes profile set --nettagadd returns 0, prints nothing, and changes nothing — nodes then boot with an empty resolv.conf
node_image slurmd's spool directory is runtime state, so it belongs in the image, not an overlay
slurm overlay import -p creates parents 0500 root:root, which locks /etc and stops munged
slurm OpenHPC ships no lmod-defaults for this compiler pairing, so the default modulefile is written by hand or every MPI binary fails on libucp.so.0
slurm RealMemory is set below the VM's actual memory because the rootfs lives in RAM

Not covered yet

Identity. A10–A60 stand up FreeIPA and enrol the stateless nodes with per-node keytabs; porting that is the next piece. Until then a tenant gets local accounts only.