DEPLOY.md now follows the installer as it is: the full-screen TUI, its order (where clusters are built, Proxmox, the site, review), the menu on a host already installed, the steps in the order they run, and screenshots from a real install on a simulated Proxmox. What you need before starting says that a simulated Proxmox, or hpcod's own DNS and CA, can stand in for a Proxmox, public DNS and a certificate. The READMEs no longer describe the portal as read-only, or as something run by hand on the Caddy host; the Ansible README says the roles reach the head node inside the template and run there on first boot, and where the site's values come from. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ansible — tenant configuration
Phase 4: turns the VMs Terraform created into a working HPC cluster. These
roles are ports of scripts/, which were verified by hand first — the
comments explaining why came with them deliberately, because each one
records a failure that cost real time to find.
Where this runs
On the head node, by itself, on first boot. hpcod setup writes these
roles to /var/lib/hpcod/ansible from inside the binary. The head node
template is built from them, carrying a copy (selfconfig). Each cluster's
head then runs site.yml locally when it first boots, from an inventory
Terraform writes into its cloud-init. Nobody runs these by hand in the
normal course of things.
The site's values are not in this tree. They are the portal domain,
the node and desktop images, and the registry. hpcod setup generates
them into group_vars/all/site.yml, which is not in the repository. Both
playbooks refuse to run without it. A change to them reaches new clusters
only once the template is rebuilt (hpcod setup -rebuild-template).
To run a play by hand, on the control plane host:
cd /var/lib/hpcod/ansible && sudo -u hpcod ANSIBLE_CONFIG=$PWD/ansible.cfg \
ansible-playbook -i inventory/tenant.yml site.yml
# once the compute nodes are powered on and have had time to PXE boot
sudo -u hpcod ANSIBLE_CONFIG=$PWD/ansible.cfg ansible-playbook -i inventory/tenant.yml site.yml --tags cluster
Ansible's scratch directory is set in ansible.cfg because the service
account has no home; without it Ansible dies at startup trying to create
~/.ansible/tmp.
Role order is not arbitrary
slurm runs before warewulf_nodes because the hpc overlay it creates
has to exist before the final overlay build. Reverse them and the nodes boot
without slurm.conf or the MUNGE key, and come up unable to join.
What the comments are protecting
| Role | Trap |
|---|---|
warewulf |
cloud-init's forced-command wrapper in root's authorized_keys gets copied to every node and locks root out of all of them |
warewulf_nodes |
--netdev must be the node's real interface name; "default" matches nothing and the node never leaves its DHCP lease |
warewulf_nodes |
profile set --nettagadd returns 0, prints nothing, and changes nothing — nodes then boot with an empty resolv.conf |
node_image |
slurmd's spool directory is runtime state, so it belongs in the image, not an overlay |
slurm |
overlay import -p creates parents 0500 root:root, which locks /etc and stops munged |
slurm |
OpenHPC ships no lmod-defaults for this compiler pairing, so the default modulefile is written by hand or every MPI binary fails on libucp.so.0 |
slurm |
RealMemory is set below the VM's actual memory because the rootfs lives in RAM |
Not covered yet
Identity. A10–A60 stand up FreeIPA and enrol the stateless nodes with
per-node keytabs; porting that is the next piece. Until then a tenant gets
local accounts only.