Files
werewulf/scripts/A10-ipa-server.sh
ilker 29bd39c135 Warewulf HPC test cluster: provisioning, identity, docs and overlay UI
A complete, verified HPC lab on Proxmox (hv01) with its own isolated subnet,
central identity and a web console for overlay management.

Infrastructure
- vmbr6: L2-only cluster fabric, deliberately without an address on the
  hypervisor so Warewulf owns DHCP/TFTP/PXE for the whole subnet.
- head01 (VM 300): Warewulf 4.7.0, dnsmasq-free DHCP/TFTP/iPXE chain,
  Slurm controller, MUNGE, NAT gateway, /opt exported read-only.
- wwnode1-4 (VM 301-304): diskless, PXE-booted, rootfs unpacked into RAM.
- idm1 (VM 305): FreeIPA (LDAP/Kerberos/Dogtag CA/DNS) plus the NFS home
  server. Dual-homed on purpose: idm1.hpc.local resolves to the cluster
  fabric so Kerberos and NFS never traverse head01's masquerade.

Stack
Rocky Linux 9.8, OpenHPC 3, Slurm 25.11.4, OpenMPI 5.0.7, FreeIPA 4.13.1.

Stateless domain join
Nodes are rebuilt in RAM every boot, so ipa-client-install cannot be used.
Enrolment happens once and is distributed through Warewulf overlays: shared
config as templates rendered per node via {{.Id}}, and a separate keytab per
node so wwnode1 cannot impersonate wwnode2.

Overlay web UI (webui/)
A Go service on the head node serving a browser interface over the Warewulf
REST API. It exists as a server rather than a static page because the API is
root-equivalent and bound to loopback; the service holds the credentials and
proxies same-origin, so the browser never sees one. Published behind Caddy
with bcrypt logins, server-side sessions and per-user/per-IP lockout.

Documentation
README plus docs/01-10 covering architecture, installation, Warewulf,
identity, storage, Slurm/MPI, operations, troubleshooting and reference,
with a 22-page PDF edition. Section 08 records twelve concrete failures hit
during the build, including a wwctl flag that fails silently.

No credentials are stored here. Every secret lives on the servers; the
documentation refers only to its path.
2026-08-29 09:07:51 +02:00

78 lines
2.5 KiB
Bash
Executable File

#!/bin/bash
# FreeIPA server (hpc.local) - Rocky 9, dual-homed (vmbr0 + vmbr6)
# Run as root on idm1.
set -euo pipefail
REALM=HPC.LOCAL
DOMAIN=hpc.local
FQDN=idm1.hpc.local
# IMPORTANT: the service IP is on the cluster fabric (vmbr6). Both head01
# (10.10.10.1) and the compute nodes reach it DIRECTLY, without NAT.
# Kerberos and NFS do not work properly behind head01's masquerade.
SVC_IP=10.10.10.5
MGMT_IP=10.0.0.31
FORWARDER=1.1.1.1
echo "==> hostname / hosts"
hostnamectl set-hostname ${FQDN}
cat > /etc/hosts <<EOF
127.0.0.1 localhost localhost.localdomain
${SVC_IP} ${FQDN} idm1
${MGMT_IP} idm1-mgmt.${DOMAIN} idm1-mgmt
::1 localhost localhost.localdomain
EOF
echo "==> packages"
dnf install -y -q dnf-plugins-core
dnf module enable -y -q idm:DL1 2>/dev/null || true
dnf install -y -q @idm:DL1/dns 2>/dev/null || dnf install -y -q ipa-server ipa-server-dns
dnf install -y -q nfs-utils firewalld chrony vim jq bind-utils
echo "==> firewalld"
systemctl enable --now firewalld
# eth1 = cluster fabric -> trusted (nodes use IPA + NFS over this)
firewall-cmd --permanent --zone=trusted --add-interface=eth1 2>/dev/null || true
# eth0 = management -> IPA services for administrator access
for s in freeipa-4 dns ntp http https kerberos kpasswd ldap ldaps nfs mountd rpc-bind; do
firewall-cmd --permanent --zone=public --add-service=$s 2>/dev/null || true
done
firewall-cmd --permanent --zone=trusted --add-service=nfs 2>/dev/null || true
firewall-cmd --reload
echo "==> generating passwords"
if [ ! -f /root/.ipa-credentials ]; then
# NOTE: "tr ... | head -c" kills the script under pipefail via SIGPIPE.
DM_PW=$(head -c 40 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' | cut -c1-20)
ADMIN_PW=$(head -c 40 /dev/urandom | base64 | tr -dc 'A-Za-z0-9' | cut -c1-20)
cat > /root/.ipa-credentials <<EOF
# FreeIPA lab credentials - $(date -u +%F)
IPA_REALM=${REALM}
IPA_DOMAIN=${DOMAIN}
IPA_SERVER=${FQDN}
DIRECTORY_MANAGER_PASSWORD=${DM_PW}
IPA_ADMIN_PASSWORD=${ADMIN_PW}
EOF
chmod 600 /root/.ipa-credentials
fi
. /root/.ipa-credentials
echo "==> ipa-server-install (with integrated DNS)"
if ! ipactl status >/dev/null 2>&1; then
ipa-server-install --unattended \
--realm="${REALM}" \
--domain="${DOMAIN}" \
--hostname="${FQDN}" \
--ip-address="${SVC_IP}" \
--ds-password="${DIRECTORY_MANAGER_PASSWORD}" \
--admin-password="${IPA_ADMIN_PASSWORD}" \
--setup-dns \
--forwarder="${FORWARDER}" \
--auto-reverse \
--no-host-dns
fi
echo "==> status"
ipactl status
echo "IPA SERVER OK (credentials: /root/.ipa-credentials)"