An installation used to be this one: the deploy scripts and the Ansible defaults carried one site's domain, hypervisor and login, the Proxmox side and the database were set up by hand, and deploying meant copying scripts and a tree of roles beside the binary and keeping them in step with it. The binary now carries the roles, the Terraform module, the schema and the catalogue (embed.go), and installs itself: - hpcod setup: a full-screen installer for administrators. It asks where clusters are built, signs in to Proxmox before asking anything else (root's password once, or tokens), offers nodes and datastores from what Proxmox has, checks every field as it is typed, runs every step with its output live, checks the result and offers a test cluster. -yes runs the same steps from a site file; make deploy uses that. - hpcod verify: each fact checked by using it. - hpcod selftest: the smallest real cluster, by the portal's own path, a job run on it, then taken down. - Closed networks: hpcod serves HTTPS itself from a CA name-constrained to the site's domains, issuing and renewing its own certificate, and answers DNS for the portal and every cluster name. - A simulated Proxmox: Proxmox VE in QEMU on the control plane host, on a private bridge with NAT, for trying it without one. Every step reads back what it changed and is safe to run again. The site's values live in one file; group_vars/all/site.yml is generated from it, and the playbooks refuse to run without it. ci_user and -ssh-user no longer default to a personal account. scripts/B10, B20 and C20 are gone: their work is the setup steps. Proven end to end on a simulated Proxmox (Proxmox VE 9 in a VM on a real host): the install, the Proxmox step by the password path, the head template, verify READY, and a test cluster that ran srun on its compute node and was destroyed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
103 lines
4.7 KiB
Bash
103 lines
4.7 KiB
Bash
# Everything that makes an hpcod installation yours rather than someone
|
|
# else's. Copy to site.env next to this file, fill it in, and run
|
|
# `make deploy` from hpcod/ - or skip this file and run `sudo hpcod setup`
|
|
# on the host, which asks for the same values. See DEPLOY.md.
|
|
#
|
|
# Read by `hpcod setup` and by make, so: KEY=value, one per
|
|
# line, double quotes only where the value has a space, no $ expansions.
|
|
#
|
|
# Nothing in here is secret. Tokens and passwords are generated on the
|
|
# control plane host by `hpcod setup` and never pass through this file.
|
|
|
|
# --- The control plane host -------------------------------------------------
|
|
# An SSH destination your workstation can reach with passwordless sudo.
|
|
# `make deploy` builds here and installs there.
|
|
DEPLOY_HOST=hpcod-cp
|
|
|
|
# --- Names ------------------------------------------------------------------
|
|
# The portal answers at PORTAL_DOMAIN. Each cluster gets its own portal at
|
|
# c<NNN>.CLUSTER_DOMAIN, so DNS needs a wildcard record for it and the
|
|
# reverse proxy a wildcard certificate. Usually the same name.
|
|
PORTAL_DOMAIN=hpc.example.org
|
|
CLUSTER_DOMAIN=hpc.example.org
|
|
|
|
# --- How users reach the portal ---------------------------------------------
|
|
# proxy: behind your own reverse proxy and certificate (deploy/Caddyfile.example).
|
|
# direct: hpcod serves HTTPS itself on 443, from a certificate authority of
|
|
# its own, constrained to the domains above. For a closed network with
|
|
# no public domain; users import http://PORTAL_DOMAIN/ca.crt once.
|
|
SERVE_MODE=proxy
|
|
# true runs a DNS server answering for PORTAL_DOMAIN and *.CLUSTER_DOMAIN,
|
|
# forwarding everything else to NAMESERVERS. Point users' resolvers at
|
|
# PUBLIC_IP, or forward the zone to it from your own DNS.
|
|
SERVE_DNS=false
|
|
# Where HTTPS and DNS listen, and what the names resolve to. Required for
|
|
# direct serving or DNS.
|
|
PUBLIC_IP=""
|
|
|
|
# --- Proxmox ----------------------------------------------------------------
|
|
# real: a Proxmox VE you have, described below.
|
|
# sim: to try hpcod without one - setup runs Proxmox VE in QEMU on the
|
|
# control plane host itself, on a private bridge with NAT, and fills
|
|
# in every Proxmox and management network value below on its own. The
|
|
# host needs /dev/kvm and nested virtualization.
|
|
PVE_MODE=real
|
|
SIM_CORES=8
|
|
SIM_MEMORY_GB=24
|
|
SIM_DISK_GB=150
|
|
|
|
# The id is what the portal calls the site, and names the environment
|
|
# variable holding its discovery token: hv01 -> HV01_TOKEN in /etc/hpcod/env.
|
|
PVE_SITE_ID=hv01
|
|
PVE_SITE_NAME="Lab cluster"
|
|
PVE_ENDPOINT=https://192.0.2.10:8006
|
|
# true while the API presents Proxmox's own self-signed certificate.
|
|
PVE_INSECURE_TLS=true
|
|
# Where the control plane reaches the hypervisor over SSH as root. The
|
|
# Terraform provider needs it to upload cloud-init snippets, and setup
|
|
# drives qm and pvesh through it.
|
|
PVE_HOST=192.0.2.10
|
|
# The Proxmox node clusters are created on by default.
|
|
PVE_NODE=hv01
|
|
# VM disks.
|
|
PVE_DATASTORE=local-zfs
|
|
# Cloud-init vendor data. Setup enables the "snippets" content type on it.
|
|
PVE_SNIPPET_DATASTORE=local
|
|
# The cloud image the head template is built from. Setup downloads it if it
|
|
# is not there.
|
|
PVE_IMAGE=local:import/Rocky-9-GenericCloud-Base.latest.x86_64.qcow2
|
|
# One SDN zone shared by every cluster; each cluster gets a vnet in it.
|
|
PVE_SDN_ZONE=hpcod
|
|
|
|
# --- Management network -----------------------------------------------------
|
|
# The network the control plane and every head node share. Only /24 is
|
|
# supported: the head node prefix is fixed at 24.
|
|
MGMT_BRIDGE=vmbr0
|
|
MGMT_GATEWAY=192.0.2.1
|
|
# A run of free addresses handed to head nodes, one per live cluster. Keep
|
|
# it out of your DHCP range. Its length bounds how many clusters can exist.
|
|
MGMT_FIRST=192.0.2.160
|
|
MGMT_COUNT=40
|
|
# A free address for the throwaway VM the head template is built in.
|
|
BUILD_IP=192.0.2.61
|
|
NAMESERVERS="1.1.1.1 8.8.8.8"
|
|
|
|
# --- Head nodes -------------------------------------------------------------
|
|
# The VMID the head template is built as, and Terraform clones from.
|
|
HEAD_TEMPLATE_VMID=9100
|
|
# The login account cloud-init creates on every head node. The control
|
|
# plane's key is installed for it, and the portal uses it to archive /home
|
|
# before a cluster is released.
|
|
CI_USER=hpcod
|
|
# More public keys to install for CI_USER on every head node, separated by
|
|
# ";". The control plane's own key is always included.
|
|
EXTRA_SSH_KEYS=""
|
|
|
|
# --- Images -----------------------------------------------------------------
|
|
# Where head nodes pull the compute node image and the desktop image from.
|
|
# The defaults are public and pull anonymously; build your own with
|
|
# image/build-push.sh and image/desktop/build-push.sh to change them.
|
|
REGISTRY_HOST=gitea.mynodes.xyz
|
|
NODE_IMAGE_URI=docker://gitea.mynodes.xyz/hpcod/node-image:rocky9-ohpc3
|
|
DESKTOP_IMAGE=docker://gitea.mynodes.xyz/hpcapps/desktop:xfce
|