Files
werewulf/hpcod/deploy/site.env.example
T
ilkerandClaude Opus 5.5 090719fc74
ci / derle (push) Successful in 30s
hpcod: clusters across the nodes of a Proxmox cluster
Discovery already saw every node of a Proxmox cluster and placement could
put a VM on any of them, but the template, the snippets and a simple SDN
zone only worked on PVE_NODE, so a cluster could land where it could not
be built.

- Setup asks which nodes clusters may use (PVE_NODES; a multi-select in
  the TUI when Proxmox has more than one). Discovery offers only those.
- Heads are placed on PVE_NODE only (head_node), where the template is, so
  no shared storage is needed; diskless compute nodes spread.
- PVE_SDN_ZONE_TYPE (auto|simple|vxlan): more than one node makes the zone
  vxlan, every node a peer, MTU 1450. A zone of the wrong type is remade
  only while no cluster has a vnet in it. Guest fabric NICs take the
  bridge MTU through virtio (mtu = 1).
- setup keeps nodes and head_node of its own site in sites.yaml, leaving
  the rest of the file as the operator wrote it.
- verify checks the nodes are online and the zone matches.

Run on the simulated Proxmox with the zone forced to vxlan: setup remade
the zone, a two-node selftest passed, both compute NICs came up at 1450,
and a DF ping of 1422 bytes passed between them while 1423 did not. Two
real Proxmox nodes have not been tested.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 19:51:35 +01:00

110 lines
5.1 KiB
Bash

# Everything that makes an hpcod installation yours rather than someone
# else's. Copy to site.env next to this file, fill it in, and run
# `make deploy` from hpcod/ - or skip this file and run `sudo hpcod setup`
# on the host, which asks for the same values. See DEPLOY.md.
#
# Read by `hpcod setup` and by make, so: KEY=value, one per
# line, double quotes only where the value has a space, no $ expansions.
#
# Nothing in here is secret. Tokens and passwords are generated on the
# control plane host by `hpcod setup` and never pass through this file.
# --- The control plane host -------------------------------------------------
# An SSH destination your workstation can reach with passwordless sudo.
# `make deploy` builds here and installs there.
DEPLOY_HOST=hpcod-cp
# --- Names ------------------------------------------------------------------
# The portal answers at PORTAL_DOMAIN. Each cluster gets its own portal at
# c<NNN>.CLUSTER_DOMAIN, so DNS needs a wildcard record for it and the
# reverse proxy a wildcard certificate. Usually the same name.
PORTAL_DOMAIN=hpc.example.org
CLUSTER_DOMAIN=hpc.example.org
# --- How users reach the portal ---------------------------------------------
# proxy: behind your own reverse proxy and certificate (deploy/Caddyfile.example).
# direct: hpcod serves HTTPS itself on 443, from a certificate authority of
# its own, constrained to the domains above. For a closed network with
# no public domain; users import http://PORTAL_DOMAIN/ca.crt once.
SERVE_MODE=proxy
# true runs a DNS server answering for PORTAL_DOMAIN and *.CLUSTER_DOMAIN,
# forwarding everything else to NAMESERVERS. Point users' resolvers at
# PUBLIC_IP, or forward the zone to it from your own DNS.
SERVE_DNS=false
# Where HTTPS and DNS listen, and what the names resolve to. Required for
# direct serving or DNS.
PUBLIC_IP=""
# --- Proxmox ----------------------------------------------------------------
# real: a Proxmox VE you have, described below.
# sim: to try hpcod without one - setup runs Proxmox VE in QEMU on the
# control plane host itself, on a private bridge with NAT, and fills
# in every Proxmox and management network value below on its own. The
# host needs /dev/kvm and nested virtualization.
PVE_MODE=real
SIM_CORES=8
SIM_MEMORY_GB=24
SIM_DISK_GB=150
# The id is what the portal calls the site, and names the environment
# variable holding its discovery token: hv01 -> HV01_TOKEN in /etc/hpcod/env.
PVE_SITE_ID=hv01
PVE_SITE_NAME="Lab cluster"
PVE_ENDPOINT=https://192.0.2.10:8006
# true while the API presents Proxmox's own self-signed certificate.
PVE_INSECURE_TLS=true
# Where the control plane reaches the hypervisor over SSH as root. The
# Terraform provider needs it to upload cloud-init snippets, and setup
# drives qm and pvesh through it.
PVE_HOST=192.0.2.10
# The Proxmox node the head template, the snippets and every head live on.
PVE_NODE=hv01
# In a Proxmox cluster: the nodes compute nodes may also use, comma
# separated, PVE_NODE among them. Empty means PVE_NODE alone. Nodes left out
# are never touched, so a cluster shared with other work stays shared.
PVE_NODES=""
# VM disks.
PVE_DATASTORE=local-zfs
# Cloud-init vendor data. Setup enables the "snippets" content type on it.
PVE_SNIPPET_DATASTORE=local
# The cloud image the head template is built from. Setup downloads it if it
# is not there.
PVE_IMAGE=local:import/Rocky-9-GenericCloud-Base.latest.x86_64.qcow2
# One SDN zone shared by every cluster; each cluster gets a vnet in it.
PVE_SDN_ZONE=hpcod
# auto: simple on one node, vxlan when PVE_NODES has more than one. A simple
# zone is one bridge per node and cannot carry a cluster across two.
PVE_SDN_ZONE_TYPE=auto
# --- Management network -----------------------------------------------------
# The network the control plane and every head node share. Only /24 is
# supported: the head node prefix is fixed at 24.
MGMT_BRIDGE=vmbr0
MGMT_GATEWAY=192.0.2.1
# A run of free addresses handed to head nodes, one per live cluster. Keep
# it out of your DHCP range. Its length bounds how many clusters can exist.
MGMT_FIRST=192.0.2.160
MGMT_COUNT=40
# A free address for the throwaway VM the head template is built in.
BUILD_IP=192.0.2.61
NAMESERVERS="1.1.1.1 8.8.8.8"
# --- Head nodes -------------------------------------------------------------
# The VMID the head template is built as, and Terraform clones from.
HEAD_TEMPLATE_VMID=9100
# The login account cloud-init creates on every head node. The control
# plane's key is installed for it, and the portal uses it to archive /home
# before a cluster is released.
CI_USER=hpcod
# More public keys to install for CI_USER on every head node, separated by
# ";". The control plane's own key is always included.
EXTRA_SSH_KEYS=""
# --- Images -----------------------------------------------------------------
# Where head nodes pull the compute node image and the desktop image from.
# The defaults are public and pull anonymously; build your own with
# image/build-push.sh and image/desktop/build-push.sh to change them.
REGISTRY_HOST=gitea.mynodes.xyz
NODE_IMAGE_URI=docker://gitea.mynodes.xyz/hpcod/node-image:rocky9-ohpc3
DESKTOP_IMAGE=docker://gitea.mynodes.xyz/hpcapps/desktop:xfce