ci / derle (push) Successful in 30s
Discovery already saw every node of a Proxmox cluster and placement could put a VM on any of them, but the template, the snippets and a simple SDN zone only worked on PVE_NODE, so a cluster could land where it could not be built. - Setup asks which nodes clusters may use (PVE_NODES; a multi-select in the TUI when Proxmox has more than one). Discovery offers only those. - Heads are placed on PVE_NODE only (head_node), where the template is, so no shared storage is needed; diskless compute nodes spread. - PVE_SDN_ZONE_TYPE (auto|simple|vxlan): more than one node makes the zone vxlan, every node a peer, MTU 1450. A zone of the wrong type is remade only while no cluster has a vnet in it. Guest fabric NICs take the bridge MTU through virtio (mtu = 1). - setup keeps nodes and head_node of its own site in sites.yaml, leaving the rest of the file as the operator wrote it. - verify checks the nodes are online and the zone matches. Run on the simulated Proxmox with the zone forced to vxlan: setup remade the zone, a two-node selftest passed, both compute NICs came up at 1450, and a DF ping of 1422 bytes passed between them while 1423 did not. Two real Proxmox nodes have not been tested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
110 lines
5.1 KiB
Bash
110 lines
5.1 KiB
Bash
# Everything that makes an hpcod installation yours rather than someone
|
|
# else's. Copy to site.env next to this file, fill it in, and run
|
|
# `make deploy` from hpcod/ - or skip this file and run `sudo hpcod setup`
|
|
# on the host, which asks for the same values. See DEPLOY.md.
|
|
#
|
|
# Read by `hpcod setup` and by make, so: KEY=value, one per
|
|
# line, double quotes only where the value has a space, no $ expansions.
|
|
#
|
|
# Nothing in here is secret. Tokens and passwords are generated on the
|
|
# control plane host by `hpcod setup` and never pass through this file.
|
|
|
|
# --- The control plane host -------------------------------------------------
|
|
# An SSH destination your workstation can reach with passwordless sudo.
|
|
# `make deploy` builds here and installs there.
|
|
DEPLOY_HOST=hpcod-cp
|
|
|
|
# --- Names ------------------------------------------------------------------
|
|
# The portal answers at PORTAL_DOMAIN. Each cluster gets its own portal at
|
|
# c<NNN>.CLUSTER_DOMAIN, so DNS needs a wildcard record for it and the
|
|
# reverse proxy a wildcard certificate. Usually the same name.
|
|
PORTAL_DOMAIN=hpc.example.org
|
|
CLUSTER_DOMAIN=hpc.example.org
|
|
|
|
# --- How users reach the portal ---------------------------------------------
|
|
# proxy: behind your own reverse proxy and certificate (deploy/Caddyfile.example).
|
|
# direct: hpcod serves HTTPS itself on 443, from a certificate authority of
|
|
# its own, constrained to the domains above. For a closed network with
|
|
# no public domain; users import http://PORTAL_DOMAIN/ca.crt once.
|
|
SERVE_MODE=proxy
|
|
# true runs a DNS server answering for PORTAL_DOMAIN and *.CLUSTER_DOMAIN,
|
|
# forwarding everything else to NAMESERVERS. Point users' resolvers at
|
|
# PUBLIC_IP, or forward the zone to it from your own DNS.
|
|
SERVE_DNS=false
|
|
# Where HTTPS and DNS listen, and what the names resolve to. Required for
|
|
# direct serving or DNS.
|
|
PUBLIC_IP=""
|
|
|
|
# --- Proxmox ----------------------------------------------------------------
|
|
# real: a Proxmox VE you have, described below.
|
|
# sim: to try hpcod without one - setup runs Proxmox VE in QEMU on the
|
|
# control plane host itself, on a private bridge with NAT, and fills
|
|
# in every Proxmox and management network value below on its own. The
|
|
# host needs /dev/kvm and nested virtualization.
|
|
PVE_MODE=real
|
|
SIM_CORES=8
|
|
SIM_MEMORY_GB=24
|
|
SIM_DISK_GB=150
|
|
|
|
# The id is what the portal calls the site, and names the environment
|
|
# variable holding its discovery token: hv01 -> HV01_TOKEN in /etc/hpcod/env.
|
|
PVE_SITE_ID=hv01
|
|
PVE_SITE_NAME="Lab cluster"
|
|
PVE_ENDPOINT=https://192.0.2.10:8006
|
|
# true while the API presents Proxmox's own self-signed certificate.
|
|
PVE_INSECURE_TLS=true
|
|
# Where the control plane reaches the hypervisor over SSH as root. The
|
|
# Terraform provider needs it to upload cloud-init snippets, and setup
|
|
# drives qm and pvesh through it.
|
|
PVE_HOST=192.0.2.10
|
|
# The Proxmox node the head template, the snippets and every head live on.
|
|
PVE_NODE=hv01
|
|
# In a Proxmox cluster: the nodes compute nodes may also use, comma
|
|
# separated, PVE_NODE among them. Empty means PVE_NODE alone. Nodes left out
|
|
# are never touched, so a cluster shared with other work stays shared.
|
|
PVE_NODES=""
|
|
# VM disks.
|
|
PVE_DATASTORE=local-zfs
|
|
# Cloud-init vendor data. Setup enables the "snippets" content type on it.
|
|
PVE_SNIPPET_DATASTORE=local
|
|
# The cloud image the head template is built from. Setup downloads it if it
|
|
# is not there.
|
|
PVE_IMAGE=local:import/Rocky-9-GenericCloud-Base.latest.x86_64.qcow2
|
|
# One SDN zone shared by every cluster; each cluster gets a vnet in it.
|
|
PVE_SDN_ZONE=hpcod
|
|
# auto: simple on one node, vxlan when PVE_NODES has more than one. A simple
|
|
# zone is one bridge per node and cannot carry a cluster across two.
|
|
PVE_SDN_ZONE_TYPE=auto
|
|
|
|
# --- Management network -----------------------------------------------------
|
|
# The network the control plane and every head node share. Only /24 is
|
|
# supported: the head node prefix is fixed at 24.
|
|
MGMT_BRIDGE=vmbr0
|
|
MGMT_GATEWAY=192.0.2.1
|
|
# A run of free addresses handed to head nodes, one per live cluster. Keep
|
|
# it out of your DHCP range. Its length bounds how many clusters can exist.
|
|
MGMT_FIRST=192.0.2.160
|
|
MGMT_COUNT=40
|
|
# A free address for the throwaway VM the head template is built in.
|
|
BUILD_IP=192.0.2.61
|
|
NAMESERVERS="1.1.1.1 8.8.8.8"
|
|
|
|
# --- Head nodes -------------------------------------------------------------
|
|
# The VMID the head template is built as, and Terraform clones from.
|
|
HEAD_TEMPLATE_VMID=9100
|
|
# The login account cloud-init creates on every head node. The control
|
|
# plane's key is installed for it, and the portal uses it to archive /home
|
|
# before a cluster is released.
|
|
CI_USER=hpcod
|
|
# More public keys to install for CI_USER on every head node, separated by
|
|
# ";". The control plane's own key is always included.
|
|
EXTRA_SSH_KEYS=""
|
|
|
|
# --- Images -----------------------------------------------------------------
|
|
# Where head nodes pull the compute node image and the desktop image from.
|
|
# The defaults are public and pull anonymously; build your own with
|
|
# image/build-push.sh and image/desktop/build-push.sh to change them.
|
|
REGISTRY_HOST=gitea.mynodes.xyz
|
|
NODE_IMAGE_URI=docker://gitea.mynodes.xyz/hpcod/node-image:rocky9-ohpc3
|
|
DESKTOP_IMAGE=docker://gitea.mynodes.xyz/hpcapps/desktop:xfce
|