Found in the lab run: pve-redfish runs with ProtectSystem=strict, so it
could not write its firmware state and dropped the reset request that
applies a staged BIOS. Tested end to end on the lab: iDRAC profile
multipart BMC update, iLO profile pull of BMC and BIOS through rack
controller pxe-rack2 with -reboot, model check and -force, resume of a
running update across a region restart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A firmware repository on the region (firmware_dir, served under
/firmware/<id>/<file>, cached by rack controllers for BMCs to pull),
each machine's firmware inventory (FirmwareInventory, classified into
bios/bmc/nic/raid/cpld/psu/drive), and update jobs (kind "firmware")
that the region drives with short rack commands: fw-start hands the
package over, fw-poll watches the task (tolerating a BMC that restarts),
fw-finish releases what was claimed, fw-inventory verifies the version.
Running updates resume after a region restart; they cannot be cancelled
once the BMC has the package.
Standard methods (multipart, SimpleUpdate pull, HttpPushUri) with known
Content-Length, plus vendor plans from their documentation or their own
tools: Dell MultipartUpload (OnReset), HPE SimpleUpdate with .fwpkg
extracted at import, Lenovo /mfwupdate with HttpPushUriTargetsBusy,
Supermicro /upload with targets and preserve options, GIGABYTE/AMI/
ASRock Rack /upload with OemParameters ImageType, OpenBMC
update-multipart, xFusion upload to /tmp/web then SimpleUpdate, KAYTUS
upload then OEM SimpleUpdate with fixed tasks, Fujitsu iRMC OEM actions;
Cisco CIMC is refused clearly (SCP only). fw_params and fw_method on
the BMC record override a plan.
CLI rollcall firmware list|import|rm|show|update, API, a Firmware tab
and a repository card in the UI. The lab Redfish emulator gains an
UpdateService (BMC applies at once, BIOS at the next reset).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CA (llmgw-style)
- region creates its CA (ECDSA P-256) and serves API/UI/images over HTTPS on tls_listen with a
certificate for every name/address it has (reissued on change or a month before expiry);
admin_http policy (allow|loopback|deny) for the admin API over plain HTTP; /ca.pem
- rack controllers enroll with one-time join codes (CA fingerprint + secret + expiry + address):
rollcall controller join-code, rollcall rack join; mutual TLS from then on, identity from the
certificate (CN rack:NAME, serial on record), removal revokes, renewal a month before expiry;
the shared rack_secret stays as a legacy fallback
- clients: CA from the server config, ROLLCALL_CA, or trusted on first connect in the console after
showing the fingerprint (kept in client.json); deployed machines trust it via cloud-init ca_certs
(allowed on RHEL-family images), overlays get .CA; rollcall ca show|issue
Power drivers
- ipmi (cipher/privilege/interface), redfish (system discovery, ResetType from AllowableValues or
ActionInfo, boot override with If-Match), idrac, ilo, supermicro, xcc, openbmc, proxmox (API
token), manual; executed on the rack controller; Redfish auto-detection after commissioning;
operator settings survive recommissioning; rollcall bmc, BMC editing in web UI and console
- lab: pve_redfish.py Redfish emulator per Proxmox VM (iDRAC, iLO, generic profiles)
IPv6
- IPv6 subnets and IPAM (netip), dual-stack default layout (all subnets of the boot VLAN),
per-family default routes, accept-ra off where addresses are planned, dnsmasq RA/DHCPv6 with
router lifetime 0 unless the rack is the gateway, AAAA host records from IPAM, IPv6 listen
addresses and URLs, rack interfaces and region certificate with IPv6, agent reports IPv6
Tests for CA/join/pinning/CSR signing, Redfish (iDRAC, iLO fallbacks, probe), Proxmox, manual,
IPv6 candidates, dual-stack plans and dnsmasq IPv6.
Verified in the lab: rack2 enrolled and running on mutual TLS only, revocation and single-use
codes; iLO auto-detected through rack2, iDRAC and Proxmox drivers, wrong password not saved;
Debian 13 (bond+VLAN), Rocky 9 and AlmaLinux 9 dual-stack with Redfish-driven deployments; CA
trusted on Debian and AlmaLinux; admin CLI over HTTPS on IPv4 and IPv6.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- fabrics, VLANs, subnets, IP ranges (dynamic/reserved/static), IPAM with adoption of
addresses machines already have; rollcall fabric|vlan|subnet|range|ip
- interface layouts (physical/bond/vlan, NICs by selector: boot, name, mac, fastest:N, ib:N)
per machine or per tag; deployment allocates addresses and writes cloud-init network-config
plus udev name rules (RHEL cloud images boot with net.ifnames=0); release frees them;
rollcall layout, rollcall net show|set
- region/rack: rollcall rack long-polls its desired state (per-MAC PXE files kept in the
database, dnsmasq configuration and host files) and a BMC command queue, caches boot and OS
images (kernels/initrds prefetched for TFTP), proxies the node API and tags requests with its
name; the region runs an embedded rack controller; install-rack.sh, rollcall-rack.service
- dnsmasq configuration generated per rack from the VLANs it serves (split pool for two racks);
reload script restarts only on change and restores the last working config if dnsmasq does
not come back (drilled in the lab)
- web: Network page, per-machine network tab, layout action, tag layout; console: Network page,
network section in the machine view, layout action
- tests for candidates, plans (default, bond+VLAN, pinned, DHCP fallback), network-config,
udev rules and dnsmasq rendering
- lab: pve-vbmc address script finds the interface per BMC address
Verified in the lab: embedded rack (10.213.77.0/24) and a remote rack controller on a second
SDN segment (10.213.78.0/24): enlistment, commissioning, BMC through the rack's queue, image
cache, deployment and phone-home through the rack; Debian 13, Ubuntu 24.04 and Rocky 9 with a
static boot NIC and with an active-backup bond carrying VLAN 200.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Profiles bind a RAM-booted root image to overlay templates; assigning a
machine allocates a node name and management IP, writes the per-MAC PXE
entry with a per-node token and exports static leases for dnsmasq. Nodes
pull rendered overlays (hosts, slurm.conf from inventory, gres.conf, SSH
host key, munge key) through rollcall-overlay.service; the runtime refresh
timer is the heartbeat. Adds the compute-debian13 image definition and a
generic live-build root image script, CLI and web UI for profiles/assign.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Go server (API, embedded web UI, CLI) backed by PostgreSQL with AES-GCM
sealed BMC secrets; Debian live commissioning agent and live-build recipe;
Proxmox lab with pve-vbmc virtual BMCs; Gitea workflows for CI, releases
and deployment; project website.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
ilkerandClaude Fable 5.1