The Redfish and Proxmox clients skipped TLS verification (BMCs ship
self-signed certificates), so a host posing as a BMC on the network got
the Basic-auth password. Before any credentials go to an HTTPS BMC the
region now reads its certificate with a handshake-only operation
(tls-fingerprint, run on the rack controller) and pins the SHA-256 on
the BMC record (extra tls_fingerprint); every later connection must show
the same certificate or nothing is sent. Changing the address, clearing
tls_fingerprint, or a BMC firmware update run by Rollcall (new firmware
often means a new certificate) pins anew. verify_tls=true still uses the
system CAs instead. The lab emulator keeps its certificate like a BMC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found in the lab run: pve-redfish runs with ProtectSystem=strict, so it
could not write its firmware state and dropped the reset request that
applies a staged BIOS. Tested end to end on the lab: iDRAC profile
multipart BMC update, iLO profile pull of BMC and BIOS through rack
controller pxe-rack2 with -reboot, model check and -force, resume of a
running update across a region restart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A firmware repository on the region (firmware_dir, served under
/firmware/<id>/<file>, cached by rack controllers for BMCs to pull),
each machine's firmware inventory (FirmwareInventory, classified into
bios/bmc/nic/raid/cpld/psu/drive), and update jobs (kind "firmware")
that the region drives with short rack commands: fw-start hands the
package over, fw-poll watches the task (tolerating a BMC that restarts),
fw-finish releases what was claimed, fw-inventory verifies the version.
Running updates resume after a region restart; they cannot be cancelled
once the BMC has the package.
Standard methods (multipart, SimpleUpdate pull, HttpPushUri) with known
Content-Length, plus vendor plans from their documentation or their own
tools: Dell MultipartUpload (OnReset), HPE SimpleUpdate with .fwpkg
extracted at import, Lenovo /mfwupdate with HttpPushUriTargetsBusy,
Supermicro /upload with targets and preserve options, GIGABYTE/AMI/
ASRock Rack /upload with OemParameters ImageType, OpenBMC
update-multipart, xFusion upload to /tmp/web then SimpleUpdate, KAYTUS
upload then OEM SimpleUpdate with fixed tasks, Fujitsu iRMC OEM actions;
Cisco CIMC is refused clearly (SCP only). fw_params and fw_method on
the BMC record override a plan.
CLI rollcall firmware list|import|rm|show|update, API, a Firmware tab
and a repository card in the UI. The lab Redfish emulator gains an
UpdateService (BMC applies at once, BIOS at the next reset).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CA (llmgw-style)
- region creates its CA (ECDSA P-256) and serves API/UI/images over HTTPS on tls_listen with a
certificate for every name/address it has (reissued on change or a month before expiry);
admin_http policy (allow|loopback|deny) for the admin API over plain HTTP; /ca.pem
- rack controllers enroll with one-time join codes (CA fingerprint + secret + expiry + address):
rollcall controller join-code, rollcall rack join; mutual TLS from then on, identity from the
certificate (CN rack:NAME, serial on record), removal revokes, renewal a month before expiry;
the shared rack_secret stays as a legacy fallback
- clients: CA from the server config, ROLLCALL_CA, or trusted on first connect in the console after
showing the fingerprint (kept in client.json); deployed machines trust it via cloud-init ca_certs
(allowed on RHEL-family images), overlays get .CA; rollcall ca show|issue
Power drivers
- ipmi (cipher/privilege/interface), redfish (system discovery, ResetType from AllowableValues or
ActionInfo, boot override with If-Match), idrac, ilo, supermicro, xcc, openbmc, proxmox (API
token), manual; executed on the rack controller; Redfish auto-detection after commissioning;
operator settings survive recommissioning; rollcall bmc, BMC editing in web UI and console
- lab: pve_redfish.py Redfish emulator per Proxmox VM (iDRAC, iLO, generic profiles)
IPv6
- IPv6 subnets and IPAM (netip), dual-stack default layout (all subnets of the boot VLAN),
per-family default routes, accept-ra off where addresses are planned, dnsmasq RA/DHCPv6 with
router lifetime 0 unless the rack is the gateway, AAAA host records from IPAM, IPv6 listen
addresses and URLs, rack interfaces and region certificate with IPv6, agent reports IPv6
Tests for CA/join/pinning/CSR signing, Redfish (iDRAC, iLO fallbacks, probe), Proxmox, manual,
IPv6 candidates, dual-stack plans and dnsmasq IPv6.
Verified in the lab: rack2 enrolled and running on mutual TLS only, revocation and single-use
codes; iLO auto-detected through rack2, iDRAC and Proxmox drivers, wrong password not saved;
Debian 13 (bond+VLAN), Rocky 9 and AlmaLinux 9 dual-stack with Redfish-driven deployments; CA
trusted on Debian and AlmaLinux; admin CLI over HTTPS on IPv4 and IPv6.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>