The Redfish and Proxmox clients skipped TLS verification (BMCs ship
self-signed certificates), so a host posing as a BMC on the network got
the Basic-auth password. Before any credentials go to an HTTPS BMC the
region now reads its certificate with a handshake-only operation
(tls-fingerprint, run on the rack controller) and pins the SHA-256 on
the BMC record (extra tls_fingerprint); every later connection must show
the same certificate or nothing is sent. Changing the address, clearing
tls_fingerprint, or a BMC firmware update run by Rollcall (new firmware
often means a new certificate) pins anew. verify_tls=true still uses the
system CAs instead. The lab emulator keeps its certificate like a BMC.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A firmware repository on the region (firmware_dir, served under
/firmware/<id>/<file>, cached by rack controllers for BMCs to pull),
each machine's firmware inventory (FirmwareInventory, classified into
bios/bmc/nic/raid/cpld/psu/drive), and update jobs (kind "firmware")
that the region drives with short rack commands: fw-start hands the
package over, fw-poll watches the task (tolerating a BMC that restarts),
fw-finish releases what was claimed, fw-inventory verifies the version.
Running updates resume after a region restart; they cannot be cancelled
once the BMC has the package.
Standard methods (multipart, SimpleUpdate pull, HttpPushUri) with known
Content-Length, plus vendor plans from their documentation or their own
tools: Dell MultipartUpload (OnReset), HPE SimpleUpdate with .fwpkg
extracted at import, Lenovo /mfwupdate with HttpPushUriTargetsBusy,
Supermicro /upload with targets and preserve options, GIGABYTE/AMI/
ASRock Rack /upload with OemParameters ImageType, OpenBMC
update-multipart, xFusion upload to /tmp/web then SimpleUpdate, KAYTUS
upload then OEM SimpleUpdate with fixed tasks, Fujitsu iRMC OEM actions;
Cisco CIMC is refused clearly (SCP only). fw_params and fw_method on
the BMC record override a plan.
CLI rollcall firmware list|import|rm|show|update, API, a Firmware tab
and a repository card in the UI. The lab Redfish emulator gains an
UpdateService (BMC applies at once, BIOS at the next reset).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One DMTF Redfish implementation, plus a profile per vendor with what
its documentation (or Ironic/sushy, MAAS, bmclib field notes) says is
different, each with sources: Dell iDRAC, HPE iLO, Lenovo XCC,
Supermicro, xFusion/Huawei iBMC, KAYTUS/Inspur, Fujitsu iRMC, Cisco
CIMC, GIGABYTE, ASRock Rack, OpenBMC, AMI MegaRAC.
Generic rules: reset types and boot targets from AllowableValues or
ActionInfo; boot target, enable and mode in one PATCH, the mode kept,
omitted or tried per profile; If-Match from header or @odata.etag with
412 (strong, then none) and 428 handling; AMI future-state /SD and
@Redfish.Settings targets; busy retries (POST, 409, 503, "Retry in");
202 task following; chassis reset fallback; emulated power cycle;
session login; 400 answers that say Success.
Probe results (reset types, boot targets, mode, profile) are kept on
the BMC record and shown in the UI; the UI's driver list comes from the
server. Each profile has a fake BMC test built from its sources.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Findings from a full review (auth/API, CA and rack channel, config
generation, integrations, UI/agents/scripts), each verified in code:
- rack binary endpoint read any region file via arch=../.. (CA key)
- render endpoint traversal; machine-reported MACs used as file paths
- delegated intermediates could mint admin/rack identities and IPs;
identities now require root-issued certs, intermediates are
ServerAuth-only with both name types constrained
- rack certs were ServerAuth for rack-chosen SANs: now client-only;
racks accept only the region's own cert; join checks the CA
fingerprint; unbound join codes cannot take an existing name; job
results bound to the rack that took them
- rack self-update: region signs binaries with the CA key, the root
unit verifies against the join-time CA in a root-owned dir
- dnsmasq reads a checked, root-owned copy limited to generated options
- commissioning could overwrite deployed machines; MAC takeover of PXE
- injection into dnsmasq/cloud-init/PXE/udev/DNS via names and
machine-reported strings; stored XSS in the UI; CSP and cross-origin
protection; deploy agent shell injection and seed traversal
- Redfish requests confined to the BMC's /redfish/ tree
- admin_http defaults to loopback; body limits and timeouts; secrets
off command lines
Also: IPAM claim/reconcile race, sync stopping on one error, watchdog
race, release wiping the wrong disk, SAN LUNs as auto targets.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CA (llmgw-style)
- region creates its CA (ECDSA P-256) and serves API/UI/images over HTTPS on tls_listen with a
certificate for every name/address it has (reissued on change or a month before expiry);
admin_http policy (allow|loopback|deny) for the admin API over plain HTTP; /ca.pem
- rack controllers enroll with one-time join codes (CA fingerprint + secret + expiry + address):
rollcall controller join-code, rollcall rack join; mutual TLS from then on, identity from the
certificate (CN rack:NAME, serial on record), removal revokes, renewal a month before expiry;
the shared rack_secret stays as a legacy fallback
- clients: CA from the server config, ROLLCALL_CA, or trusted on first connect in the console after
showing the fingerprint (kept in client.json); deployed machines trust it via cloud-init ca_certs
(allowed on RHEL-family images), overlays get .CA; rollcall ca show|issue
Power drivers
- ipmi (cipher/privilege/interface), redfish (system discovery, ResetType from AllowableValues or
ActionInfo, boot override with If-Match), idrac, ilo, supermicro, xcc, openbmc, proxmox (API
token), manual; executed on the rack controller; Redfish auto-detection after commissioning;
operator settings survive recommissioning; rollcall bmc, BMC editing in web UI and console
- lab: pve_redfish.py Redfish emulator per Proxmox VM (iDRAC, iLO, generic profiles)
IPv6
- IPv6 subnets and IPAM (netip), dual-stack default layout (all subnets of the boot VLAN),
per-family default routes, accept-ra off where addresses are planned, dnsmasq RA/DHCPv6 with
router lifetime 0 unless the rack is the gateway, AAAA host records from IPAM, IPv6 listen
addresses and URLs, rack interfaces and region certificate with IPv6, agent reports IPv6
Tests for CA/join/pinning/CSR signing, Redfish (iDRAC, iLO fallbacks, probe), Proxmox, manual,
IPv6 candidates, dual-stack plans and dnsmasq IPv6.
Verified in the lab: rack2 enrolled and running on mutual TLS only, revocation and single-use
codes; iLO auto-detected through rack2, iDRAC and Proxmox drivers, wrong password not saved;
Debian 13 (bond+VLAN), Rocky 9 and AlmaLinux 9 dual-stack with Redfish-driven deployments; CA
trusted on Debian and AlmaLinux; admin CLI over HTTPS on IPv4 and IPv6.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>