rollcall/CHANGELOG.md

Changelog

All notable changes to Rollcall. The format follows Keep a Changelog; the release workflow uses the section of the pushed tag as the release notes and refuses to publish a tag that has no section here.

[Unreleased]

Added

• Proxy DHCP: a VLAN in proxy mode (rollcall vlan add ... -proxy-dhcp, web UI, bootstrap.sh --proxy-dhcp) leaves addresses to the site's DHCP server and only answers PXE; machines keep DHCP unless a static range is set aside; doctor and discover know the mode.
• DNS provider: dns in config.json writes machines' A/AAAA records into the site's DNS, through the PowerDNS API (ownership by RRset comment) or RFC 2136 dynamic update with TSIG (ownership by a _rollcall.<name> TXT marker and update prerequisites, so foreign records are never overwritten; drift is detected by querying). rollcall dns, rollcall dns sync [-dry], a doctor check.
• Firmware updates over Redfish. A package repository on the region (rollcall firmware import from a file or URL, served to BMCs through rack controllers) and each machine's firmware inventory.
  • Update jobs: hand-over, task watch through BMC restarts, optional power cycle, version check against the inventory. They resume after a region restart.
  • Vendor plans from documentation or vendor tools: Dell, HPE (.fwpkg unpacked), Lenovo, Supermicro, GIGABYTE/AMI/ASRock Rack, OpenBMC, xFusion, KAYTUS, Fujitsu iRMC. Cisco CIMC is refused (SCP only).
  • fw_method, fw_params and firmware_base on the BMC record.
  • CLI rollcall firmware, a Firmware tab and repository in the web UI.
• Redfish vendor profiles: one standard driver plus per-vendor behaviour from public documentation, for Dell iDRAC, HPE iLO, Lenovo XCC, Supermicro, xFusion/Huawei iBMC, KAYTUS/Inspur, Fujitsu iRMC, Cisco CIMC, GIGABYTE, ASRock Rack, OpenBMC and AMI MegaRAC.
  • Boot mode kept, omitted or tried per vendor.
  • If-Match with 412/428 handling; future-state (/SD) and @Redfish.Settings targets.
  • Busy retries and task following.
  • Chassis reset fallback and an emulated power cycle.
  • Session login (auth=session).
  • Probe results (reset types, boot targets, boot mode, profile) are kept on the BMC and shown in the UI, and the UI's driver list comes from the server.

Security

A review of the whole code base; the fixes below. Upgrading: rerun deploy/install.sh on the region and deploy/install-rack.sh on rack hosts (new dnsmasq layout, signed rack updates).

• Region file read (critical). A rack controller could read any file on the region (CA key, config) through /api/v1/rack/binary?arch=../... arch is now a known architecture.
• Path traversal. In /api/v1/render/... (admin) as well.
• Machine-reported MACs. A MAC could be a path, letting a PXE-booting host make the rack write outside the TFTP directory. Only real hardware addresses are accepted. Rack controllers write only 01-xx-.. file names.
• Delegated intermediates. An ACME intermediate could mint admin or rack certificates (name constraints do not cover the CN), and IP addresses when only a domain was permitted.
  • Admin and rack identities must now be signed by the root itself.
  • Intermediates carry a ServerAuth-only EKU, and the unconstrained name type is closed.
• Rack certificates. These were ServerAuth certificates for any address the rack reported, so one rack could pose as the region.
  • They are now client-only with no names.
  • Racks accept only the region's own certificate.
  • rack join checks the returned CA against the fingerprint in the code.
  • An unbound join code cannot take over an existing rack's name.
  • Job results are accepted only from the rack that took the job.
• Rack self-update. The root unit ran a binary the unprivileged rack user staged.
  • The region now signs each binary with its CA key.
  • The root unit verifies the signature against the CA from rack join, in a root-owned directory, without following links.
• dnsmasq as root. dnsmasq (root) read configuration the rollcall user writes, and the reload script copied files through links in rollcall's directory.
  • It now reads a checked copy in /var/lib/rollcall-dnsmasq.
  • The copy may hold only the options Rollcall generates: nothing that runs a program or includes a file.
• Machine record takeover. Any host on the provisioning network could overwrite a deployed machine's record (and BMC) by reporting its UUID.
  • Known machines are re-commissioned only when ready, recommission or failed.
  • A MAC another machine owns keeps its boot file.
  • The rack controller of a ready machine is not changed by whoever reports it.
• Injection into dnsmasq, cloud-init, PXE menus, udev rules and DNS updates. Subnet domain and gateway, host names, NIC names, reservation names and machine-reported identity strings are now validated or reduced to one line. Machine IDs are letters and digits only.
• Web UI.
  • Escaping is fixed for machine-reported inventory fields (stored XSS).
  • A Content-Security-Policy, frame-ancestors 'none', nosniff and no-referrer are set.
  • Cross-origin state-changing requests are refused (http.CrossOriginProtection), so a page elsewhere cannot use a browser's admin certificate.
  • ?token= is no longer logged.
• Deploy agent.
  • The image URL no longer goes into a shell command; the checksum is taken over the bytes written.
  • Seed files cannot leave the target root.
  • The BMC password is not kept in the fallback inventory file.
• Redfish. Requests stay on the BMC and under /redfish/: paths the BMC returns cannot send the credentials elsewhere, and scheme:// addresses are reduced to scheme and host.
• Machine tokens. Job and node tokens are readable from the TFTP pxelinux files. They are now accepted only from the machine's own address (its static lease); through a rack, the rack's X-Forwarded-For entry counts. A machine without a fixed address binds a job to its first user. agent_source_check: false turns this off.
• BMC certificates. HTTPS BMCs (Redfish, Proxmox) are pinned on first use. The certificate is read in a handshake that carries no credentials, and a changed certificate gets none (tls_fingerprint; cleared on an address change or a BMC firmware update by Rollcall).
• BMC hints. bmc-hint no longer returns the password. The region attaches the configured credentials itself and checks the BMC from the rack controller.
• Rack certificate renewal. Renewal could not overwrite the root-owned files from rack join. Renewed certificates now go atomically to /var/lib/rollcall/rack-cert, the previous serial is accepted for 7 days after a renewal, and rollcall rack renew renews on demand.
• Defaults and limits.
  • admin_http defaults to loopback.
  • Request bodies are capped (commissioning 4 MB, all 16 MB), with read and idle timeouts.
  • bootstrap.sh and publish-image.sh keep the database password and API tokens off command lines.

Fixed

• IPAM: a reconcile running during an allocation could delete the record just reserved in NetBox.
• IPAM and DNS sync: one failing record no longer stops the rest of the pass, deletes included. Reservations are stored in one spelling.
• The job watchdog could fail a job that had just started.
• Release without -disk wiped the smallest disk, not the one the OS was deployed to. Automatic disk choice skips SAN LUNs (iSCSI, FC, SRP).
• RFC 2136 lookups follow the request context. The marker is not removed when the database cannot say whether other records need it.
• Smaller fixes:
  • seq in templates is capped.
  • Layout selector ib:-1 no longer panics.
  • Redfish detection no longer runs on a machine that cannot be read.
• dnsmasq busy loop in proxy mode (a dnsmasq bug with tagged proxy ranges, found with gdb): proxy ranges are untagged, PXE services limited by interface; dhcp-boot/dhcp-match only for served subnets, no dhcp-host lines for proxied ones.

[0.2.1] - 2026-10-28

Fixed

• bootstrap.sh --image-url with a release URL took the release's SHA256SUMS (the binaries') instead of commissioning-SHA256SUMS; it now prefers the image's and says when a private repository needs IMAGE_TOKEN.

[0.2.0] - 2026-10-28

Second release: install from scratch, network model and region/rack split, external IPAM (NetBox), a certificate authority with delegation, many power drivers, IPv6, and a web UI and console for fleets.

Added

• Machine tags (MAAS-style): manual tags per machine, auto tags from match rules over the inventory ({{ gt .Node.GPUs 0 }}), re-evaluated at commissioning and when a tag changes.
• Tags carry a cloud-init fragment merged into deployment user-data (deploy/tags/gpu-nvidia.yaml installs the NVIDIA driver on GPU nodes) and extra overlays for stateless nodes.
• rollcall tag, rollcall tags, rollcall list -tag, deploy tag:NAME / power tag:NAME fan-out; tags column, editor and definitions panel in the web UI; ?tag= filter on the API.
• Web UI redesigned for fleets: sidebar navigation (Overview, Machines, Jobs, Tags, Profiles, OS images), state/tag filter chips, search, sorting and paging, multi-select with bulk power/deploy/assign/tag/re-commission/release, machine detail with tabs, dialog-driven actions, fleet-wide /api/v1/events and /api/v1/jobs, ?token= sign-in for bookmarks.
• Physical location per machine (building, room, rack, unit, asset tag): rollcall location, CSV import matched by id/serial/MAC/node name with dry run, rack definitions with height, list -rack/-room/-building, location column and chips in the UI, Racks page with elevation views, bulk "Set location…", .Node.Rack etc. in templates and tag rules.
• Terminal console: a bare rollcall in a terminal opens a full-screen UI in the llmgw style (Bubble Tea): overview, machines with filter/selection/bulk actions, machine detail, racks with elevation, jobs, events, tags, profiles, OS images; connect screen that saves ~/.config/rollcall/client.json, which the CLI also reads; rollcall tui -dump renders one screen. First Go tests (console rendering and keys).
• Network model after MAAS: fabrics, VLANs, subnets, IP ranges (dynamic/reserved/static) and IPAM; addresses already in use are adopted. rollcall fabric|vlan|subnet|range|ip.
• Interface layouts (physical, bond, VLAN; NICs by selector) per machine or per tag; deployment writes cloud-init network-config and udev name rules into the image; release frees addresses. rollcall layout, rollcall net show|set.
• Region/rack split: rollcall rack (dnsmasq state, per-MAC PXE files, BMC commands, image cache, node API proxy) long-polls the region; the region runs an embedded rack controller. deploy/install-rack.sh, rollcall controllers, rollcall dhcp.
• dnsmasq configuration generated from the model, restarted only when it changes and rolled back if dnsmasq does not come back.
• Web UI: Network page (rack controllers, subnets with ranges and usage, layouts), network tab per machine, layout action; console: Network page, network section, layout action.
• Certificate authority on the region (llmgw-style): HTTPS listener with an auto-renewed region certificate, admin_http policy, /ca.pem; rack controllers enroll with one-time join codes (rollcall controller join-code, rollcall rack join) and use mutual TLS, renewing their certificate themselves; removal revokes; deployed machines and overlays trust the CA; rollcall ca show|issue; the console trusts a CA on first connect after showing its fingerprint.
• Power drivers: ipmi, redfish, idrac, ilo, supermicro, xcc, openbmc, proxmox, manual, run on the rack controller; Redfish auto-detection after commissioning; rollcall bmc, BMC editing in the web UI and console. Lab Redfish emulator with iDRAC/iLO/generic profiles.
• IPv6: subnets, IPAM, dual-stack default layout, per-family default routes, dnsmasq RA/DHCPv6, AAAA host records, IPv6 listen addresses and URLs, IPv6 in the commissioning inventory.
• External IPAM authority (NetBox driver): allocation registers each address in NetBox first and skips addresses others hold there; ownership by tag + rollcall_machine custom field; releases and a periodic reconcile create, fix and delete only Rollcall-owned records; dns_name per address for DNS sync; delegated prefixes (rollcall ipam prefixes|import); rollcall ipam [sync -dry], /api/v1/ipam.
• Web UI: network topology drawing (region, external IPAM, rack controllers, one bus per VLAN with its subnets, machines wired to every VLAN they have an address on; large groups collapse) on the Network page and, compact, on the Overview. Overview redesigned: fleet state bar, per-rack counts, attention list incl. IPAM sync, capacity and free pool, OS, hardware, BMC drivers, locations, tags, recent deployments.
• CA delegation: new roots allow one level of intermediates; rollcall ca intermediate NAME -permit DOMAIN,CIDR [-csr FILE] signs a name-constrained intermediate (for an ACME server such as Caddy's acme_server); rollcall ca rotate -yes replaces a root that cannot delegate; tag fragment deploy/tags/acme-cert.yaml fetches and renews a node certificate with certbot.
• Rack controllers update themselves to the region's version: checksum- and version-checked download over mutual TLS, installed by a root path unit with automatic rollback; rack_binaries for other architectures, auto_update to opt out; versions in rollcall controllers and the web UI.
• Addresses on secondary subnets with a gateway get their own routing table and a source rule in network-config, so replies from a storage or tenant VLAN leave through that VLAN's gateway.
• With admin_http other than allow, browsers that open the web UI over HTTP are redirected to HTTPS (nodes keep HTTP for images and the agent).
• Admin client certificates: rollcall admin-cert request NAME (key stays on the client, CSR only), list, revoke; the region accepts them as admin over HTTPS; CLI/console use them from client.json, the web UI works without a token when the browser presents one (.p12 written).
• deploy/bootstrap.sh: a region on a fresh Debian 13 host. Day 0 needs nothing about the network: packages, PostgreSQL, TFTP tree, config, services and a verified commissioning image, the region reachable on its management address with no DHCP or subnets. Run again with --iface once the provisioning network is known: server address updated, subnet with ranges, DHCP, optional NAT.
• rollcall net discover: per interface link, speed, addresses and implied subnets, switch/port/VLAN from LLDP, DHCP servers that answer, neighbours, and what to do with it.
• commission_append follows server_url when left empty.
• rollcall doctor: checks configuration, database, listeners, certificates, TFTP files, image checksums, dnsmasq, DNS as machines see it, clock, foreign DHCP servers on the provisioning network (DHCPDISCOVER probe), rack controllers, OS images and IPAM, with what to do.
• rollcall version; CD-built binary carries git describe.
• Release notes from CHANGELOG.md.

Fixed

• RHEL-family deployments (Rocky, Alma, CentOS) did not trust the Rollcall CA: cloud-init's ca_certs ran update-ca-certificates there and failed. The CA now goes into the image's trust anchors and a bootcmd rebuilds the store with the distribution's tool.
• A deploy whose power command was lost (rack or region restarting) stayed pending forever and the event looked like a success: power errors are now part of the event (pxe boot failed), and a job the machine has not started within job_start_timeout (default 30m, room for servers that take 4-5 minutes to POST) fails with a hint.
• CentOS Stream 9 image URL (the 502 was a missing file name).
• Rack controllers' dnsmasq had no upstream DNS on Debian (it looks for a resolvconf file that is usually missing) and refused every name outside the cluster domain; the generated configuration now forwards to systemd-resolved's stub (keeping split DNS) or the host's resolv.conf servers.
• Virtual BMC addresses are configured in netplan on the lab PXE server (they were lost on a network reconfigure, which made every BMC call fail).
• Commissioning console no longer shows a failing autologin (noautologin).

[0.1.0] - 2026-10-27

First tagged release.

Added

• Commissioning: Debian 13 live image with firmware, IB and NVMe tooling; hardware inventory (DMI, DIMMs, CPUs, disks, NVMe, NICs, InfiniBand, GPUs, PCI, LLDP, TPM); in-band BMC account creation with out-of-band hints for VMs; secrets sealed with AES-256-GCM; ipmitool server-side.
• Single static Go binary: HTTP API, embedded web UI, CLI, PostgreSQL schema.
• Stateless (Warewulf-style) nodes: profiles, node names and IP pools, rendered overlays, dnsmasq export, Slurm compute image and controller sync.
• OS deployment: image catalog with nightly upstream sync and vendor checksum verification (Debian 13, Ubuntu 24.04, Rocky 9, AlmaLinux 9, CentOS Stream 9), streamed zstd | dd writes with sha256 and live progress, cloud-init NoCloud seed, SELinux relabel, UEFI boot entry, phone-home, release/wipe jobs.
• Gitea Actions: CI, deploy to the PXE server, website publish, release on tags.
• Lab: Proxmox SDN subnet, pve-vbmc IPMI emulator for VMs.