Files
Ilker ManapandClaude Opus 5.5 05861bd413
CI / go (push) Successful in 51s
CI / agent (push) Successful in 3s
CI / site (push) Successful in 3s
Intermediates renew themselves through the region
The region keeps every intermediate it signs: its name constraints,
certificate, the root that signed it and a renewal token (stored hashed,
printed once). `rollcall ca intermediate` goes through the API now (so it
works from any admin client) and writes NAME.token beside NAME.pem;
`rollcall ca intermediates` lists them.

The host with the intermediate's key runs deploy/rollcall-intermediate-
renew.sh daily (rollcall-intermediate-renew@NAME.timer): it posts a CSR
from the same key with the token to /api/v1/ca/intermediates/NAME/renew.
The region signs again in the last third of the certificate's life, or
at once when another root signs since a rotation's switch, always with
the constraints it was first given; otherwise 204. The script checks the
answer (a CA certificate for its key, from a root it holds), keeps the
old file, keeps the region's roots and the active root current from
/ca-bundle.pem, and reloads (RELOAD).

Attention: an intermediate under 30 days from its end, whose last renewal
failed or whose host has not asked for three days shows in rollcall
doctor, on the web UI's overview, and one from the old root in the
rotation status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 17:33:31 +01:00

56 KiB
Raw Permalink Blame History

Rollcall

Rollcall is a small, self-contained commissioning service for bare-metal servers, in the spirit of MAAS enlistment and commissioning, without the rest of MAAS.

A server that PXE-boots for the first time gets an ephemeral Debian live image. The image takes a full hardware inventory (DMI, DIMMs, CPUs, disks, NVMe, NICs, InfiniBand HCAs, GPUs, PCI, LLDP neighbours, TPM), creates an administrator account on the server's BMC through in-band IPMI, reports everything to the Rollcall server and powers the machine off. From then on the server is known: it shows up in the web UI and the CLI, its BMC can be driven through the API, and it will not boot the commissioning image again unless you ask for it.

Everything server-side is one static Go binary: HTTP API, embedded web UI, CLI, PostgreSQL schema management and the PXE bookkeeping.

How it works

unknown server ──PXE──▶ dnsmasq (DHCP+TFTP) ──▶ pxelinux default menu
                                                  │ 8 s timeout
                                                  ▼
                               Debian live image over HTTP (kernel+initrd via TFTP,
                               squashfs via HTTP, runs entirely from RAM)
                                                  │
                      rollcall-agent: inventory + BMC user + POST /api/v1/commission
                                                  │
                     rollcall server: PostgreSQL ◀─┘──▶ writes pxelinux.cfg/01-<mac>
                                                       (next boot: "commissioned → power off")
  • Enlist: a machine is identified by its DMI system UUID (fallback: serial, then boot MAC). First contact creates it in state new.
  • Commission: the inventory upload moves it to ready. The BMC password the agent generated is sealed with AES-256-GCM (key in /etc/rollcall/master.key) before it touches the database. It never leaves the server again: power and boot-device operations run ipmitool server-side.
  • Re-commission: removes the per-MAC PXE override; optionally sets bootdev=pxe and power-cycles the machine through its BMC.

Repository layout

Path What
cmd/rollcall/ Go server + CLI + embedded web UI (web/)
agent/ rollcall-agent.py (runs inside the live image), its systemd unit, build-image.sh (live-build recipe)
deploy/ install.sh (region), install-rack.sh + rack.example.json (rack controller), units, config.example.json, os-sources.json, tags/, overlays/
images/ stateless root image recipes (build-rootfs.sh, compute-debian13/)
lab/ Proxmox test lab: pve-vbmc (virtual IPMI BMC for VMs backed by the Proxmox API), cloud-init for the PXE server VM
site/ Project website (static)
.gitea/workflows/ CI (fmt/vet/build/tests), release on tags, deploy on push to main

Server

New site: INSTALL.md (deploy/bootstrap.sh, from day 0 without a network plan to the first deployed machine). The steps below are the manual equivalent.

# build
cd cmd/rollcall && CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o rollcall .

# PostgreSQL role + database
sudo -u postgres psql -c "CREATE ROLE rollcall LOGIN PASSWORD '...';" -c "CREATE DATABASE rollcall OWNER rollcall;"

# config (see deploy/config.example.json), tokens from `rollcall keygen`
sudo install -d -m 0750 /etc/rollcall && sudo install -m 0640 config.json /etc/rollcall/config.json

# install binary, systemd unit, pxelinux menu, tftp symlinks
cd deploy && sudo ./install.sh

config.json keys: listen, image_dir (served under /rollcall/), tftp_root, database_url, master_key_file, admin_token (web UI + management API), agent_token (what the live image presents), commission_append (kernel command line baked into generated PXE files), bmc_hints (out-of-band BMC mapping for machines without in-band IPMI, e.g. VMs).

Console (same binary)

A bare rollcall in a terminal opens a full-screen console, built like the llmgw one (Bubble Tea): the fleet down the left (Overview, Machines, Racks, Jobs, Events) and provisioning (Tags, Profiles, OS images), the chosen page on the right, keys along the bottom. It is keyboard only, so it works on serial consoles and BMC KVMs.

  • Machines: / filter (name, serial, MAC, IP, rack, tag), s cycles the state filter, Space selects, * selects everything shown, Enter opens a machine, a opens the actions for the selection or the current row: power, boot device, deploy OS, assign/unassign profile, tags, location, re-commission, release, cancel job, delete. Destructive ones ask first.
  • Racks: Enter shows the elevation; Enter on a unit opens that machine.
  • Jobs and events open the machine they belong to; tags, profiles and OS images show their definition. r refreshes (it also refreshes on its own, every 3 s while a job runs), ? lists the keys, q quits.

On the server as root it reads /etc/rollcall/config.json. Anywhere else it asks for the server URL and admin token once and keeps them in ~/.config/rollcall/client.json (mode 0600), which the CLI then uses as well. In a pipe or a script a bare rollcall prints its help as before. rollcall tui -dump 140x40 -page machines -plain renders one screen to stdout.

CLI (same binary)

rollcall list
rollcall show m-2ef1dab828c8 [-json]
rollcall events m-2ef1dab828c8
rollcall power m-2ef1dab828c8 status|on|off|soft|cycle|reset
rollcall bootdev m-2ef1dab828c8 pxe|disk|cdrom|bios|none
rollcall recommission m-2ef1dab828c8 [-now]
rollcall delete m-2ef1dab828c8

Reads ROLLCALL_URL / ROLLCALL_TOKEN, or the config file when readable.

API

Method Path Auth Purpose
POST /api/v1/commission agent token inventory upload from the live image
GET /api/v1/bmc-hint?macs= agent token out-of-band BMC hint for VMs
GET /api/v1/machines admin list (summaries)
GET /api/v1/machines/{id} admin full record: inventory, BMC (no secret), events
POST /api/v1/machines/{id}/power admin {"action":"on"} etc., runs ipmitool server-side
POST /api/v1/machines/{id}/bootdev admin {"device":"pxe"}
POST /api/v1/machines/{id}/recommission[?now=1] admin boot the image again on next PXE (now=1: via BMC immediately)
DELETE /api/v1/machines/{id} admin forget the machine
GET/POST/DELETE /api/v1/profiles[/{name}] admin stateless profiles
POST /api/v1/machines/{id}/assign /unassign admin stateless node assignment
GET /api/v1/os admin OS image catalog
POST /api/v1/machines/{id}/deploy admin {"os":"rocky-9","disk":"/dev/sda","hostname":"","package_upgrade":false,"boot":true}
POST /api/v1/machines/{id}/release admin wipe job
GET / POST /api/v1/machines/{id}/jobs[/cancel] admin job history / cancel
GET /api/v1/jobs/self, POST …/progress job token (X-Rollcall-Job) deploy agent
POST /api/v1/phone-home/{token} job token in path cloud-init phone_home
GET /os/* none OS disk images
GET /rollcall/* none image files (kernel, initrd, squashfs)
GET / token in browser web UI

Admin auth is Authorization: Bearer <admin_token>; the agent sends X-Rollcall-Token.

Commissioning image

agent/build-image.sh builds a Debian 13 live image with live-build (about 8 minutes, ~780 MB squashfs). It contains the full Debian kernel with non-free firmware, ipmitool, freeipmi, rdma-core, infiniband-diags, mstflint, nvme-cli, smartmontools, lldpd, lshw and an SSH server (root, key only) for debugging. Output goes to image_dir.

Kernel parameters understood by the agent: rollcall.server=URL (required), rollcall.token=…, rollcall.bmcuser=NAME (default commissioner), rollcall.nopoweroff (stay up after reporting).

BMC account

In-band (/dev/ipmi0): find the LAN channel, pick the existing commissioner slot or the first empty user ID ≥ 3, set name/password, ADMINISTRATOR privilege, channel access, enable, SOL payload. The password is random (16 chars) and is reported once to the server. If no in-band IPMI exists the agent asks the server for a hint (bmc_hints) and verifies it out-of-band.

Vendor quirks (iDRAC/iLO password policies, missing empty slots) are reported in bmc.errors and visible in the UI.

Stateless nodes (Warewulf-style)

For HPC compute nodes Rollcall can boot a root image straight into RAM and dress it with per-node files, instead of installing an OS on disk.

  • Images live in images_dir/<name>/{vmlinuz,initrd.img,filesystem.squashfs} and are built with images/build-rootfs.sh <name> from images/<name>/{packages.list,hook.chroot,includes/} (live-build, same toolchain as the commissioning image). images/compute-debian13 is a Debian 13 compute node: slurmd, munge, rdma-core, NFS client, firmware.
  • Profiles bind an image to overlays, extra kernel parameters, a node name format (node%03d) and variables: rollcall profile add compute -image compute-debian13 -overlays compute.
  • Assigning a commissioned machine to a profile allocates a node name and IPs from node_pools, writes a per-MAC PXE entry that boots the image with hostname=, rollcall.server= and a per-node token, and exports static DHCP leases plus host names for dnsmasq (dnsmasq_dir, re-read on HUP via rollcall-dnsmasq.path). rollcall assign <id> compute -boot also power-cycles the machine through its BMC.
  • Overlays are directory trees under overlays_dir/<overlay>/{system,runtime}; files ending in .tmpl are Go templates rendered with the node, all nodes of the cluster, the profile and cluster_vars. The compute overlay generates /etc/hosts, slurm.conf (node lines from the commissioning inventory: sockets, cores, threads, memory, GPUs), gres.conf, a persistent SSH host key per node (sealed in the database), root authorized_keys and the munge key. Inside the image rollcall-overlay.service applies the system overlay before munge/slurmd/ssh start; a timer refreshes the runtime overlay every five minutes, which doubles as the node heartbeat (booted state, last seen).
  • The controller renders the same files: rollcall render compute etc/slurm/slurm.conf > /etc/slurm/slurm.conf.

States: ready → stateless → booted; rollcall unassign releases name and IPs.

OS deployment (install to disk)

For nodes that should run an operating system from their own disk, Rollcall writes a vendor cloud image onto the disk and lets cloud-init do first-boot configuration. No installer, no kickstart/preseed: a deployment is a streamed disk write and takes about as long as the image download.

rollcall deploy <id> rocky-9 -boot -watch
   │ server: picks the disk (smallest that fits, or -disk), allocates hostname + IP,
   │         creates a job with a one-time token, writes pxelinux.cfg/01-<mac>
   │         ("deploy" entry = commissioning image with rollcall.mode=deploy), power-cycles via BMC
   ▼
commissioning image, deploy mode (agent/rollcall-deploy.py)
   GET /api/v1/jobs/self        → image URL, sha256, disk, rendered cloud-init seed
   curl | tee sha256 | pv | zstd -d | dd of=/dev/sda      (progress every 3 s)
   mount root fs → /var/lib/cloud/seed/nocloud/{meta-data,user-data}, datasource_list: [NoCloud, None]
   (SELinux images: /.autorelabel; UEFI hosts: efibootmgr entry for the ESP loader)
   POST progress state=done  → server: state deployed, PXE entry = local disk, reboot
   ▼
first boot: cloud-init sets hostname/fqdn, SSH keys (root + default user), optional
package_upgrade, then phone_home → POST /api/v1/phone-home/<token> (shown as "phoned home")

Image catalog and freshness

os_dir/<name>/<version>/disk.raw.zst     raw disk image, zstd
os_dir/<name>/<version>/meta.json        sizes, sha256, upstream checksum, import time
os_dir/<name>/current -> <version>       what new deployments use

os_sources (deploy/os-sources.json) lists upstream "latest" URLs with the vendor checksum file for each: Debian 13, Ubuntu 24.04, Rocky 9, AlmaLinux 9, CentOS Stream 9. rollcall os sync fetches the checksum file, and only when the vendor checksum differs from every imported version does it download the image, verify it, convert to raw (qemu-img), compress, write a new version directory and flip current atomically. rollcall-os-sync.timer runs it nightly; keep versions are retained per source, nothing younger than a day is pruned, and a job always pins the exact version it started with. Sources with "track": "pinned" still download new versions but keep current until rollcall os promote <name> <version>. Local files go in with rollcall os import <name> <file.qcow2>. Image freshness is monthly at best upstream; the deploy option -upgrade (package_upgrade) pulls the rest on first boot.

rollcall os list | sync [NAME...] [-force] | import NAME FILE [-family debian|rhel] | promote NAME VERSION | rm NAME VERSION
rollcall deploy ID OS [-version V] [-disk /dev/nvme0n1] [-hostname H] [-upgrade] [-packages a,b] [-user-data FILE] [-boot] [-watch]
rollcall release ID [-wipe quick|full] [-boot] [-watch]     # wipe the disk, back to ready
rollcall jobs ID [-watch] | job cancel ID

States: ready → deploying → deployed (deploy-failed on error; the agent stays up for ssh). A deployed machine's PXE entry defaults to local disk, so boot order can stay "network first" on BIOS hosts; on UEFI hosts the agent adds a boot entry for the installed loader. rollcall recommission or release take it back.

Tags

Tags single out machines for special treatment, as in MAAS. A tag has a name and optionally a match rule, a cloud-config fragment and overlays.

rollcall tags add gpu -match '{{ gt .Node.GPUs 0 }}' -cloud-config deploy/tags/gpu-nvidia.yaml -comment "NVIDIA nodes"
rollcall tags add rack-3 -comment "rack 3, row B"        # manual only
rollcall tag m-2ef1dab828c8 +rack-3 +burn-in -staging     # manual tags on one machine
rollcall list -tag gpu
rollcall deploy tag:gpu rocky-9 -boot                    # every gpu machine
rollcall power tag:rack-3 off
  • Auto tags come from the match rule, a Go template over .Node (GPUs, IB GUIDs, CPUs, memory, disks, NVMe, PCI vendor ids, manufacturer, product, OS, existing tags). They are recomputed at every commissioning and whenever a tag definition is saved; manual tags are never touched by rules.
  • At deployment the fragments of the machine's tags are appended to the generated user-data as MIME parts (packages, runcmd, write_files merge additively; a #! fragment runs as a script). deploy/tags/gpu-nvidia.yaml installs the NVIDIA driver on Debian, Ubuntu and the RHEL family and reboots.
  • For stateless nodes a tag's overlays are applied after the profile's, so a gpu tag can add gres.conf lines or driver modules without a second profile.

Physical location

Every machine can carry building / room / rack / rack unit and an asset tag.

rollcall location m-2ef1dab828c8 -building DC1 -room "Hall A" -rack R12 -unit 24 -asset A-0042
rollcall location tag:gpu -building DC1 -room "Hall B"             # bulk by tag
rollcall location import racks.csv -dry                             # from a DCIM / asset export
rollcall location racks | rack R12 -building DC1 -room "Hall A" -height 48
rollcall list -rack R12

The CSV needs a header; rows are matched by id, serial, mac or node_name and may carry building, room, rack, unit, asset_tag. Racks are created on first use (42U) and can be resized. The web UI shows a location column with building › room › rack chips, a Racks page with elevation views (machines drawn at their U, coloured by state) and bulk "Set location…". Templates and tag rules see .Node.Building, .Node.Room, .Node.Rack, .Node.Unit, so a rack can drive a tag ({{ eq .Node.Rack "R12" }}) or a Slurm topology.conf overlay.

Server heights are not typed in. unit is the lowest U a machine occupies; how many it takes comes from the firmware (SMBIOS type 3 "Height", read at commissioning), else from the model (a table of checked heights for Dell, HPE, Lenovo, xFusion, Fujitsu, Inspur/KAYTUS and GIGABYTE models, see hwsize.go), else from rack_units in config.json for models neither knows ("rack_units": {"SYS-6029P-TRT": 2}; it also overrides the others). Sleds of a multi-node chassis (Dell C-series, blades) share the chassis's units: the elevation draws them as one block, and NetBox gets a device type of height 0 placed in the rack without a position. The elevation spans each server over its height, marks unknown heights ?U and warns about overlaps.

Sites, rooms and floor plans. The Sites page puts sites on an OpenStreetMap map (Leaflet, shipped in the binary; the browser loads tiles from tile.openstreetmap.org), coloured by the worst machine state; a site opens its rooms, a room its floor plan: racks on 600 mm floor tiles, one wide and two deep, a thick edge for the front, coloured by their machines with a bar for the units used; a rack opens its front (the elevation above), and #/sites/SITE/ROOM/RACK links to it. Edit layout drags racks to their place and rotates them; racks nobody placed are laid out in rows facing alternate aisles (drawn dashed). With NetBox as the IPAM authority, sites (name, coordinates, address), locations (rooms) and racks (name, height) are read from it on every reconcile and never written back; without NetBox, or for a site NetBox has no coordinates for, sites are placed by clicking the map. Rack positions are Rollcall's either way (NetBox has no floor coordinates). API: /api/v1/sites, /api/v1/rooms, POST /api/v1/racks/positions.

Network and rack controllers

Model

As in MAAS: fabric → VLAN → subnet → IP ranges.

  • A fabric is a switched layer-2 domain (ethernet or infiniband).
  • A VLAN has a VID (0 = untagged; a partition key on IB), an MTU, and says whether rollcall serves DHCP/PXE on it and which rack controller does (a second one splits the dynamic pool with it).
  • A subnet has a purpose (provisioning, mgmt, bmc, storage, ib, other), a gateway, DNS servers and a domain.
  • Ranges inside a subnet are dynamic (the DHCP pool PXE and commissioning use), reserved (never handed out: routers, switches, controllers) and static (where machines get their addresses; without one, anything outside dynamic/reserved).
  • IPAM records every address handed out (rollcall ip list), pinned ones (rollcall net set ID -static ens18=10.0.0.60) and reserved single addresses (rollcall ip reserve 10.0.0.5 -name switch-1). Addresses a machine already had are adopted when its subnet is defined.
rollcall fabric add hall-a
rollcall subnet add 10.50.0.0/24 -fabric hall-a -vid 0 -purpose provisioning -gateway 10.50.0.1 -dns 10.50.0.1 -domain dc.example
rollcall range add 10.50.0.0/24 reserved 10.50.0.1 10.50.0.20
rollcall range add 10.50.0.0/24 static   10.50.0.50 10.50.0.199
rollcall range add 10.50.0.0/24 dynamic  10.50.0.200 10.50.0.250
rollcall vlan add hall-a 0 -dhcp -rack hall-a-rack1      # this rack controller serves DHCP/PXE here
rollcall subnet add 10.60.0.0/24 -fabric hall-a -vid 200 -purpose storage
rollcall dhcp hall-a-rack1                                 # the dnsmasq configuration it generates

Machine interfaces

A layout describes interfaces in terms that fit many machines: NICs are chosen by selectors (boot, name:X, mac:X, fastest:N, ib:N, all), not by MAC. Interfaces are physical, bond (802.3ad by default) or vlan; each link names a subnet (CIDR, name, or boot) and a mode: auto (an address from the subnet), static (a given one), dhcp or link_up.

{"interfaces": [
  {"type": "physical", "select": "boot", "links": [{"subnet": "boot", "mode": "auto", "default_gateway": true}]},
  {"type": "bond", "name": "bond0", "members": ["fastest:2"], "mtu": 9000, "links": [{"mode": "link_up"}]},
  {"type": "vlan", "parent": "bond0", "vid": 200, "links": [{"subnet": "10.60.0.0/24", "mode": "auto"}]},
  {"type": "physical", "select": "ib:0", "links": [{"subnet": "ib0", "mode": "auto"}]}
]}

rollcall layout add gpu-bond layout.json, then either per machine (rollcall net set ID -layout gpu-bond) or per tag (rollcall tags add gpu … -layout gpu-bond). Without a layout the boot NIC gets an auto address on its subnet; without any subnet model, deployed machines use DHCP as before.

At deployment the layout is resolved against the machine's inventory, addresses are allocated, and two files go into the image next to the cloud-init seed: network-config (cloud-init v2; cloud-init renders it for netplan, NetworkManager or ifupdown) and a udev rule that pins each NIC's name to its MAC, because RHEL-family cloud images boot with net.ifnames=0. Release returns the allocated addresses. rollcall net show ID previews it all.

Verified in the lab: Debian 13 and Ubuntu 24.04 (netplan) and Rocky 9 (NetworkManager), with a static boot NIC and with an active-backup bond carrying VLAN 200.

Region and rack controllers

rollcall serve is the region: database, API, web UI, IPAM, image store. Rack controllers sit next to the machines, one or two per network segment (hall, building, site):

  • they serve DHCP and TFTP with dnsmasq on the VLANs assigned to them; the configuration, per-MAC PXE files and host files are computed by the region;
  • they run BMC commands (BMC networks are rarely routed to the region);
  • they serve boot and OS images from a local cache, filled from the region on first use (kernels and initrds are prefetched, TFTP cannot wait);
  • they pass the node API (commissioning, deploy jobs, overlays, phone-home) through to the region and tell it which rack a machine came through.

A rack controller only needs to reach the region: it long-polls its desired state and a command queue. The region host runs an embedded rack controller (rack_name, default its hostname), which is all a single-server installation needs. Remote ones:

rollcall controller join-code -name hall-b -addr 10.0.0.234:443     # on the region: one word, one use, 60 min
# on the rack host (Debian/Ubuntu), from deploy/ with the binary next to it:
./rollcall rack join rollcall-join1-... -url http://10.213.78.1     # writes /etc/rollcall/rack.json + certificate
sh install-rack.sh                                                  # dnsmasq, syslinux, cache dirs, rollcall-rack.service
rollcall controllers                                                # on the region: it shows up online
rollcall vlan add hall-b 0 -dhcp -rack hall-b

dnsmasq is restarted only when its generated configuration changes, and only if it comes back; otherwise the last working configuration is restored (a hand-written DHCP configuration for the same interface is the usual cause: install.sh points it out). A machine that moved to another segment gets rollcall net set ID -rack NAME.

The lab runs both: pxe-server (embedded, 10.213.77.0/24) and pxe-rack2 (VM 213, remote, 10.213.78.0/24 on SDN vnet pxenet2), with node002 behind pxe-rack2 and its virtual BMC on pxe-rack2. Rack ↔ region traffic is mutual TLS (next section), so the BMC passwords of queued commands are encrypted.

Agents over HTTPS

Agents talk to their rack controller over HTTPS: the commissioning agent, the deploy agent and the image download. The region gives each rack a server certificate for its own URL's host (CN=rack-server:NAME, server use only, never one of the region's addresses). The rack serves the agents' endpoints on <host>:443 as well as on HTTP; tls_listen in rack.json changes the port, off turns it off. The embedded rack uses the region's own HTTPS listener.

The pxelinux command line carries rollcall.tls=https://HOST and rollcall.ca=<SHA-256 of the CA>. An agent fetches /ca.pem, accepts it only by that fingerprint, and from then on uses HTTPS with it. When TLS was asked for and fails, it does not fall back to HTTP.

What stays on HTTP: live-boot fetching the commissioning squashfs, and /ca.pem itself (checked by its fingerprint). This keeps BMC passwords and seed files from anyone who only listens, and from an HTTP server posing as the rack. TFTP and the kernel and initrd stay unauthenticated, so an attacker who can rewrite boot files on the wire is not stopped; that needs UEFI Secure Boot with signed images.

DHCP already on the network: proxy DHCP

Where the site's DHCP server keeps handing out addresses (a corporate DHCP, Infoblox, a router), a VLAN can be served in proxy mode: the rack controller does not hand out addresses, it only answers PXE clients with where to boot from (dnsmasq proxy DHCP), and TFTP and HTTP stay on the rack.

rollcall vlan add FABRIC VID -proxy-dhcp -rack RACK      # web UI: subnet, DHCP / PXE: proxy
deploy/bootstrap.sh --iface eth1 --proxy-dhcp           # the same at install time
  • The subnet needs no ranges. Without a static range, deployed machines keep getting their addresses from the site's server (dhcp links); set a static range aside, outside the site's pool, to give them fixed ones.
  • No dhcp-host lines are written for proxy subnets, and the router and DNS options come from the site's server.
  • rollcall doctor expects the foreign DHCP server here, and fails if none answers.
  • dnsmasq (2.91, still in 2.93) loops forever in its proxy code when a proxy range carries a set: tag: Rollcall's generated configuration leaves the tag out and limits the PXE services by interface instead.

Verified in the lab with a separate DHCP server on the provisioning network: commissioning and three deployments in a row through the proxy.

Fixed leases on the site's DHCP server (Kea, OpenWrt)

With a reservation driver, machines on proxy VLANs still get the addresses Rollcall plans: every such address becomes a fixed lease (the MAC of its interface, the address, the hostname) on the site's server. This is the counterpart of Infoblox's fixedaddress. Leases are reconciled on every address change and every sync_interval. Rollcall changes or deletes only its own leases; one held by someone else (same MAC or address) is reported as a conflict.

"dhcp_reservations": {
  "provider": "kea",
  "url": "http://kea.example:8000",
  "user": "rollcall",
  "password_file": "/etc/rollcall/kea.pw"
}
  • kea: the control API with the host_cmds hook, open source since Kea 3.0; Debian's Kea 2.6 lacks it and the driver says so.

    • Ownership goes in user-context {"rollcall": machine}.
    • With a hosts database the leases are kept there. Without one they go into the running configuration, and config-write saves them (persist: false turns that off).
  • openwrt: rpcd over HTTP (url: http://router). Fixed leases are host sections in /etc/config/dhcp with option rollcall, followed by uci commit and a dnsmasq reload. Give Rollcall its own rpcd login with only this:

    /usr/share/rpcd/acl.d/rollcall.json
    { "rollcall": { "description": "Rollcall fixed DHCP leases",
        "read":  { "ubus": { "uci": ["get"] }, "uci": ["dhcp"] },
        "write": { "ubus": { "uci": ["add", "delete", "commit"], "rc": ["init"] }, "uci": ["dhcp"] } } }
    

    and a config login in /etc/config/rpcd (username, an MD5-crypt password, list read/write 'rollcall').

rollcall reservations            # provider, last sync, conflicts
rollcall reservations sync -dry

IPv6 addresses are skipped: DHCPv6 reservations are keyed on DUIDs, not MACs. So are interfaces without a single MAC, such as bonds.

Verified in the lab against Kea 3.0.5 and OpenWrt 25.12.6: the machine got its planned address from the site's server and not from the pool.

Certificate authority and TLS

The region is the installation's certificate authority, as the llmgw control plane is: ca_dir (default /etc/rollcall/ca) holds ca.pem and ca-key.pem (ECDSA P-256, ten years), created on first start.

  • HTTPS. The region serves API, UI and images on tls_listen (default :443) with a certificate for every name and address it has plus tls_names; it is reissued when they change or a month before it ends. admin_http decides what plain HTTP may still do with the admin API: allow (default, for compatibility), loopback (the CLI on the region host) or deny. Nodes keep booting over HTTP: firmware and the live image cannot check a certificate before they have one.

  • Rack controllers join with a code from rollcall controller join-code: one word carrying the first half of the CA's SHA-256, a one-time secret, an expiry and the region's address. rollcall rack join checks the region against the fingerprint, sends a CSR with the secret and gets a client+server certificate (rack:<name>, a year). From then on every rack call is mutual TLS and the region identifies the rack by its certificate; rollcall controller rm revokes it at once. Racks renew a month before expiry. The old rack_secret is still accepted when configured.

  • Updates. Racks follow the region's version: the region names the binary (version, architecture, SHA-256) in each rack's state and serves it over the rack's mutual-TLS channel. The rack stages it in /var/lib/rollcall/update after checking the checksum and that it reports the expected version; rollcall-rack-update.path (root) installs it, restarts the rack and puts the previous binary back if the rack does not register again within a minute (last-result says what happened). Racks on another architecture get rack_binaries/rollcall-linux-<arch>; "auto_update": false in rack.json opts out. rollcall controllers shows each rack's version.

  • Admin certificates. Instead of the admin token, a person or laptop can hold a client certificate: rollcall admin-cert request NAME (with the token once) makes a key locally, sends only a CSR and gets admin:NAME back; the CLI and console use it from client.json, and the .p12 it writes goes into the browser's keychain, after which the web UI signs in by itself. rollcall admin-cert list | revoke NAME (revocation takes effect within 30 seconds). Certificates are checked against the CA, the client-auth usage and the serials the region recorded.

  • Clients. On the region host the CLI reads the CA from ca_dir; elsewhere ROLLCALL_CA=ca.pem, or the console shows the CA fingerprint on first connect and keeps the CA in ~/.config/rollcall/client.json once you confirm it (rollcall ca show prints it on the region). /ca.pem serves the certificate; the web login links to it.

  • Machines trust it: deployments write the CA into the trust anchors of both families (/usr/local/share/ca-certificates, /etc/pki/ca-trust/source/anchors) and rebuild the store at every boot (update-ca-certificates or update-ca-trust; cloud-init's ca_certs fails on RHEL-family systems); stateless overlays get it as .CA. rollcall ca issue NAME -host h1,h2 [-client] issues leaves for anything else (Slurm REST, a registry, a web front end).

  • Delegation. The root may sign one level of intermediate CAs, each limited by name constraints to the domains and address ranges it serves: rollcall ca intermediate caddy -permit int.example.com,10.213.0.0/16 -csr caddy.csr signs a CSR (the key stays on the host that made it). An ACME server under that intermediate (Caddy's acme_server, step-ca) then issues short-lived certificates to nodes with certbot or acme.sh; everything that trusts the Rollcall CA trusts them, and nothing outside the permitted names can be issued. Roots created before this (path length 0) cannot delegate: rotate the root (below). deploy/tags/acme-cert.yaml is a tag that fetches a node certificate at first boot and renews it.

  • Intermediates renew themselves. The region keeps every intermediate it signs and prints a renewal token once (NAME.token next to NAME.pem). On the host with the intermediate's key, deploy/rollcall-intermediate-renew.sh (installed as /usr/local/sbin/rollcall-intermediate-renew) and the rollcall-intermediate-renew@NAME.timer ask the region every day with that token and a CSR from the same key; settings in /etc/rollcall-intermediate/NAME.env:

    URL=https://region.example
    TOKEN_FILE=/etc/rollcall-intermediate/caddy.token
    CERT=/etc/caddy/pki/int.pem
    KEY=/etc/caddy/pki/int-key.pem
    ROOT=/etc/caddy/pki/root.pem          # optional: kept to the active root
    RELOAD='systemctl restart caddy'      # optional: after a new certificate
    

    The region signs again in the last third of the certificate's life, or right away when another root signs since a rotation's switch, always with the name constraints it was first given; the script checks what comes back (a CA certificate for its key, from a root it holds), keeps the old file as .prev, keeps the region's roots current from /ca-bundle.pem, and runs RELOAD. rollcall ca intermediates lists them; one under 30 days from its end, whose renewal failed or whose host has not asked for three days shows in rollcall doctor, on the web UI's overview and in the rotation status. -replace signs one the region keeps again with a new token (a lost token); DELETE /api/v1/ca/intermediates?name= stops keeping it (it stays valid to its end: there is no revocation list).

  • Root rotation without re-joining. In three steps, each followed by a restart of the region (systemctl restart rollcall):

    rollcall ca rotate prepare   # a new root; the old one still signs
    rollcall ca rotate           # status: which rack has what
    rollcall ca rotate switch    # the new root signs; the old one stays trusted
    rollcall ca rotate retire    # the old root is no longer trusted
    

    Everything that trusts the region gets the trust bundle (all current roots, /ca-bundle.pem). Rack controllers receive it signed by the active root and their root unit installs it only if that signature verifies against the roots they already trust; after switch they renew their client and server certificates from the new root by themselves. switch waits until every rack has the bundle, retire until every rack runs on certificates from the new root (-force goes on; racks that missed it join again). Machines deployed by this version get the roots one per file and a daily rollcall-ca-refresh timer that fetches the bundle over HTTPS from their rack, checked against the roots they trust, so they pick up a new root during prepare and drop the old one after retire: leave at least a day between prepare and switch (the rotation status cannot see deployed machines). A machine that missed it says so in its journal and needs the bundle by hand. Machines deployed earlier need it once, during prepare (curl --cacert <old root> https://<rack>/ca-bundle.pem; after switch the servers no longer show the old root). Admin certificates and intermediates signed by the old root work until retire; issue them again after switch (the status lists the admin certificates; an ACME server under an intermediate also needs its stored certificates issued again, e.g. Caddy's certificates/<ca> directory moved away). /ca.pem stays a single certificate (the root of that server's own certificate), which agents check by fingerprint.

Power drivers

Every machine's BMC record names a driver; commands run on the rack controller that reaches the BMC network.

Driver For
ipmi IPMI 2.0 over LAN (ipmitool lanplus), any vendor; settings cipher (17 on newer BMCs), privilege, interface
redfish and the vendor drivers below DMTF Redfish (see below)
proxmox Proxmox VE VMs: API token (user = token id, password = secret), settings node, vmid
manual no power control; actions say so

Redfish across vendors

There is one Redfish implementation of the DMTF standard. These rules hold for every vendor:

  • Reset types and boot targets. Read from what the BMC advertises (@Redfish.AllowableValues or the action's ActionInfo), never assumed.
  • Boot override. Target and enable go in one PATCH.
  • If-Match. Sent with the resource's ETag (header or @odata.etag). On 412 it is retried with the strong form, then without; on 428 it is fetched first.
  • Pending settings. A BMC that keeps them elsewhere (.../SD, @Redfish.Settings) gets the same PATCH there.
  • Busy answers. Retried for up to a minute: the server is in POST, 409 right after a power change, 503, "Retry in N seconds".
  • 202 answers. Followed to their task monitor.
  • Reset on the chassis. Used when the system has no reset of its own.
  • Missing power cycle. ForceOff, wait for Off, then On.
  • Paths from the BMC. Only followed if they stay on the BMC and under /redfish/.

What a vendor documents differently is a profile in cmd/rollcall/redfish.go, with its sources. The profile decides the driver name, how the BMC is recognised, its usual system path, the boot mode rule, how to clear an override and the login method.

Driver Vendor Boot mode in an override Notes
idrac Dell iDRAC 8/9/10 not sent: a change starts a BIOS job System.Embedded.1; iDRAC 10 takes boot settings at .../Settings
ilo HPE iLO 4 (2.30+), 5, 6 current one, without it if refused no PowerCycle (ForceRestart); no boot changes during POST (retried)
xcc Lenovo XCC, XCC2 current one, without it if refused override is Once only
supermicro Supermicro X11+, H12+ always the current one left out, some boards switch to Legacy
xfusion xFusion / Huawei iBMC current one, without it if refused session login; If-Match on every PATCH; ForcePowerCycle
kaytus KAYTUS / Inspur current one, without it if refused no public Redfish guide: reset may be on the chassis, a PATCH that worked can answer 400
irmc Fujitsu / Fsas iRMC S5, S6 not sent Systems/0; ETag from the body; clearing is Target None (never Enabled Disabled); no GracefulShutdown (power button)
cimc Cisco UCS C-series CIMC current one, without it if refused Systems/<serial>, found through the collection
gigabyte GIGABYTE (MegaRAC SP-X) always the current one Systems/Self; If-Match required; short reset list
asrockrack ASRock Rack (MegaRAC) always the current one boot settings in Systems/Self/SD
openbmc OpenBMC (bmcweb) current one, without it if refused Systems/system
ami AMI MegaRAC SP-X boards always the current one Systems/Self; If-Match required

Settings a machine can override:

Setting Effect
system= the system path
boot_mode=UEFI|Legacy always send this mode
auth=session|basic login method
verify_tls=true check the BMC's certificate

Every profile is tested against a fake service that behaves as its sources describe (redfish_test.go). The profiles have not yet run against the real machines.

HTTPS BMCs (Redfish, Proxmox) are trusted by their certificate. Before credentials first go to such a BMC, the rack controller reads its certificate with a TLS handshake that sends nothing. The region pins the SHA-256 on the BMC record as extra tls_fingerprint. A later connection showing a different certificate sends no credentials: the command fails with both fingerprints, and the machine gets the event bmc certificate changed.

To trust a replaced BMC, clear the pin with rollcall bmc ID -set tls_fingerprint=. Changing the address does the same, and so does a BMC firmware update run by Rollcall. To check against the system's CAs instead, set verify_tls=true.

After commissioning, a BMC still on IPMI is asked whether it speaks Redfish with the same account; if it does, the driver for its vendor takes over (event bmc driver). An operator's choice is never overridden.

rollcall bmc drivers
rollcall bmc m-1234 -driver idrac -address 10.0.5.21 -user root -password - -verify < pw
rollcall bmc m-1234 -driver redfish -set system=/redfish/v1/Systems/1 -set verify_tls=true
rollcall bmc m-1234 -probe            # detect Redfish again

Firmware updates (Redfish)

Rollcall keeps firmware packages in a repository on the region, reads each machine's firmware inventory from its BMC, and runs updates as jobs.

# on the region: a file (or from anywhere: an https URL, which the region downloads)
sudo rollcall firmware import BIOS_R760_WN64_2.3.5.EXE -vendor idrac -component bios -version 2.3.5 -models R760 -sha256 ...
rollcall firmware list
rollcall firmware show m-1234 -refresh        # what is installed, what the repository offers
rollcall firmware update m-1234 idrac-bios-2.3.5-2c49752f -reboot -watch
rollcall firmware update tag:r760 idrac-bmc-7.20.10.50-097a4a54

The package. A package names the BMC driver it is for (idrac, ilo, supermicro, ...), a component (bios, bmc, nic, raid, cpld, psu, drive, other), the version the component reports once installed, and optionally the models it fits. An update refuses a package for another vendor or model unless -force. The BMC checks the package itself: signature, model and downgrade policy. An HPE .fwpkg is unpacked at import to the iLO .bin or UEFI .signed.flash inside, because iLO's SimpleUpdate accepts a .fwpkg and then flashes nothing.

The job. The region drives the update through the rack controller next to the BMC:

  1. It hands the package over: an upload, or the BMC downloads it from the rack controller's /firmware/. Set firmware_base= on the BMC record when BMCs reach another address.
  2. It watches the BMC's task. A BMC that restarts after flashing itself is waited for, up to 25 minutes.
  3. It reads the inventory until the new version shows.

-reboot power-cycles the host when the update waits for a reset (BIOS on most vendors), then verifies. Without it, a staged update ends as "accepted, applies at the next reboot". A running update survives a region restart. It cannot be cancelled once the BMC has the package.

How each vendor takes a package. Each vendor profile in cmd/rollcall/redfish_fwplans.go carries its documented request, with sources. Without a profile, the standard methods are tried in this order: multipart push, SimpleUpdate pull, HttpPushUri.

Driver How Packages Notes
idrac MultipartUpload (SimpleUpdate pull on older iDRAC 9) DUP .EXE, .d9 OnReset by default; BIOS/RAID/NIC apply at the next reboot
ilo SimpleUpdate: iLO pulls over HTTP iLO .bin, UEFI .signed.flash (from .fwpkg) components only an OS agent flashes need SUM/iSUT
xcc /mfwupdate, after taking HttpPushUriTargetsBusy .uxz update XCC before UEFI
supermicro /UpdateService/upload with targets and preserve options .bin licence SFT-OOB-LIC/SFT-DCMS-SINGLE; host powered off for BIOS
gigabyte, ami, asrockrack /UpdateService/upload with OemParameters ImageType .ima, .ima_enc, .hpm, .bin the BMC restarts after a BMC flash; ASRock Rack unverified
openbmc update-multipart (BMC Immediate, others OnReset) signed .tar
xfusion upload to /tmp/web, then SimpleUpdate .hpm BMC, BIOS, CPLD
kaytus upload, then OEM SimpleUpdate (FlashItem) .hpm, .bin from KAYTUS's own SDK; BMC and BIOS
irmc OEM actions (FWUpdate, BiosUpdate) iRMC .bin, BIOS .UPC
cimc not supported Cisco documents only SCP pulls

Per machine, on the BMC record:

Setting Effect
fw_method=multipart|simple|push skip the vendor plan and use this standard method
fw_params={...} replace the plan's UpdateParameters (e.g. Supermicro Gen 13 BIOS preserve options)
firmware_base=http://ADDR the address BMCs download from

The lab Redfish emulator has an UpdateService of the same shape: iDRAC profile multipart, iLO profile pull. The BMC version applies at once, the BIOS at the next reset.

The lab has lab/pve-vbmc/pve_redfish.py, a Redfish service per Proxmox VM with iDRAC, iLO and generic profiles, next to the IPMI emulator.

IPv6

Subnets can be IPv6 (rollcall subnet add fd00:50::/64 ...). IPAM hands out addresses from static ranges (walked from their start), deployments get them in network-config with a separate default route per family, and the default layout links every subnet of the boot NIC's VLAN, so an IPv6 prefix next to the IPv4 one makes machines dual-stack. Interfaces with planned addresses ignore router advertisements. dnsmasq serves router advertisements and DHCPv6 on IPv6 subnets with dhcp on; the rack offers itself as default router only when it is the subnet's gateway. Host files carry AAAA records, rack controllers report their IPv6 addresses, the region certificate covers them, and region, racks and clients work over IPv6 (https://[fd00::1]).

PXE stays IPv4: syslinux cannot netboot over IPv6, so the provisioning VLAN needs an IPv4 subnet for commissioning and deployment.

Verified in the lab: Debian 13 (with a bond and VLAN), Rocky 9 and AlmaLinux 9 dual-stack on two segments, AAAA lookups through dnsmasq, the admin CLI over https://[fdc3:...].

External IPAM (NetBox, Infoblox)

Rollcall works on its own, but at a site with an established network plan the address authority is somewhere else. With ipam in config.json every address Rollcall hands out is registered there first:

"ipam": {
  "provider": "netbox",
  "url": "https://netbox.int.example.com",
  "token_file": "/etc/rollcall/netbox.token",
  "ca_file": "/etc/rollcall/netbox-ca.pem",
  "dns_zone": "int.example.com",
  "sync_interval": "5m"
}
  • Allocation asks the authority. An address that exists in NetBox and is not Rollcall's is skipped and the next candidate is tried, so Rollcall never collides with a printer, a VIP or another team's reservation. If NetBox is unreachable, allocation fails instead of guessing.
  • Ownership is explicit. Rollcall's records carry the tag rollcall and the machine ID in the custom field rollcall_machine (create both in NetBox; on Infoblox these are an extensible attribute). Records without both are never changed or deleted, only reported.
  • Releases reconcile. Releasing, unassigning or deleting a machine frees addresses locally and triggers a reconcile, which also runs every sync_interval: missing records are created, names and owners fixed, and Rollcall-owned records that are no longer in use deleted.
  • DNS follows the record. dns_name is <host>.<zone> for a machine's provisioning/mgmt address and <host>-<iface>.<zone> elsewhere (a subnet's domain replaces the zone when it lies inside it; pxe.local stays local). A NetBox-to-DNS sync (PowerDNS here, a host record on Infoblox) publishes it; Rollcall writes one object per address.
  • Prefixes are delegated in NetBox. Tag a prefix rollcall; rollcall ipam prefixes lists them and rollcall ipam import creates the missing subnets (VLAN and rack are then set as usual).
rollcall ipam                 # provider, last reconcile
rollcall ipam sync -dry       # what would change
rollcall ipam sync
rollcall ipam prefixes | import

Devices in NetBox

With device_sync in the ipam block, the NetBox driver also keeps every commissioned machine as a dcim device:

"ipam": { "provider": "netbox", ..., "device_sync": { "site": "dc1", "role": "server" } }
  • What is written. Name (node name, else the machine ID), serial, status (deployed/booted/stateless are active, commissioned ones staged, failed jobs failed), the device type from the DMI manufacturer and product, the role (default server), interfaces with their MACs (NetBox 4.2+ MAC objects, set as primary MAC), Rollcall's addresses assigned to those interfaces, and the management address as primary IP. Machines still new are skipped.
  • Location comes from NetBox. The machine's building, room and rack are matched by name to a site, a location and a rack (in that site); the unit becomes the rack position, the asset tag the device's. Rollcall never creates sites, locations or racks; a machine whose building matches no site goes to device_sync.site, which must exist. Manufacturers, device types and the role are created when missing (a device needs them).
  • Ownership as for addresses. Devices carry the tag rollcall and the machine ID in rollcall_machine (enable the field for devices too). A device of the same name without them is reported as a conflict and left alone; Rollcall's devices whose machine is gone are deleted. Interfaces are removed only when they carry the tag.
  • It runs after the address reconcile, in the same loop; rollcall ipam sync reports both (devices in the JSON).

The NetBox user needs view/add/change/delete on dcim.device, dcim.interface and dcim.macaddress, view/add on dcim.manufacturer, dcim.devicetype and dcim.devicerole, and view on dcim.location and dcim.rack, on top of the IPAM permissions.

NetBox's UI through Rollcall

When NetBox runs under a path (its BASE_PATH, e.g. BASE_PATH = 'netbox/') and ipam.url carries it ("url": "https://netbox.example/netbox/"), Rollcall serves NetBox's web UI at that same path on a listener of its own, https://<rollcall>:8444/netbox/ (ipam.ui_listen, default :8444), for admins only (the admin certificate, as for the API). NetBox keeps its own login. This opens NetBox wherever Rollcall opens, also when the browser cannot reach NetBox itself. The separate port is on purpose: another origin than Rollcall's UI, so a script in a NetBox page (a custom link, an export template) can neither read the Rollcall UI's stored admin token nor call Rollcall's API as the admin. Two settings on the NetBox side:

BASE_PATH = 'netbox/'
CSRF_TRUSTED_ORIGINS = ['https://rollcall.example:8444']   # where browsers post from

Signing in with the admin certificate. With "ui_login_secret_file" in the ipam block, the proxy tells NetBox who the admin is: X-Rollcall-User (the certificate's admin name) with the shared secret in X-Rollcall-Proxy-Secret. NetBox takes the user from that header and creates it on first use:

REMOTE_AUTH_ENABLED = True
REMOTE_AUTH_BACKEND = 'netbox.authentication.RemoteUserBackend'
REMOTE_AUTH_HEADER = 'HTTP_X_ROLLCALL_USER'
REMOTE_AUTH_AUTO_CREATE_USER = True
REMOTE_AUTH_DEFAULT_GROUPS = ['rollcall-admins']   # give this group the permissions admins should have

The web server in front of NetBox must drop X-Rollcall-User from every request that does not come from Rollcall with the secret, or anyone who reaches NetBox directly could claim a user. With Caddy:

@notrollcall not {
	remote_ip 10.0.0.5                       # the region
	header X-Rollcall-Proxy-Secret "<secret>"
}
request_header @notrollcall -X-Rollcall-User
request_header -X-Rollcall-Proxy-Secret

Rollcall drops both headers when a browser sends them, in any spelling a WSGI server would read as the same (X-Rollcall_User). An admin with a token instead of a certificate signs in to NetBox by hand. lab/netbox/install.sh sets all of this up for the lab NetBox.

The web UI then links to NetBox: the sidebar, a site, room and rack on the Sites pages, a switch, and a machine's device. ipam.ui_url points those links elsewhere (NetBox reached directly) when that is preferred; a NetBox at the root of its host gets links to ui_url only.

Cabling and switches

GET /api/v1/cabling gives each located machine's ports and the switch port each is plugged into, from the LLDP its commissioning saw (lldpd in the commissioning image). A switch's place is NetBox's (the device with that name, its rack and position, read with the layout on every reconcile; the NetBox user needs view on dcim.device) or set by hand (POST /api/v1/switches {"name","building","room","rack","unit"}, which NetBox does not move; DELETE ?name= forgets it). The 3D room draws the backs of the servers, the switches and the cables; a switch with no place is drawn where most of its cables are, marked as estimated.

Where LLDP sees nothing, a port's cable comes from NetBox: the cables recorded on Rollcall's devices (through patch panels, to the far interface), read with the layout; the NetBox user needs view on dcim.interface and dcim.cable. Then from one entered by hand (POST /api/v1/cables {"machine","port","switch","switch_port"}, the ✎ in the cabling list). The BMC's own port is never seen by the host's LLDP: its cable to the out-of-band switch is NetBox's (an interface marked management-only or named iDRAC, iLO, XCC, IMM, IPMI, BMC, OOB or mgmt) or entered by hand with port bmc. When NetBox records another switch port than LLDP sees, the port shows what NetBox says. A switch whose NetBox role names out-of-band or management, or that only BMCs are cabled to, is marked OOB.

Infoblox

The same contract over Infoblox WAPI (v2.12 by default; give a full .../wapi/vX.Y URL for another):

"ipam": {
  "provider": "infoblox",
  "url": "https://gm.example.com",
  "user": "rollcall",
  "password_file": "/etc/rollcall/infoblox.pw",
  "ca_file": "/etc/rollcall/infoblox-ca.pem",
  "dns_zone": "int.example.com",
  "network_view": "default",
  "dns_view": "default"
}
  • One host record per address. Named by its DNS name (rollcall-<address> without one) and configured for DNS in dns_view, so Infoblox publishes the A/AAAA record itself; "dns_view": "-" keeps them IPAM-only. The zone must exist in that view.
  • Ownership is the extensible attribute rollcall_machine (define it in the grid as String). Rollcall changes it with extattrs+, so a host's other attributes stay; hosts without it are never changed or deleted.
  • Taken is anything else on the address: another host, a lease, a fixed address, a DHCP or reserved range, the network or broadcast address.
  • Delegated networks carry the extensible attribute rollcall (any value; define it as String); Site and VLAN attributes are shown when present.
  • The WAPI user needs read/write on host records and read on networks and IPv4/IPv6 addresses in the network view (and the DNS view when publishing).

Device sync (above) is NetBox-only.

The provider is an interface (reserve, owned records, update, delete, delegated prefixes) with the NetBox and Infoblox drivers.

DNS provider (PowerDNS, RFC 2136)

Without an IPAM authority whose DNS sync publishes names, Rollcall can write its machines' A/AAAA records into the site's DNS itself. Names follow the IPAM rule above (<host>.<zone>, <host>-<iface>.<zone>).

"dns": {
  "provider": "powerdns",
  "zone": "int.example.com",
  "url": "http://dns.int.example.com:8081",
  "api_key_file": "/etc/rollcall/pdns.key"
}
"dns": {
  "provider": "rfc2136",
  "zone": "int.example.com",
  "server": "ns1.int.example.com:53",
  "tsig_name": "rollcall-key",
  "tsig_algorithm": "hmac-sha256",
  "tsig_secret_file": "/etc/rollcall/tsig.secret"
}
  • powerdns uses the HTTP API. Rollcall's RRsets carry a comment with account rollcall; RRsets without it are never changed or deleted.
  • rfc2136 is a TSIG-signed dynamic update, for BIND, Knot, PowerDNS, Windows DNS and Infoblox. Next to each name Rollcall writes a marker _rollcall.<name> TXT "heritage=rollcall". A change to an existing record needs the marker as a prerequisite; a new record needs the name to be free. The server checks both atomically, so a record someone else owns is never overwritten. What Rollcall wrote is kept in the database, and each sync queries those records. A record changed behind Rollcall's back is repaired when the marker is still there. When someone else's record has taken the name, it is reported as a conflict. For BIND, grant the key the zone: update-policy { grant rollcall-key zonesub ANY; };.

A sync runs on every allocation and release and every sync_interval (default 5m), with ttl defaulting to 300.

rollcall dns                  # provider, last sync, conflicts
rollcall dns sync -dry        # what would change
rollcall dns sync

Conflicts and errors appear there, in the log and in rollcall doctor.

Lab

lab/ documents a Proxmox SDN test setup: an isolated vnet, a PXE server VM running dnsmasq + Rollcall, two diskless VMs, and pve-vbmc, a tiny IPMI-over-LAN BMC emulator that drives VMs through the Proxmox API so that ipmitool power on and chassis bootdev pxe work against virtual machines.

Status

Commissioning, stateless nodes and disk deployment work end to end in the lab (virtual machines with 30 GB disks, SeaBIOS, out-of-band BMC hints). The in-band BMC path and the UEFI boot-entry step are implemented but have not yet been exercised on real hardware.