With ipam.ui_login_secret_file, the proxy tells NetBox who the admin is
(X-Rollcall-User: the certificate's admin name) together with a shared
secret (X-Rollcall-Proxy-Secret). NetBox takes the user from that header
(REMOTE_AUTH) and creates it on first use; the web server in front of
NetBox drops the header from any request that lacks the secret or does
not come from the region, so reaching NetBox directly claims no one.
Both headers coming from a browser are always dropped by Rollcall; an
admin signed in with a token instead of a certificate logs in to NetBox
by hand. A secret under 16 characters or an unreadable file leaves the
proxy off, with a log line.
The lab NetBox install sets it up: REMOTE_AUTH settings, a guarded
Caddyfile (readable by caddy only: the secret is in it), the secret in
secrets.env, and a rollcall-admins group that may do everything. Rollcall's
NetBox user may now view cables.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The host's LLDP never sees the BMC's own port. A port's cable now comes
from LLDP first (what is really there), then from a cable recorded in
NetBox on the machine's device (read with the layout, traced through
patch panels to the far interface via connected_endpoints), then from
one entered by hand (POST/DELETE /api/v1/cables, the pencil in the
cabling list). A NetBox interface marked management-only or named like
a BMC (iDRAC, iLO, XCC, IMM, IPMI, BMC, OOB, mgmt) is the machine's
"bmc" port, so BMCs get their cable to the out-of-band switch. When
NetBox records another switch port than LLDP sees, the port says what
NetBox says.
A switch whose NetBox role names out-of-band or management, or that
only BMCs are cabled to, is marked OOB. In 3D the BMC cable leaves the
BMC port on the back (thinner); the side panel lists every machine's
ports with their source, the BMC always, and switches carry an OOB tag.
Estimated switch places now go around the units placed switches take.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With NetBox as the IPAM under a path (its BASE_PATH) and ipam.url
carrying it, Rollcall serves NetBox's UI at that same path, for admins
only (the admin certificate, as for the API); NetBox keeps its own
login. Redirects that name NetBox's own address are turned back to
Rollcall's, Rollcall's credentials are not passed on. NetBox opens
wherever Rollcall does: a lab NetBox on a segment the browser cannot
reach, say.
The web UI links to NetBox: the sidebar, sites, rooms, racks, switches
and a machine's device (by its rollcall_machine field). ipam.ui_url
points the links at NetBox directly instead.
NetBox paging under a BASE_PATH doubled the path on the next page
(the next link carries it already); fixed. The lab NetBox moves under
/netbox/ and trusts the region's addresses for its forms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Sites page shows sites on an OpenStreetMap map (Leaflet 1.9.4 in the
binary, checked against cdnjs's SRI; the CSP allows OSM tiles and the
tile layer sends a referrer as OSM's policy asks). A site opens its
rooms, a room its floor plan: racks on 600 mm tiles, one wide and two
deep, front edge marked, coloured by their machines with a bar for the
units used; a rack opens its front, and the rack is part of the URL.
An edit mode drags racks into place and rotates them; racks nobody
placed are laid out in rows facing alternate aisles.
With NetBox as the IPAM authority, sites (coordinates, address),
locations and racks are read on every reconcile, never written back.
Otherwise sites are placed by clicking the map. Rack positions are
Rollcall's (NetBox has none).
Checked in headless Chrome with a synthetic site, including a scripted
drag, rotate and save of a rack.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nobody types in how many units a server takes. Commissioning reads the
SMBIOS chassis height; without one the model decides, from a table of
heights checked against vendor documentation (Dell, HPE, Lenovo,
xFusion, Fujitsu, Inspur/KAYTUS, GIGABYTE; Supermicro model numbers do
not carry the height), and rack_units in config.json covers the rest and
overrides both. Sleds of a multi-node chassis share its units.
NetBox device types get the height (0 for sleds, which go into the rack
without a position; Rollcall's own types are corrected). The rack
elevation spans each server over its height, draws a chassis's sleds as
one block, marks unknown heights and lists overlaps.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rollcall-ca-refresh reports a server it no longer trusts (the rotation
went past prepare before the machine refreshed) instead of exiting
quietly; the README asks for a day between prepare and switch. The
status lists admin certificates from before the switch, and retire
prints them: they stop working with the old root.
Tested end to end in the lab: prepare, switch and retire on pxe-server
with pxe-rack2 following without a new join (bundle installed by its
root unit, client and server certificates renewed), node001 (Debian) and
node002 (Rocky) moving to the new root with the refresh timer, and the
netops Caddy intermediate signed again by the new root.
Closes#12
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CA directory holds the active root and, during a rotation, the next
or the previous one; everything that trusts the region gets the trust
bundle (all of them). Rack controllers receive it in their state, signed
by the active root; the root update unit installs it only when the
signature verifies against the roots it already trusts, and binary
signatures verify against any of them. After switch racks renew their
client and server certificates from the new root by themselves and
report what they run on; switch and retire wait for every rack (-force).
/ca.pem stays one certificate, the root of that server's certificate, and
the PXE command line names that root, so agents in older images keep
working; /ca-bundle.pem is the bundle. Deployed machines get one file per
root and a daily timer that refreshes them from their rack over HTTPS.
Refs #12
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ipam.provider infoblox: one record:host per address, named by its DNS
name and configured for DNS in dns_view ("-" = IPAM only); ownership is
the rollcall_machine extensible attribute, changed with extattrs+ so a
host's other attributes stay. Anything else on the address (a host, a
lease, a fixed address, a DHCP or reserved range, network/broadcast) makes
it taken. Delegated networks carry the rollcall attribute. WAPI paging.
The reconcile no longer compares fields an authority does not keep
(Infoblox has no prefix length or status on a host). Built from the WAPI
reference and tested against a fake WAPI; no real grid yet.
Closes#11
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ipam.device_sync mirrors commissioned machines as dcim devices: serial,
device type and role (created when missing), status, interfaces with MAC
objects, Rollcall's addresses assigned to them and the management address
as primary IP. Site, location and rack are matched by name from the
machine's location and never created. Ownership by tag and
rollcall_machine; a foreign device of the same name is a conflict.
lab/netbox/install.sh sets up a NetBox on the lab segment so the drivers
are tested without writing to a real NetBox.
Refs #10
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Found in the lab run: pve-redfish runs with ProtectSystem=strict, so it
could not write its firmware state and dropped the reset request that
applies a staged BIOS. Tested end to end on the lab: iDRAC profile
multipart BMC update, iLO profile pull of BMC and BIOS through rack
controller pxe-rack2 with -reboot, model check and -force, resume of a
running update across a region restart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One DMTF Redfish implementation, plus a profile per vendor with what
its documentation (or Ironic/sushy, MAAS, bmclib field notes) says is
different, each with sources: Dell iDRAC, HPE iLO, Lenovo XCC,
Supermicro, xFusion/Huawei iBMC, KAYTUS/Inspur, Fujitsu iRMC, Cisco
CIMC, GIGABYTE, ASRock Rack, OpenBMC, AMI MegaRAC.
Generic rules: reset types and boot targets from AllowableValues or
ActionInfo; boot target, enable and mode in one PATCH, the mode kept,
omitted or tried per profile; If-Match from header or @odata.etag with
412 (strong, then none) and 428 handling; AMI future-state /SD and
@Redfish.Settings targets; busy retries (POST, 409, 503, "Retry in");
202 task following; chassis reset fallback; emulated power cycle;
session login; 400 answers that say Success.
Probe results (reset types, boot targets, mode, profile) are kept on
the BMC record and shown in the UI; the UI's driver list comes from the
server. Each profile has a fake BMC test built from its sources.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rollcall writes its machines' A/AAAA records into the site's DNS when
no IPAM authority publishes them. Names follow the IPAM rule.
- powerdns: RRsets owned by comment account "rollcall"; others are
reported as conflicts and left alone.
- rfc2136: TSIG-signed updates with a _rollcall.<name> TXT marker.
Changing a record requires the marker, creating one requires the
name to be free, so foreign records are never overwritten. Written
records are kept in dns_records and queried on every sync, so
out-of-band changes are repaired (marker present) or reported
(someone else's record).
- Sync on allocation/release and every sync_interval; rollcall dns,
rollcall dns sync [-dry], doctor check.
Tested against netops PowerDNS and BIND 9 (conflict, repair, wrong
TSIG secret).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A VLAN in proxy mode leaves addresses to the site's DHCP server; the rack
only answers PXE (dnsmasq proxy range + pxe-service), machines keep DHCP
unless a static range is set aside. CLI -proxy-dhcp, web UI mode select,
bootstrap --proxy-dhcp, doctor/discover aware.
dnsmasq 2.91 (and 2.93) loops forever in its proxy path when the proxy
range carries a set: tag: option_filter appends a request's tags to the
context's persistent list, so the next request's stack interface tag
points at itself (found with gdb on the lab rack). Proxy ranges are
untagged and PXE services limited by interface; dhcp-boot/dhcp-match only
for served subnets; no dhcp-host lines for proxied ones.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cloud-init's ca_certs ran update-ca-certificates on Rocky and failed, so
those machines never trusted the Rollcall CA. The CA is now written into
both trust-anchor directories and a bootcmd rebuilds the store with
update-ca-trust or update-ca-certificates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
INSTALL.md follows a clean run from day 0 to a deployed machine.
Release assets now include rollcall-<tag>-linux-<arch>.tar.gz (binary,
deploy/, agent/). deploy/publish-image.sh uploads a region's commissioning
image to a release (commissioning-*); bootstrap.sh downloads either layout,
with IMAGE_TOKEN for private repositories. bootstrap: --ssh-key (default:
the operator's keys), keeps server_url on reruns without --iface.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rollcall admin-cert request NAME makes a key locally and sends a CSR; the
region signs admin:NAME (client auth) and records the serial; a verified,
recorded, unrevoked certificate is an admin. CLI/console read it from
client.json, a .p12 is written for the browser; the web UI no longer insists
on a token.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The region offers its binary (version, arch, sha256) in each rack's state and
serves it over mutual TLS; the rack stages it after checking checksum and
version; a root path unit installs it, restarts the rack and rolls back if it
does not register within a minute.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New roots carry path length 1. `rollcall ca intermediate` signs a CSR (or a
generated key) as an intermediate limited to the permitted domains and
address ranges; a root with path length 0 refuses and `rollcall ca rotate`
replaces it. deploy/tags/acme-cert.yaml gets a node certificate from an
ACME server under such an intermediate and renews it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every address Rollcall hands out is registered in the configured authority
first; addresses others hold there are skipped. Rollcall's records carry the
tag `rollcall` and the machine ID in the `rollcall_machine` custom field;
only those are changed or deleted. Releases trigger a reconcile that also
runs periodically. dns_name per address feeds the site's DNS sync. Delegated
prefixes can be listed and imported as subnets. `rollcall ipam`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CA (llmgw-style)
- region creates its CA (ECDSA P-256) and serves API/UI/images over HTTPS on tls_listen with a
certificate for every name/address it has (reissued on change or a month before expiry);
admin_http policy (allow|loopback|deny) for the admin API over plain HTTP; /ca.pem
- rack controllers enroll with one-time join codes (CA fingerprint + secret + expiry + address):
rollcall controller join-code, rollcall rack join; mutual TLS from then on, identity from the
certificate (CN rack:NAME, serial on record), removal revokes, renewal a month before expiry;
the shared rack_secret stays as a legacy fallback
- clients: CA from the server config, ROLLCALL_CA, or trusted on first connect in the console after
showing the fingerprint (kept in client.json); deployed machines trust it via cloud-init ca_certs
(allowed on RHEL-family images), overlays get .CA; rollcall ca show|issue
Power drivers
- ipmi (cipher/privilege/interface), redfish (system discovery, ResetType from AllowableValues or
ActionInfo, boot override with If-Match), idrac, ilo, supermicro, xcc, openbmc, proxmox (API
token), manual; executed on the rack controller; Redfish auto-detection after commissioning;
operator settings survive recommissioning; rollcall bmc, BMC editing in web UI and console
- lab: pve_redfish.py Redfish emulator per Proxmox VM (iDRAC, iLO, generic profiles)
IPv6
- IPv6 subnets and IPAM (netip), dual-stack default layout (all subnets of the boot VLAN),
per-family default routes, accept-ra off where addresses are planned, dnsmasq RA/DHCPv6 with
router lifetime 0 unless the rack is the gateway, AAAA host records from IPAM, IPv6 listen
addresses and URLs, rack interfaces and region certificate with IPv6, agent reports IPv6
Tests for CA/join/pinning/CSR signing, Redfish (iDRAC, iLO fallbacks, probe), Proxmox, manual,
IPv6 candidates, dual-stack plans and dnsmasq IPv6.
Verified in the lab: rack2 enrolled and running on mutual TLS only, revocation and single-use
codes; iLO auto-detected through rack2, iDRAC and Proxmox drivers, wrong password not saved;
Debian 13 (bond+VLAN), Rocky 9 and AlmaLinux 9 dual-stack with Redfish-driven deployments; CA
trusted on Debian and AlmaLinux; admin CLI over HTTPS on IPv4 and IPv6.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- fabrics, VLANs, subnets, IP ranges (dynamic/reserved/static), IPAM with adoption of
addresses machines already have; rollcall fabric|vlan|subnet|range|ip
- interface layouts (physical/bond/vlan, NICs by selector: boot, name, mac, fastest:N, ib:N)
per machine or per tag; deployment allocates addresses and writes cloud-init network-config
plus udev name rules (RHEL cloud images boot with net.ifnames=0); release frees them;
rollcall layout, rollcall net show|set
- region/rack: rollcall rack long-polls its desired state (per-MAC PXE files kept in the
database, dnsmasq configuration and host files) and a BMC command queue, caches boot and OS
images (kernels/initrds prefetched for TFTP), proxies the node API and tags requests with its
name; the region runs an embedded rack controller; install-rack.sh, rollcall-rack.service
- dnsmasq configuration generated per rack from the VLANs it serves (split pool for two racks);
reload script restarts only on change and restores the last working config if dnsmasq does
not come back (drilled in the lab)
- web: Network page, per-machine network tab, layout action, tag layout; console: Network page,
network section in the machine view, layout action
- tests for candidates, plans (default, bond+VLAN, pinned, DHCP fallback), network-config,
udev rules and dnsmasq rendering
- lab: pve-vbmc address script finds the interface per BMC address
Verified in the lab: embedded rack (10.213.77.0/24) and a remote rack controller on a second
SDN segment (10.213.78.0/24): enlistment, commissioning, BMC through the rack's queue, image
cache, deployment and phone-home through the rack; Debian 13, Ubuntu 24.04 and Rocky 9 with a
static boot NIC and with an active-backup bond carrying VLAN 200.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Bubble Tea + lipgloss with the same frame as llmgw: title bar, sidebar, bordered panel,
key line, dialog overlay; keyboard only
- pages: overview, machines (filter, state filter, multi-select, bulk actions), machine
detail, racks with elevation, jobs, events, tags, profiles, OS images
- actions: power, boot device, deploy OS, assign/unassign, tags, location, re-commission,
release, cancel job, delete; confirmation for destructive ones; per-machine results
- connect screen when no server is configured; URL + token saved to
~/.config/rollcall/client.json (0600), also used by the CLI
- batched runes outside text fields are handled key by key
- rollcall tui -dump WxH renders one screen; first Go tests for rendering and keys
- web: unnamed machines show their id instead of the live image's hostname
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- machines.building/room/rack/rack_unit/asset_tag, racks table (height, comment)
- API: POST /machines/{id}/location, POST /locations/import (CSV or JSON, matched by
id/serial/mac/node_name, ?dry=1), GET/POST/DELETE /racks, ?building/?room/?rack filters
- CLI: rollcall location ..., location import, location racks/rack, list -rack/-room/-building
- UI: location column and building > room > rack chips, bulk Set location…, Racks page with
elevation views and CSV import dialog, location in machine detail
- .Node.Building/Room/Rack/Unit/AssetTag in overlay templates and tag match rules
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- tags table + machines.tags/auto_tags; match rules as Go templates over .Node
(GPUs, IB, CPUs, memory, disks, NVMe, PCI vendors, manufacturer/product, OS)
re-evaluated at commissioning and on tag changes
- deployment: tag cloud-config fragments appended as MIME parts with additive merge;
deploy/tags/gpu-nvidia.yaml installs the NVIDIA driver per distro family
- stateless: tag overlays appended to the profile's overlays
- CLI: tag, tags, list -tag, deploy/power tag:NAME fan-out; UI: tags column, per-machine
editor, tag definitions panel; API: /api/v1/tags, /machines/{id}/tags, ?tag=
- CHANGELOG.md; release workflow publishes the tag's section as release notes and
refuses tags without one
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Profiles bind a RAM-booted root image to overlay templates; assigning a
machine allocates a node name and management IP, writes the per-MAC PXE
entry with a per-node token and exports static leases for dnsmasq. Nodes
pull rendered overlays (hosts, slurm.conf from inventory, gres.conf, SSH
host key, munge key) through rollcall-overlay.service; the runtime refresh
timer is the heartbeat. Adds the compute-debian13 image definition and a
generic live-build root image script, CLI and web UI for profiles/assign.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Go server (API, embedded web UI, CLI) backed by PostgreSQL with AES-GCM
sealed BMC secrets; Debian live commissioning agent and live-build recipe;
Proxmox lab with pve-vbmc virtual BMCs; Gitea workflows for CI, releases
and deployment; project website.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
