Network model, IPAM, interface layouts and the region/rack split (MAAS-style)
CI / agent (push) Successful in 3s
CI / site (push) Successful in 2s
CI / go (push) Successful in 32s

- fabrics, VLANs, subnets, IP ranges (dynamic/reserved/static), IPAM with adoption of
  addresses machines already have; rollcall fabric|vlan|subnet|range|ip
- interface layouts (physical/bond/vlan, NICs by selector: boot, name, mac, fastest:N, ib:N)
  per machine or per tag; deployment allocates addresses and writes cloud-init network-config
  plus udev name rules (RHEL cloud images boot with net.ifnames=0); release frees them;
  rollcall layout, rollcall net show|set
- region/rack: rollcall rack long-polls its desired state (per-MAC PXE files kept in the
  database, dnsmasq configuration and host files) and a BMC command queue, caches boot and OS
  images (kernels/initrds prefetched for TFTP), proxies the node API and tags requests with its
  name; the region runs an embedded rack controller; install-rack.sh, rollcall-rack.service
- dnsmasq configuration generated per rack from the VLANs it serves (split pool for two racks);
  reload script restarts only on change and restores the last working config if dnsmasq does
  not come back (drilled in the lab)
- web: Network page, per-machine network tab, layout action, tag layout; console: Network page,
  network section in the machine view, layout action
- tests for candidates, plans (default, bond+VLAN, pinned, DHCP fallback), network-config,
  udev rules and dnsmasq rendering
- lab: pve-vbmc address script finds the interface per BMC address

Verified in the lab: embedded rack (10.213.77.0/24) and a remote rack controller on a second
SDN segment (10.213.78.0/24): enlistment, commissioning, BMC through the rack's queue, image
cache, deployment and phone-home through the rack; Debian 13, Ubuntu 24.04 and Rocky 9 with a
static boot NIC and with an active-backup bond carrying VLAN 200.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-27 23:58:40 +01:00
1 parent e3d9373201
commit 736a7e38a3
31 files changed
+4357 -176

No files matched your search

+109 -1
View File
@@ -44,7 +44,7 @@ unknown server ──PXE──▶ dnsmasq (DHCP+TFTP) ──▶ pxelinux default
|---|---|
| `cmd/rollcall/` | Go server + CLI + embedded web UI (`web/`) |
| `agent/` | `rollcall-agent.py` (runs inside the live image), its systemd unit, `build-image.sh` (live-build recipe) |
| `deploy/` | `install.sh`, `rollcall.service`, `config.example.json`, `pxelinux-default`, `os-sources.json`, sync/dnsmasq/controller units, `overlays/` |
| `deploy/` | `install.sh` (region), `install-rack.sh` + `rack.example.json` (rack controller), units, `config.example.json`, `os-sources.json`, `tags/`, `overlays/` |
| `images/` | stateless root image recipes (`build-rootfs.sh`, `compute-debian13/`) |
| `lab/` | Proxmox test lab: `pve-vbmc` (virtual IPMI BMC for VMs backed by the Proxmox API), cloud-init for the PXE server VM |
| `site/` | Project website (static) |
@@ -295,6 +295,114 @@ tag rules see `.Node.Building`, `.Node.Room`, `.Node.Rack`, `.Node.Unit`, so a
rack can drive a tag (`{{ eq .Node.Rack "R12" }}`) or a Slurm `topology.conf`
overlay.
## Network and rack controllers
### Model
As in MAAS: **fabric → VLAN → subnet → IP ranges**.
* A **fabric** is a switched layer-2 domain (`ethernet` or `infiniband`).
* A **VLAN** has a VID (0 = untagged; a partition key on IB), an MTU, and
says whether rollcall serves DHCP/PXE on it and which **rack controller**
does (a second one splits the dynamic pool with it).
* A **subnet** has a purpose (provisioning, mgmt, bmc, storage, ib, other), a
gateway, DNS servers and a domain.
* **Ranges** inside a subnet are `dynamic` (the DHCP pool PXE and
commissioning use), `reserved` (never handed out: routers, switches,
controllers) and `static` (where machines get their addresses; without one,
anything outside dynamic/reserved).
* **IPAM** records every address handed out (`rollcall ip list`), pinned ones
(`rollcall net set ID -static ens18=10.0.0.60`) and reserved single
addresses (`rollcall ip reserve 10.0.0.5 -name switch-1`). Addresses a machine
already had are adopted when its subnet is defined.
```sh
rollcall fabric add hall-a
rollcall subnet add 10.50.0.0/24 -fabric hall-a -vid 0 -purpose provisioning -gateway 10.50.0.1 -dns 10.50.0.1 -domain dc.example
rollcall range add 10.50.0.0/24 reserved 10.50.0.1 10.50.0.20
rollcall range add 10.50.0.0/24 static 10.50.0.50 10.50.0.199
rollcall range add 10.50.0.0/24 dynamic 10.50.0.200 10.50.0.250
rollcall vlan add hall-a 0 -dhcp -rack hall-a-rack1 # this rack controller serves DHCP/PXE here
rollcall subnet add 10.60.0.0/24 -fabric hall-a -vid 200 -purpose storage
rollcall dhcp hall-a-rack1 # the dnsmasq configuration it generates
```
### Machine interfaces
A **layout** describes interfaces in terms that fit many machines: NICs are
chosen by selectors (`boot`, `name:X`, `mac:X`, `fastest:N`, `ib:N`, `all`),
not by MAC. Interfaces are `physical`, `bond` (802.3ad by default) or `vlan`;
each link names a subnet (CIDR, name, or `boot`) and a mode: `auto` (an address
from the subnet), `static` (a given one), `dhcp` or `link_up`.
```json
{"interfaces": [
{"type": "physical", "select": "boot", "links": [{"subnet": "boot", "mode": "auto", "default_gateway": true}]},
{"type": "bond", "name": "bond0", "members": ["fastest:2"], "mtu": 9000, "links": [{"mode": "link_up"}]},
{"type": "vlan", "parent": "bond0", "vid": 200, "links": [{"subnet": "10.60.0.0/24", "mode": "auto"}]},
{"type": "physical", "select": "ib:0", "links": [{"subnet": "ib0", "mode": "auto"}]}
]}
```
`rollcall layout add gpu-bond layout.json`, then either per machine
(`rollcall net set ID -layout gpu-bond`) or per tag (`rollcall tags add gpu
… -layout gpu-bond`). Without a layout the boot NIC gets an `auto` address on
its subnet; without any subnet model, deployed machines use DHCP as before.
At deployment the layout is resolved against the machine's inventory,
addresses are allocated, and two files go into the image next to the
cloud-init seed: `network-config` (cloud-init v2; cloud-init renders it for
netplan, NetworkManager or ifupdown) and a udev rule that pins each NIC's name
to its MAC, because RHEL-family cloud images boot with `net.ifnames=0`.
Release returns the allocated addresses. `rollcall net show ID` previews it all.
Verified in the lab: Debian 13 and Ubuntu 24.04 (netplan) and Rocky 9
(NetworkManager), with a static boot NIC and with an active-backup bond
carrying VLAN 200.
### Region and rack controllers
`rollcall serve` is the **region**: database, API, web UI, IPAM, image store.
**Rack controllers** sit next to the machines, one or two per network segment
(hall, building, site):
* they serve DHCP and TFTP with dnsmasq on the VLANs assigned to them; the
configuration, per-MAC PXE files and host files are computed by the region;
* they run BMC commands (BMC networks are rarely routed to the region);
* they serve boot and OS images from a local cache, filled from the region on
first use (kernels and initrds are prefetched, TFTP cannot wait);
* they pass the node API (commissioning, deploy jobs, overlays, phone-home)
through to the region and tell it which rack a machine came through.
A rack controller only needs to reach the region: it long-polls its desired
state and a command queue. The region host runs an **embedded** rack
controller (`rack_name`, default its hostname), which is all a single-server
installation needs. Remote ones:
```sh
# region: config.json gets "rack_secret": "<rollcall keygen>"
# rack host (Debian/Ubuntu), from deploy/ with the binary next to it:
cat > /etc/rollcall/rack.json <<EOF
{"name": "hall-b", "region": "http://10.0.0.234", "secret": "<rack_secret>",
"url": "http://10.213.78.1", "listen": "10.213.78.1:80"}
EOF
sh install-rack.sh # dnsmasq, syslinux, cache dirs, rollcall-rack.service
rollcall controllers # on the region: it shows up online
rollcall vlan add hall-b 0 -dhcp -rack hall-b
```
dnsmasq is restarted only when its generated configuration changes, and only
if it comes back; otherwise the last working configuration is restored (a
hand-written DHCP configuration for the same interface is the usual cause:
`install.sh` points it out). A machine that moved to another segment gets
`rollcall net set ID -rack NAME`.
The lab runs both: `pxe-server` (embedded, 10.213.77.0/24) and `pxe-rack2`
(VM 213, remote, 10.213.78.0/24 on SDN vnet `pxenet2`), with node002 behind
pxe-rack2 and its virtual BMC on pxe-rack2. Rack ↔ region traffic is plain
HTTP in the lab; BMC passwords travel on it for queued commands, so put the
region behind TLS on real networks.
## Lab
`lab/` documents a Proxmox SDN test setup: an isolated vnet, a PXE server VM