A VLAN in proxy mode leaves addresses to the site's DHCP server; the rack only answers PXE (dnsmasq proxy range + pxe-service), machines keep DHCP unless a static range is set aside. CLI -proxy-dhcp, web UI mode select, bootstrap --proxy-dhcp, doctor/discover aware. dnsmasq 2.91 (and 2.93) loops forever in its proxy path when the proxy range carries a set: tag: option_filter appends a request's tags to the context's persistent list, so the next request's stack interface tag points at itself (found with gdb on the lab rack). Proxy ranges are untagged and PXE services limited by interface; dhcp-boot/dhcp-match only for served subnets; no dhcp-host lines for proxied ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
159 lines
6.8 KiB
Markdown
159 lines
6.8 KiB
Markdown
# Installing Rollcall at a new site
|
|
|
|
This follows a site from nothing to a deployed machine. It was written from a
|
|
clean run: a fresh Debian 13 VM became the region, discovered its interfaces,
|
|
was given a provisioning network, and an empty machine commissioned itself and
|
|
was deployed with Debian 13.
|
|
|
|
The network does not have to be decided before you start. Day 0 installs the
|
|
region on its management address; the provisioning network is added when it
|
|
exists.
|
|
|
|
## What you need
|
|
|
|
| | |
|
|
|---|---|
|
|
| Region host | Debian 13, 2 CPUs, 4 GB RAM, 30 GB disk (plus ~1 GB per OS image), root |
|
|
| Network access | a default route; the internet or a mirror for packages and cloud images |
|
|
| Later: provisioning network | an interface of this host (or of a rack controller) on the VLAN where machines PXE-boot |
|
|
| Later: BMC network | the rack controller must reach the machines' BMCs (IPMI, Redfish) |
|
|
| Ports | DHCP 67/udp and TFTP 69/udp on the provisioning network; HTTP 80 for machines; HTTPS 443 for people and rack controllers |
|
|
|
|
The release bundle `rollcall-<version>-linux-amd64.tar.gz` holds the binary,
|
|
`deploy/` and `agent/`. From a checkout, `deploy/bootstrap.sh` builds the binary
|
|
itself when Go is installed.
|
|
|
|
## Day 0: install
|
|
|
|
```sh
|
|
tar xzf rollcall-v0.2.1-linux-amd64.tar.gz && cd rollcall-v0.2.1-linux-amd64
|
|
sudo ./deploy/bootstrap.sh --image-url https://gitea.example/OWNER/rollcall/releases/download/v0.2.1
|
|
```
|
|
|
|
- **Packages, database, TFTP.** PostgreSQL with its own role and a random password,
|
|
dnsmasq, the BIOS and UEFI syslinux files, lldpd.
|
|
- **Configuration.** `/etc/rollcall/config.json` with generated tokens. The admin
|
|
API only answers over HTTPS from other hosts (`admin_http: loopback`).
|
|
- **Commissioning image.** Downloaded and checked against `SHA256SUMS`. For a
|
|
private repository set `IMAGE_TOKEN`. Without a release to download from,
|
|
`--image-url http://EXISTING-REGION/rollcall` copies it from another region,
|
|
and `--build-image` builds it here with live-build (about 20 minutes).
|
|
- **SSH keys.** The keys of the user running sudo go to root on deployed
|
|
machines; add more with `--ssh-key KEY_OR_FILE`.
|
|
|
|
At the end the region answers on its management address: the web UI at
|
|
`https://ADDRESS`, no DHCP, no subnets. Check it:
|
|
|
|
```sh
|
|
sudo rollcall doctor
|
|
```
|
|
|
|
On day 0 the only warning should be that the host serves no DHCP yet.
|
|
|
|
## Find the provisioning network
|
|
|
|
```sh
|
|
sudo rollcall net discover
|
|
```
|
|
|
|
```
|
|
INTERFACE KIND LINK SPEED ADDRESSES SWITCH / PORT / VLAN DHCP SERVERS NEIGH
|
|
eth0* physical up - 10.0.0.116/24 - 10.0.0.1 4
|
|
eth1 physical up - 10.214.0.1/24 - none 0
|
|
|
|
eth0 uplink: management network with its own DHCP (10.0.0.1); keep it for the region's access
|
|
eth1 candidate provisioning network 10.214.0.0/24: bootstrap.sh --iface eth1
|
|
```
|
|
|
|
For each interface: link and speed, addresses and the subnet they imply, the
|
|
switch port and VLAN that LLDP reports, and DHCP servers that answer a probe.
|
|
The last column matters most: a DHCP server already on the provisioning VLAN
|
|
competes with Rollcall for the machines' PXE boot.
|
|
|
|
| What discover shows | What to do |
|
|
|---|---|
|
|
| An interface with an address and no DHCP | the provisioning network: next step |
|
|
| Link up, no address, LLDP shows the VLAN | give the interface its address on that VLAN first |
|
|
| Another DHCP server on the provisioning VLAN | keep it: `--proxy-dhcp` makes this host answer only PXE, the site's server keeps the addresses (or the network team turns it off for that VLAN) |
|
|
| No link | cable it |
|
|
|
|
## Add the provisioning network
|
|
|
|
```sh
|
|
sudo ./deploy/bootstrap.sh --iface eth1 --os debian-13
|
|
```
|
|
|
|
- **Server address.** The host's address on `eth1` becomes the server address
|
|
machines use. The management address stays in the certificate, so the web UI
|
|
keeps working there.
|
|
- **Subnet.** Modelled with ranges: reserved `.1-.20`, static `.50-.99` (where
|
|
deployed machines get their addresses), dynamic `.100-.200` (PXE and commissioning).
|
|
- **DHCP and PXE** on `eth1`, unless `--no-dhcp`.
|
|
- **Routing.** Without `--gateway`, this host routes for the machines: forwarding
|
|
plus masquerade out of the uplink. Give `--gateway IP` when the network has a router.
|
|
- **First OS image.** `--os debian-13` imports it now; a nightly timer keeps the
|
|
catalog current.
|
|
|
|
The same can be done in the web UI (Network, New subnet) or the CLI
|
|
(`rollcall subnet add`, `range add`, `vlan add ... -dhcp`), or imported from
|
|
NetBox (`rollcall ipam import`). Several provisioning VLANs, bonds and VLAN
|
|
layouts per machine are described in the README, under Network.
|
|
|
|
`rollcall doctor` should now report no failures. It also sends a DHCP
|
|
DISCOVER on the provisioning interface and fails if any other server answers.
|
|
|
|
## First machines
|
|
|
|
Set a machine to PXE-boot on the provisioning network and power it on. An
|
|
unknown machine commissions itself: it boots the live image, reports its
|
|
hardware, and appears as `ready`.
|
|
|
|
```sh
|
|
sudo rollcall list
|
|
sudo rollcall deploy MACHINE debian-13 -hostname node01 -boot -watch
|
|
```
|
|
|
|
`-boot` power-cycles the machine through its BMC. Set the BMC first if
|
|
commissioning did not find it (`rollcall bmc MACHINE ...`, or the machine's page
|
|
in the web UI). Without a BMC, power the machine on by hand after `deploy`.
|
|
|
|
A deployed machine gets the address from the static range, the domain, a route,
|
|
DNS through the rack controller, root SSH keys and trust in the Rollcall CA.
|
|
|
|
## Admin access from your laptop
|
|
|
|
```sh
|
|
# on the laptop, once, with the token (sudo jq -r .admin_token /etc/rollcall/config.json on the region)
|
|
ROLLCALL_URL=https://REGION ROLLCALL_TOKEN=... ROLLCALL_CA=ca.pem rollcall admin-cert request mylaptop
|
|
```
|
|
|
|
The key is made on the laptop; only a certificate request travels. The CLI then
|
|
works without the token. The `.p12` it writes goes into the browser's keychain,
|
|
and the web UI signs in by itself. Trust the CA (`/ca.pem`) in the browser once.
|
|
|
|
## More segments: rack controllers
|
|
|
|
A provisioning VLAN the region has no interface on gets its own rack controller:
|
|
|
|
```sh
|
|
# on the region
|
|
sudo rollcall controller join-code -name hall-b -addr REGION:443
|
|
# on the rack host (Debian 13, an interface on that VLAN), from the bundle
|
|
sudo ./rollcall rack join CODE -url http://RACK-ADDRESS-ON-THAT-VLAN
|
|
sudo sh deploy/install-rack.sh
|
|
```
|
|
|
|
Rack controllers talk to the region over mutual TLS and update themselves to the
|
|
region's version.
|
|
|
|
## Integrations
|
|
|
|
- **NetBox (or Infoblox) as the address authority.** `ipam` in `config.json`: see README, External IPAM.
|
|
- **Certificates for machines.** Delegate a name-constrained intermediate to an ACME
|
|
server: see README, Certificate authority.
|
|
|
|
## Upgrades
|
|
|
|
Replace the binary on the region (or let CD do it) and restart `rollcall`. Rack
|
|
controllers follow by themselves. `rollcall doctor` afterwards.
|