161 lines
7.0 KiB
Markdown
161 lines
7.0 KiB
Markdown
# Installing Rollcall at a new site
|
|
|
|
This follows a site from nothing to a deployed machine. It was written from a
|
|
clean run: a fresh Debian 13 VM became the region, discovered its interfaces,
|
|
was given a provisioning network, and an empty machine commissioned itself and
|
|
was deployed with Debian 13.
|
|
|
|
The network does not have to be decided before you start. Day 0 installs the
|
|
region on its management address; the provisioning network is added when it
|
|
exists.
|
|
|
|
## What you need
|
|
|
|
| | |
|
|
|---|---|
|
|
| Region host | Debian 13, 2 CPUs, 4 GB RAM, 30 GB disk (plus ~1 GB per OS image), root |
|
|
| Network access | a default route; the internet or a mirror for packages and cloud images |
|
|
| Later: provisioning network | an interface of this host (or of a rack controller) on the VLAN where machines PXE-boot |
|
|
| Later: BMC network | the rack controller must reach the machines' BMCs (IPMI, Redfish) |
|
|
| Ports | DHCP 67/udp and TFTP 69/udp on the provisioning network; HTTP 80 and HTTPS 443 for machines (agents use 443); HTTPS 443 for people and rack controllers |
|
|
|
|
The release bundle `rollcall-<version>-linux-amd64.tar.gz` holds the binary,
|
|
`deploy/` and `agent/`. From a checkout, `deploy/bootstrap.sh` builds the binary
|
|
itself when Go is installed.
|
|
|
|
## Day 0: install
|
|
|
|
```sh
|
|
tar xzf rollcall-v0.2.1-linux-amd64.tar.gz && cd rollcall-v0.2.1-linux-amd64
|
|
sudo ./deploy/bootstrap.sh --image-url https://gitea.example/OWNER/rollcall/releases/download/v0.2.1
|
|
```
|
|
|
|
- **Packages, database, TFTP.** PostgreSQL with its own role and a random password,
|
|
dnsmasq, the BIOS and UEFI syslinux files, lldpd.
|
|
- **Configuration.** `/etc/rollcall/config.json` with generated tokens. The admin
|
|
API only answers over HTTPS from other hosts (`admin_http: loopback`).
|
|
- **Commissioning image.** Downloaded and checked against `SHA256SUMS`. For a
|
|
private repository set `IMAGE_TOKEN`. Without a release to download from,
|
|
`--image-url http://EXISTING-REGION/rollcall` copies it from another region,
|
|
and `--build-image` builds it here with live-build (about 20 minutes).
|
|
- **SSH keys.** The keys of the user running sudo go to root on deployed
|
|
machines; add more with `--ssh-key KEY_OR_FILE`.
|
|
|
|
At the end the region answers on its management address: the web UI at
|
|
`https://ADDRESS`, no DHCP, no subnets. Check it:
|
|
|
|
```sh
|
|
sudo rollcall doctor
|
|
```
|
|
|
|
On day 0 the only warning should be that the host serves no DHCP yet.
|
|
|
|
## Find the provisioning network
|
|
|
|
```sh
|
|
sudo rollcall net discover
|
|
```
|
|
|
|
```
|
|
INTERFACE KIND LINK SPEED ADDRESSES SWITCH / PORT / VLAN DHCP SERVERS NEIGH
|
|
eth0* physical up - 10.0.0.116/24 - 10.0.0.1 4
|
|
eth1 physical up - 10.214.0.1/24 - none 0
|
|
|
|
eth0 uplink: management network with its own DHCP (10.0.0.1); keep it for the region's access
|
|
eth1 candidate provisioning network 10.214.0.0/24: bootstrap.sh --iface eth1
|
|
```
|
|
|
|
For each interface: link and speed, addresses and the subnet they imply, the
|
|
switch port and VLAN that LLDP reports, and DHCP servers that answer a probe.
|
|
The last column matters most: a DHCP server already on the provisioning VLAN
|
|
competes with Rollcall for the machines' PXE boot.
|
|
|
|
| What discover shows | What to do |
|
|
|---|---|
|
|
| An interface with an address and no DHCP | the provisioning network: next step |
|
|
| Link up, no address, LLDP shows the VLAN | give the interface its address on that VLAN first |
|
|
| Another DHCP server on the provisioning VLAN | keep it: `--proxy-dhcp` makes this host answer only PXE, the site's server keeps the addresses (or the network team turns it off for that VLAN) |
|
|
| No link | cable it |
|
|
|
|
## Add the provisioning network
|
|
|
|
```sh
|
|
sudo ./deploy/bootstrap.sh --iface eth1 --os debian-13
|
|
```
|
|
|
|
- **Server address.** The host's address on `eth1` becomes the server address
|
|
machines use. The management address stays in the certificate, so the web UI
|
|
keeps working there.
|
|
- **Subnet.** Modelled with ranges: reserved `.1-.20`, static `.50-.99` (where
|
|
deployed machines get their addresses), dynamic `.100-.200` (PXE and commissioning).
|
|
- **DHCP and PXE** on `eth1`, unless `--no-dhcp`.
|
|
- **Routing.** Without `--gateway`, this host routes for the machines: forwarding
|
|
plus masquerade out of the uplink. Give `--gateway IP` when the network has a router.
|
|
- **First OS image.** `--os debian-13` imports it now; a nightly timer keeps the
|
|
catalog current.
|
|
|
|
The same can be done in the web UI (Network, New subnet) or the CLI
|
|
(`rollcall subnet add`, `range add`, `vlan add ... -dhcp`), or imported from
|
|
NetBox (`rollcall ipam import`). Several provisioning VLANs, bonds and VLAN
|
|
layouts per machine are described in the README, under Network.
|
|
|
|
`rollcall doctor` should now report no failures. It also sends a DHCP
|
|
DISCOVER on the provisioning interface and fails if any other server answers.
|
|
|
|
## First machines
|
|
|
|
Set a machine to PXE-boot on the provisioning network and power it on. An
|
|
unknown machine commissions itself: it boots the live image, reports its
|
|
hardware, and appears as `ready`.
|
|
|
|
```sh
|
|
sudo rollcall list
|
|
sudo rollcall deploy MACHINE debian-13 -hostname node01 -boot -watch
|
|
```
|
|
|
|
`-boot` power-cycles the machine through its BMC. Set the BMC first if
|
|
commissioning did not find it (`rollcall bmc MACHINE ...`, or the machine's page
|
|
in the web UI). Without a BMC, power the machine on by hand after `deploy`.
|
|
|
|
A deployed machine gets the address from the static range, the domain, a route,
|
|
DNS through the rack controller, root SSH keys and trust in the Rollcall CA.
|
|
|
|
## Admin access from your laptop
|
|
|
|
```sh
|
|
# on the laptop, once, with the token (sudo jq -r .admin_token /etc/rollcall/config.json on the region)
|
|
ROLLCALL_URL=https://REGION ROLLCALL_TOKEN=... ROLLCALL_CA=ca.pem rollcall admin-cert request mylaptop
|
|
```
|
|
|
|
The key is made on the laptop; only a certificate request travels. The CLI then
|
|
works without the token. The `.p12` it writes goes into the browser's keychain,
|
|
and the web UI signs in by itself. Trust the CA (`/ca.pem`) in the browser once.
|
|
|
|
## More segments: rack controllers
|
|
|
|
A provisioning VLAN the region has no interface on gets its own rack controller:
|
|
|
|
```sh
|
|
# on the region
|
|
sudo rollcall controller join-code -name hall-b -addr REGION:443
|
|
# on the rack host (Debian 13, an interface on that VLAN), from the bundle
|
|
sudo ./rollcall rack join CODE -url http://RACK-ADDRESS-ON-THAT-VLAN
|
|
sudo sh deploy/install-rack.sh
|
|
```
|
|
|
|
Rack controllers talk to the region over mutual TLS and update themselves to the
|
|
region's version.
|
|
|
|
## Integrations
|
|
|
|
- **NetBox (or Infoblox) as the address authority.** `ipam` in `config.json`: see README, External IPAM.
|
|
- **DNS names without an IPAM authority.** `dns` in `config.json` (PowerDNS API or RFC 2136
|
|
with TSIG): see README, DNS provider.
|
|
- **Certificates for machines.** Delegate a name-constrained intermediate to an ACME
|
|
server: see README, Certificate authority.
|
|
|
|
## Upgrades
|
|
|
|
Replace the binary on the region (or let CD do it) and restart `rollcall`. Rack
|
|
controllers follow by themselves. `rollcall doctor` afterwards.
|