Region as certificate authority, BMC power drivers, IPv6
CA (llmgw-style) - region creates its CA (ECDSA P-256) and serves API/UI/images over HTTPS on tls_listen with a certificate for every name/address it has (reissued on change or a month before expiry); admin_http policy (allow|loopback|deny) for the admin API over plain HTTP; /ca.pem - rack controllers enroll with one-time join codes (CA fingerprint + secret + expiry + address): rollcall controller join-code, rollcall rack join; mutual TLS from then on, identity from the certificate (CN rack:NAME, serial on record), removal revokes, renewal a month before expiry; the shared rack_secret stays as a legacy fallback - clients: CA from the server config, ROLLCALL_CA, or trusted on first connect in the console after showing the fingerprint (kept in client.json); deployed machines trust it via cloud-init ca_certs (allowed on RHEL-family images), overlays get .CA; rollcall ca show|issue Power drivers - ipmi (cipher/privilege/interface), redfish (system discovery, ResetType from AllowableValues or ActionInfo, boot override with If-Match), idrac, ilo, supermicro, xcc, openbmc, proxmox (API token), manual; executed on the rack controller; Redfish auto-detection after commissioning; operator settings survive recommissioning; rollcall bmc, BMC editing in web UI and console - lab: pve_redfish.py Redfish emulator per Proxmox VM (iDRAC, iLO, generic profiles) IPv6 - IPv6 subnets and IPAM (netip), dual-stack default layout (all subnets of the boot VLAN), per-family default routes, accept-ra off where addresses are planned, dnsmasq RA/DHCPv6 with router lifetime 0 unless the rack is the gateway, AAAA host records from IPAM, IPv6 listen addresses and URLs, rack interfaces and region certificate with IPv6, agent reports IPv6 Tests for CA/join/pinning/CSR signing, Redfish (iDRAC, iLO fallbacks, probe), Proxmox, manual, IPv6 candidates, dual-stack plans and dnsmasq IPv6. Verified in the lab: rack2 enrolled and running on mutual TLS only, revocation and single-use codes; iLO auto-detected through rack2, iDRAC and Proxmox drivers, wrong password not saved; Debian 13 (bond+VLAN), Rocky 9 and AlmaLinux 9 dual-stack with Redfish-driven deployments; CA trusted on Debian and AlmaLinux; admin CLI over HTTPS on IPv4 and IPv6. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
736a7e38a3
commit
c8465c235a
29 files changed
+3156
-241
No files matched your search
@@ -380,14 +380,11 @@ controller (`rack_name`, default its hostname), which is all a single-server
|
||||
installation needs. Remote ones:
|
||||
|
||||
```sh
|
||||
# region: config.json gets "rack_secret": "<rollcall keygen>"
|
||||
# rack host (Debian/Ubuntu), from deploy/ with the binary next to it:
|
||||
cat > /etc/rollcall/rack.json <<EOF
|
||||
{"name": "hall-b", "region": "http://10.0.0.234", "secret": "<rack_secret>",
|
||||
"url": "http://10.213.78.1", "listen": "10.213.78.1:80"}
|
||||
EOF
|
||||
sh install-rack.sh # dnsmasq, syslinux, cache dirs, rollcall-rack.service
|
||||
rollcall controllers # on the region: it shows up online
|
||||
rollcall controller join-code -name hall-b -addr 10.0.0.234:443 # on the region: one word, one use, 60 min
|
||||
# on the rack host (Debian/Ubuntu), from deploy/ with the binary next to it:
|
||||
./rollcall rack join rollcall-join1-... -url http://10.213.78.1 # writes /etc/rollcall/rack.json + certificate
|
||||
sh install-rack.sh # dnsmasq, syslinux, cache dirs, rollcall-rack.service
|
||||
rollcall controllers # on the region: it shows up online
|
||||
rollcall vlan add hall-b 0 -dhcp -rack hall-b
|
||||
```
|
||||
|
||||
@@ -399,9 +396,89 @@ hand-written DHCP configuration for the same interface is the usual cause:
|
||||
|
||||
The lab runs both: `pxe-server` (embedded, 10.213.77.0/24) and `pxe-rack2`
|
||||
(VM 213, remote, 10.213.78.0/24 on SDN vnet `pxenet2`), with node002 behind
|
||||
pxe-rack2 and its virtual BMC on pxe-rack2. Rack ↔ region traffic is plain
|
||||
HTTP in the lab; BMC passwords travel on it for queued commands, so put the
|
||||
region behind TLS on real networks.
|
||||
pxe-rack2 and its virtual BMC on pxe-rack2. Rack ↔ region traffic is mutual
|
||||
TLS (next section), so the BMC passwords of queued commands are encrypted.
|
||||
|
||||
## Certificate authority and TLS
|
||||
|
||||
The region is the installation's certificate authority, as the llmgw control
|
||||
plane is: `ca_dir` (default `/etc/rollcall/ca`) holds `ca.pem` and
|
||||
`ca-key.pem` (ECDSA P-256, ten years), created on first start.
|
||||
|
||||
* **HTTPS.** The region serves API, UI and images on `tls_listen` (default
|
||||
`:443`) with a certificate for every name and address it has plus
|
||||
`tls_names`; it is reissued when they change or a month before it ends.
|
||||
`admin_http` decides what plain HTTP may still do with the admin API:
|
||||
`allow` (default, for compatibility), `loopback` (the CLI on the region
|
||||
host) or `deny`. Nodes keep booting over HTTP: firmware and the live image
|
||||
cannot check a certificate before they have one.
|
||||
* **Rack controllers** join with a code from `rollcall controller join-code`:
|
||||
one word carrying the first half of the CA's SHA-256, a one-time secret, an
|
||||
expiry and the region's address. `rollcall rack join` checks the region
|
||||
against the fingerprint, sends a CSR with the secret and gets a
|
||||
client+server certificate (`rack:<name>`, a year). From then on every rack
|
||||
call is mutual TLS and the region identifies the rack by its certificate;
|
||||
`rollcall controller rm` revokes it at once. Racks renew a month before
|
||||
expiry. The old `rack_secret` is still accepted when configured.
|
||||
* **Clients.** On the region host the CLI reads the CA from `ca_dir`;
|
||||
elsewhere `ROLLCALL_CA=ca.pem`, or the console shows the CA fingerprint on
|
||||
first connect and keeps the CA in `~/.config/rollcall/client.json` once you
|
||||
confirm it (`rollcall ca show` prints it on the region). `/ca.pem` serves
|
||||
the certificate; the web login links to it.
|
||||
* **Machines** trust it: deployments add it with cloud-init `ca_certs`
|
||||
(allowed on RHEL-family images, where cloud-init marks the module
|
||||
unverified), stateless overlays get it as `.CA`. `rollcall ca issue NAME
|
||||
-host h1,h2 [-client]` issues leaves for anything else (Slurm REST, a
|
||||
registry, a web front end).
|
||||
|
||||
## Power drivers
|
||||
|
||||
Every machine's BMC record names a driver; commands run on the rack controller
|
||||
that reaches the BMC network.
|
||||
|
||||
| Driver | For |
|
||||
|---|---|
|
||||
| `ipmi` | IPMI 2.0 over LAN (ipmitool lanplus), any vendor; settings `cipher` (17 on newer BMCs), `privilege`, `interface` |
|
||||
| `redfish` | DMTF Redfish, any vendor: system discovered, ResetType chosen from what the BMC allows, boot override once (with If-Match where the BMC wants it) |
|
||||
| `idrac` | Dell iDRAC 8/9 (Redfish, `System.Embedded.1`) |
|
||||
| `ilo` | HPE iLO 4 (2.30+), 5, 6 (Redfish; no PowerCycle, so ForceRestart; PushPowerButton for soft off on older firmware) |
|
||||
| `supermicro`, `xcc`, `openbmc` | Supermicro, Lenovo XClarity, OpenBMC (Redfish) |
|
||||
| `proxmox` | Proxmox VE VMs: API token (user = token id, password = secret), settings `node`, `vmid` |
|
||||
| `manual` | no power control; actions say so |
|
||||
|
||||
After commissioning, a BMC still on IPMI is asked whether it speaks Redfish
|
||||
with the same account; if it does, the driver for its vendor takes over
|
||||
(event `bmc driver`). An operator's choice is never overridden.
|
||||
|
||||
```sh
|
||||
rollcall bmc drivers
|
||||
rollcall bmc m-1234 -driver idrac -address 10.0.5.21 -user root -password - -verify < pw
|
||||
rollcall bmc m-1234 -driver redfish -set system=/redfish/v1/Systems/1 -set verify_tls=true
|
||||
rollcall bmc m-1234 -probe # detect Redfish again
|
||||
```
|
||||
|
||||
The lab has `lab/pve-vbmc/pve_redfish.py`, a Redfish service per Proxmox VM
|
||||
with iDRAC, iLO and generic profiles, next to the IPMI emulator.
|
||||
|
||||
## IPv6
|
||||
|
||||
Subnets can be IPv6 (`rollcall subnet add fd00:50::/64 ...`). IPAM hands out
|
||||
addresses from static ranges (walked from their start), deployments get them
|
||||
in `network-config` with a separate default route per family, and the default
|
||||
layout links every subnet of the boot NIC's VLAN, so an IPv6 prefix next to
|
||||
the IPv4 one makes machines dual-stack. Interfaces with planned addresses
|
||||
ignore router advertisements. dnsmasq serves router advertisements and
|
||||
DHCPv6 on IPv6 subnets with dhcp on; the rack offers itself as default router
|
||||
only when it is the subnet's gateway. Host files carry AAAA records, rack
|
||||
controllers report their IPv6 addresses, the region certificate covers them,
|
||||
and region, racks and clients work over IPv6 (`https://[fd00::1]`).
|
||||
|
||||
PXE stays IPv4: syslinux cannot netboot over IPv6, so the provisioning VLAN
|
||||
needs an IPv4 subnet for commissioning and deployment.
|
||||
|
||||
Verified in the lab: Debian 13 (with a bond and VLAN), Rocky 9 and AlmaLinux 9
|
||||
dual-stack on two segments, AAAA lookups through dnsmasq, the admin CLI over
|
||||
`https://[fdc3:...]`.
|
||||
|
||||
## Lab
|
||||
|
||||
|
||||
Reference in new issue
Block a user