Proxy DHCP for networks whose DHCP server stays in charge
A VLAN in proxy mode leaves addresses to the site's DHCP server; the rack only answers PXE (dnsmasq proxy range + pxe-service), machines keep DHCP unless a static range is set aside. CLI -proxy-dhcp, web UI mode select, bootstrap --proxy-dhcp, doctor/discover aware. dnsmasq 2.91 (and 2.93) loops forever in its proxy path when the proxy range carries a set: tag: option_filter appends a request's tags to the context's persistent list, so the next request's stack interface tag points at itself (found with gdb on the lab rack). Proxy ranges are untagged and PXE services limited by interface; dhcp-boot/dhcp-match only for served subnets; no dhcp-host lines for proxied ones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed
1 parent
56eae695d9
commit
7a007ef048
14 files changed
+266
-37
No files matched your search
@@ -6,6 +6,16 @@ publish a tag that has no section here.
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Proxy DHCP: a VLAN in `proxy` mode (`rollcall vlan add ... -proxy-dhcp`, web UI, `bootstrap.sh
|
||||
--proxy-dhcp`) leaves addresses to the site's DHCP server and only answers PXE; machines keep DHCP
|
||||
unless a static range is set aside; doctor and discover know the mode.
|
||||
|
||||
### Fixed
|
||||
- dnsmasq busy loop in proxy mode (a dnsmasq bug with tagged proxy ranges, found with gdb): proxy
|
||||
ranges are untagged, PXE services limited by interface; `dhcp-boot`/`dhcp-match` only for served
|
||||
subnets, no `dhcp-host` lines for proxied ones.
|
||||
|
||||
## [0.2.1] - 2026-10-28
|
||||
|
||||
### Fixed
|
||||
|
||||
+1
-1
@@ -74,7 +74,7 @@ competes with Rollcall for the machines' PXE boot.
|
||||
|---|---|
|
||||
| An interface with an address and no DHCP | the provisioning network: next step |
|
||||
| Link up, no address, LLDP shows the VLAN | give the interface its address on that VLAN first |
|
||||
| Another DHCP server on the provisioning VLAN | the network team turns it off for that VLAN, or keeps it and points it at this host (next-server, or a DHCP relay), then `--no-dhcp` |
|
||||
| Another DHCP server on the provisioning VLAN | keep it: `--proxy-dhcp` makes this host answer only PXE, the site's server keeps the addresses (or the network team turns it off for that VLAN) |
|
||||
| No link | cable it |
|
||||
|
||||
## Add the provisioning network
|
||||
|
||||
@@ -401,6 +401,31 @@ The lab runs both: `pxe-server` (embedded, 10.213.77.0/24) and `pxe-rack2`
|
||||
pxe-rack2 and its virtual BMC on pxe-rack2. Rack ↔ region traffic is mutual
|
||||
TLS (next section), so the BMC passwords of queued commands are encrypted.
|
||||
|
||||
### DHCP already on the network: proxy DHCP
|
||||
|
||||
Where the site's DHCP server keeps handing out addresses (a corporate DHCP,
|
||||
Infoblox, a router), a VLAN can be served in **proxy** mode: the rack
|
||||
controller does not hand out addresses, it only answers PXE clients with where
|
||||
to boot from (dnsmasq proxy DHCP), and TFTP and HTTP stay on the rack.
|
||||
|
||||
```
|
||||
rollcall vlan add FABRIC VID -proxy-dhcp -rack RACK # web UI: subnet, DHCP / PXE: proxy
|
||||
deploy/bootstrap.sh --iface eth1 --proxy-dhcp # the same at install time
|
||||
```
|
||||
|
||||
- The subnet needs no ranges. Without a static range, deployed machines keep
|
||||
getting their addresses from the site's server (`dhcp` links); set a static
|
||||
range aside, outside the site's pool, to give them fixed ones.
|
||||
- No `dhcp-host` lines are written for proxy subnets, and the router and DNS
|
||||
options come from the site's server.
|
||||
- `rollcall doctor` expects the foreign DHCP server here, and fails if none answers.
|
||||
- dnsmasq (2.91, still in 2.93) loops forever in its proxy code when a proxy
|
||||
range carries a `set:` tag: Rollcall's generated configuration leaves the tag
|
||||
out and limits the PXE services by interface instead.
|
||||
|
||||
Verified in the lab with a separate DHCP server on the provisioning network:
|
||||
commissioning and three deployments in a row through the proxy.
|
||||
|
||||
## Certificate authority and TLS
|
||||
|
||||
The region is the installation's certificate authority, as the llmgw control
|
||||
|
||||
@@ -195,7 +195,7 @@ func discoSuggest(d discoIface) string {
|
||||
case v4 == "":
|
||||
return "link up, no IPv4 address" + vlan + ": give it its address on the provisioning network, then bootstrap.sh --iface " + d.Name
|
||||
case foreign > 0:
|
||||
return v4 + " already has DHCP (" + strings.Join(d.DHCP, ",") + "): either the network team turns it off for this VLAN, or keep it and point it at this host (next-server " + strings.Split(d.Addrs[0], "/")[0] + ", or a DHCP relay); then bootstrap.sh --iface " + d.Name + " --no-dhcp"
|
||||
return v4 + " already has DHCP (" + strings.Join(d.DHCP, ",") + "): keep it and answer only PXE from here - bootstrap.sh --iface " + d.Name + " --proxy-dhcp - or the network team turns it off for this VLAN"
|
||||
default:
|
||||
return "candidate provisioning network " + v4 + vlan + ": bootstrap.sh --iface " + d.Name + " (or Network -> New subnet in the web UI)"
|
||||
}
|
||||
|
||||
+17
-2
@@ -212,7 +212,18 @@ func doctorCmd(args []string) error {
|
||||
}
|
||||
}
|
||||
|
||||
// other DHCP servers on the provisioning network
|
||||
// other DHCP servers on the provisioning network (expected when this rack proxies)
|
||||
proxying := false
|
||||
if c, err := newClient(); err == nil {
|
||||
var subs []Subnet
|
||||
if c.do("GET", "/api/v1/subnets", nil, &subs) == nil {
|
||||
for _, s := range subs {
|
||||
if s.DHCPMode == "proxy" && (s.PrimaryRack == cfg.RackName || s.SecondaryRack == cfg.RackName) {
|
||||
proxying = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if iface != "" && !*noProbe && !ports["udp/67"] {
|
||||
d.ok("dhcp probe", "skipped: this host serves no DHCP yet")
|
||||
} else if iface != "" && !*noProbe {
|
||||
@@ -228,9 +239,13 @@ func doctorCmd(args []string) error {
|
||||
}
|
||||
}
|
||||
switch {
|
||||
case proxying && len(foreign) > 0:
|
||||
d.ok("dhcp probe", "the site's DHCP server "+strings.Join(foreign, ", ")+" hands out addresses; this host answers PXE (proxy)")
|
||||
case proxying:
|
||||
d.fail("dhcp probe", "proxy DHCP, but no DHCP server answers on "+iface, "machines would get no address: the site's DHCP server must serve this VLAN (or a relay must reach it)")
|
||||
case len(foreign) > 0:
|
||||
d.fail("dhcp probe", "another DHCP server answers on "+iface+": "+strings.Join(foreign, ", "),
|
||||
"machines may PXE-boot from it; switch it off for this VLAN, or use --no-dhcp and let it hand out next-server "+srvHost)
|
||||
"machines may PXE-boot from it: switch it off for this VLAN, or keep it and proxy (bootstrap.sh --iface IFACE --proxy-dhcp, or DHCP mode proxy in the web UI)")
|
||||
case len(servers) == 0:
|
||||
d.warn("dhcp probe", "no DHCP server answered on "+iface, "is a VLAN with dhcp assigned to this rack? (rollcall vlan add ... -dhcp)")
|
||||
default:
|
||||
|
||||
@@ -60,7 +60,7 @@ func usage() {
|
||||
rollcall location racks | rack NAME -building B -room R [-height 42]
|
||||
rollcall list [-tag T] [-building B] [-room R] [-rack K]
|
||||
rollcall fabric list | add NAME [-kind ethernet|infiniband] | rm NAME
|
||||
rollcall vlan list | add FABRIC VID [-dhcp] [-rack R] [-secondary R] [-mtu N] [-name N] | rm FABRIC VID
|
||||
rollcall vlan list | add FABRIC VID [-dhcp | -proxy-dhcp] [-rack R] [-secondary R] [-mtu N] [-name N] | rm FABRIC VID
|
||||
rollcall subnet list | show CIDR | add CIDR -fabric F [-vid N] [-purpose P] [-gateway G] [-dns a,b] [-domain D] | rm CIDR [-force]
|
||||
rollcall range add CIDR dynamic|reserved|static START END | rm ID
|
||||
rollcall ip list [-subnet CIDR] | reserve IP [-name N] | release IP
|
||||
|
||||
+15
-4
@@ -71,6 +71,12 @@ func (a *App) handleVLANs(w http.ResponseWriter, r *http.Request) {
|
||||
fail(w, 400, errors.New("fabric and vid (0 untagged, 1-4094, or an IB pkey) are required"))
|
||||
return
|
||||
}
|
||||
if v.DHCPMode == "proxy" {
|
||||
v.DHCP = true
|
||||
} else if v.DHCPMode != "" && v.DHCPMode != "serve" {
|
||||
fail(w, 400, fmt.Errorf("dhcp_mode %q: serve or proxy", v.DHCPMode))
|
||||
return
|
||||
}
|
||||
if v.DHCP && v.PrimaryRack == "" {
|
||||
v.PrimaryRack = a.cfg.RackName
|
||||
}
|
||||
@@ -511,9 +517,9 @@ func (c *client) netCmd(cmd string, args []string) error {
|
||||
if err := c.do("GET", "/api/v1/vlans", nil, &vs); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("%-14s %5s %-14s %5s %-5s %-14s %s\n", "FABRIC", "VID", "NAME", "MTU", "DHCP", "RACK", "SECONDARY")
|
||||
fmt.Printf("%-14s %5s %-14s %5s %-6s %-14s %s\n", "FABRIC", "VID", "NAME", "MTU", "DHCP", "RACK", "SECONDARY")
|
||||
for _, v := range vs {
|
||||
fmt.Printf("%-14s %5d %-14s %5d %-5v %-14s %s\n", v.Fabric, v.VID, v.Name, v.MTU, v.DHCP, v.PrimaryRack, v.SecondaryRack)
|
||||
fmt.Printf("%-14s %5d %-14s %5d %-6s %-14s %s\n", v.Fabric, v.VID, v.Name, v.MTU, orDash(v.DHCPMode), v.PrimaryRack, v.SecondaryRack)
|
||||
}
|
||||
case "add", "set":
|
||||
need(3)
|
||||
@@ -521,15 +527,20 @@ func (c *client) netCmd(cmd string, args []string) error {
|
||||
name := fs.String("name", "", "name")
|
||||
mtu := fs.Int("mtu", 1500, "MTU")
|
||||
dhcp := fs.Bool("dhcp", false, "serve DHCP/PXE on this VLAN")
|
||||
proxy := fs.Bool("proxy-dhcp", false, "another DHCP server hands out addresses here; only answer PXE (proxy DHCP)")
|
||||
rack := fs.String("rack", "", "rack controller serving it (default: the region's embedded one)")
|
||||
sec := fs.String("secondary", "", "second rack controller (the dynamic range is split)")
|
||||
_ = fs.Parse(args[3:])
|
||||
vid, _ := strconv.Atoi(args[2])
|
||||
var v VLAN
|
||||
if err := c.do("POST", "/api/v1/vlans", VLAN{Fabric: args[1], VID: vid, Name: *name, MTU: *mtu, DHCP: *dhcp, PrimaryRack: *rack, SecondaryRack: *sec}, &v); err != nil {
|
||||
mode := ""
|
||||
if *proxy {
|
||||
mode = "proxy"
|
||||
}
|
||||
if err := c.do("POST", "/api/v1/vlans", VLAN{Fabric: args[1], VID: vid, Name: *name, MTU: *mtu, DHCP: *dhcp || *proxy, DHCPMode: mode, PrimaryRack: *rack, SecondaryRack: *sec}, &v); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("vlan %s/%d: dhcp=%v rack=%s\n", v.Fabric, v.VID, v.DHCP, v.PrimaryRack)
|
||||
fmt.Printf("vlan %s/%d: dhcp=%s rack=%s\n", v.Fabric, v.VID, orDash(v.DHCPMode), v.PrimaryRack)
|
||||
case "rm":
|
||||
need(3)
|
||||
return c.do("DELETE", "/api/v1/vlans/"+args[1]+"/"+args[2], nil, nil)
|
||||
|
||||
+94
-14
@@ -33,7 +33,8 @@ type VLAN struct {
|
||||
VID int `json:"vid"` // 0 = untagged; an InfiniBand partition key on IB fabrics
|
||||
Name string `json:"name"`
|
||||
MTU int `json:"mtu"`
|
||||
DHCP bool `json:"dhcp"` // rollcall serves DHCP/PXE on it
|
||||
DHCP bool `json:"dhcp"` // a rack controller answers DHCP/PXE on it
|
||||
DHCPMode string `json:"dhcp_mode"` // serve (addresses + PXE) | proxy (PXE only; another server hands out addresses)
|
||||
PrimaryRack string `json:"primary_rack"` // rack controller that serves it
|
||||
SecondaryRack string `json:"secondary_rack"` // second one; the dynamic range is split between them
|
||||
}
|
||||
@@ -55,6 +56,7 @@ type Subnet struct {
|
||||
VID int `json:"vid"`
|
||||
MTU int `json:"mtu"`
|
||||
DHCP bool `json:"dhcp"`
|
||||
DHCPMode string `json:"dhcp_mode"` // serve | proxy (from the VLAN)
|
||||
PrimaryRack string `json:"primary_rack"`
|
||||
SecondaryRack string `json:"secondary_rack"`
|
||||
Purpose string `json:"purpose"` // provisioning | mgmt | bmc | storage | ib | other
|
||||
@@ -140,7 +142,7 @@ func (s *Store) DeleteFabric(ctx context.Context, name string) error {
|
||||
}
|
||||
|
||||
func (s *Store) VLANs(ctx context.Context) ([]VLAN, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT id, fabric, vid, coalesce(name,''), mtu, dhcp, coalesce(primary_rack,''), coalesce(secondary_rack,'') FROM vlans ORDER BY fabric, vid`)
|
||||
rows, err := s.pool.Query(ctx, `SELECT id, fabric, vid, coalesce(name,''), mtu, dhcp, dhcp_mode, coalesce(primary_rack,''), coalesce(secondary_rack,'') FROM vlans ORDER BY fabric, vid`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -148,23 +150,35 @@ func (s *Store) VLANs(ctx context.Context) ([]VLAN, error) {
|
||||
out := []VLAN{}
|
||||
for rows.Next() {
|
||||
var v VLAN
|
||||
if err := rows.Scan(&v.ID, &v.Fabric, &v.VID, &v.Name, &v.MTU, &v.DHCP, &v.PrimaryRack, &v.SecondaryRack); err != nil {
|
||||
if err := rows.Scan(&v.ID, &v.Fabric, &v.VID, &v.Name, &v.MTU, &v.DHCP, &v.DHCPMode, &v.PrimaryRack, &v.SecondaryRack); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
v.DHCPMode = dhcpMode(v.DHCP, v.DHCPMode)
|
||||
out = append(out, v)
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
||||
// dhcpMode normalises: no DHCP -> "", DHCP without a mode -> serve.
|
||||
func dhcpMode(dhcp bool, mode string) string {
|
||||
switch {
|
||||
case !dhcp:
|
||||
return ""
|
||||
case mode == "proxy":
|
||||
return "proxy"
|
||||
}
|
||||
return "serve"
|
||||
}
|
||||
|
||||
// PutVLAN upserts by (fabric, vid) and returns the id.
|
||||
func (s *Store) PutVLAN(ctx context.Context, v VLAN) (int64, error) {
|
||||
if v.MTU <= 0 {
|
||||
v.MTU = 1500
|
||||
}
|
||||
var id int64
|
||||
err := s.pool.QueryRow(ctx, `INSERT INTO vlans(fabric, vid, name, mtu, dhcp, primary_rack, secondary_rack) VALUES ($1,$2,$3,$4,$5,nullif($6,''),nullif($7,''))
|
||||
ON CONFLICT (fabric, vid) DO UPDATE SET name=$3, mtu=$4, dhcp=$5, primary_rack=nullif($6,''), secondary_rack=nullif($7,'') RETURNING id`,
|
||||
v.Fabric, v.VID, v.Name, v.MTU, v.DHCP, v.PrimaryRack, v.SecondaryRack).Scan(&id)
|
||||
err := s.pool.QueryRow(ctx, `INSERT INTO vlans(fabric, vid, name, mtu, dhcp, primary_rack, secondary_rack, dhcp_mode) VALUES ($1,$2,$3,$4,$5,nullif($6,''),nullif($7,''),$8)
|
||||
ON CONFLICT (fabric, vid) DO UPDATE SET name=$3, mtu=$4, dhcp=$5, primary_rack=nullif($6,''), secondary_rack=nullif($7,''), dhcp_mode=$8 RETURNING id`,
|
||||
v.Fabric, v.VID, v.Name, v.MTU, v.DHCP, v.PrimaryRack, v.SecondaryRack, dhcpMode(v.DHCP, v.DHCPMode)).Scan(&id)
|
||||
return id, err
|
||||
}
|
||||
|
||||
@@ -175,7 +189,7 @@ func (s *Store) DeleteVLAN(ctx context.Context, fabric string, vid int) error {
|
||||
|
||||
func (s *Store) Subnets(ctx context.Context) ([]Subnet, error) {
|
||||
rows, err := s.pool.Query(ctx, `SELECT s.cidr, coalesce(s.name,''), coalesce(s.vlan_id,0), coalesce(v.fabric,''), coalesce(v.vid,0), coalesce(v.mtu,1500),
|
||||
coalesce(v.dhcp,false), coalesce(v.primary_rack,''), coalesce(v.secondary_rack,''), s.purpose, coalesce(s.gateway,''), s.dns,
|
||||
coalesce(v.dhcp,false), coalesce(v.dhcp_mode,''), coalesce(v.primary_rack,''), coalesce(v.secondary_rack,''), s.purpose, coalesce(s.gateway,''), s.dns,
|
||||
coalesce(s.domain,''), coalesce(s.comment,''), (SELECT count(*) FROM ip_addresses a WHERE a.subnet=s.cidr)
|
||||
FROM subnets s LEFT JOIN vlans v ON v.id = s.vlan_id ORDER BY s.cidr`)
|
||||
if err != nil {
|
||||
@@ -186,11 +200,12 @@ func (s *Store) Subnets(ctx context.Context) ([]Subnet, error) {
|
||||
for rows.Next() {
|
||||
var sn Subnet
|
||||
var dns []byte
|
||||
if err := rows.Scan(&sn.CIDR, &sn.Name, &sn.VLANID, &sn.Fabric, &sn.VID, &sn.MTU, &sn.DHCP, &sn.PrimaryRack, &sn.SecondaryRack,
|
||||
if err := rows.Scan(&sn.CIDR, &sn.Name, &sn.VLANID, &sn.Fabric, &sn.VID, &sn.MTU, &sn.DHCP, &sn.DHCPMode, &sn.PrimaryRack, &sn.SecondaryRack,
|
||||
&sn.Purpose, &sn.Gateway, &dns, &sn.Domain, &sn.Comment, &sn.Used); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
_ = json.Unmarshal(dns, &sn.DNS)
|
||||
sn.DHCPMode = dhcpMode(sn.DHCP, sn.DHCPMode)
|
||||
if sn.DNS == nil {
|
||||
sn.DNS = []string{}
|
||||
}
|
||||
@@ -719,6 +734,11 @@ type NetPlan struct {
|
||||
// VLAN (an IPv6 prefix next to the IPv4 one makes the machine dual-stack).
|
||||
func defaultLayout(bootSubnet *Subnet, subnets []Subnet) *NetLayout {
|
||||
links := []LayoutLink{{Subnet: "boot", Mode: "auto", Gateway: true}}
|
||||
if bootSubnet != nil && bootSubnet.DHCPMode == "proxy" && !hasStaticRange(*bootSubnet) {
|
||||
// the site's DHCP server owns the addresses; without a static range set aside
|
||||
// for Rollcall, machines keep getting theirs from it
|
||||
links = []LayoutLink{{Subnet: "boot", Mode: "dhcp"}}
|
||||
}
|
||||
if bootSubnet == nil {
|
||||
links = []LayoutLink{{Subnet: "boot", Mode: "dhcp"}}
|
||||
} else if bootSubnet.VLANID != 0 {
|
||||
@@ -1201,11 +1221,6 @@ expand-hosts
|
||||
log-dhcp
|
||||
enable-tftp
|
||||
tftp-root=%s
|
||||
dhcp-match=set:efi64,option:client-arch,7
|
||||
dhcp-match=set:efi64,option:client-arch,9
|
||||
dhcp-match=set:bios,option:client-arch,0
|
||||
dhcp-boot=tag:bios,pxelinux.0
|
||||
dhcp-boot=tag:efi64,efi64/syslinux.efi
|
||||
dhcp-hostsfile=%s/hosts.dhcp
|
||||
addn-hosts=%s/hosts
|
||||
`
|
||||
@@ -1259,7 +1274,7 @@ func dnsmasqConfig(rc RackController, subnets []Subnet, dnsDir string) (string,
|
||||
shared := false
|
||||
domains := []string{}
|
||||
for _, sn := range served {
|
||||
shared = shared || sn.SecondaryRack != ""
|
||||
shared = shared || sn.SecondaryRack != "" || sn.DHCPMode == "proxy"
|
||||
if sn.Domain != "" && !hasTag(domains, sn.Domain) {
|
||||
domains = append(domains, sn.Domain)
|
||||
}
|
||||
@@ -1297,10 +1312,66 @@ func dnsmasqConfig(rc RackController, subnets []Subnet, dnsDir string) (string,
|
||||
for _, d := range domains {
|
||||
fmt.Fprintf(&b, "local=/%s/\n", d)
|
||||
}
|
||||
// architecture tags for the boot files of served subnets; left out when every
|
||||
// subnet is proxied (pxe-service picks by architecture itself, and these shared
|
||||
// tags are part of the cyclic tag list dnsmasq 2.91 builds in proxy mode)
|
||||
for _, sn := range served {
|
||||
if sn.DHCPMode != "proxy" && familyOf(sn.CIDR) == 4 {
|
||||
b.WriteString("dhcp-match=set:efi64,option:client-arch,7\ndhcp-match=set:efi64,option:client-arch,9\ndhcp-match=set:bios,option:client-arch,0\n")
|
||||
break
|
||||
}
|
||||
}
|
||||
pxeDone := map[string]bool{}
|
||||
for i, sn := range served {
|
||||
tag := fmt.Sprintf("s%d", i)
|
||||
v6 := familyOf(sn.CIDR) == 6
|
||||
if sn.DHCPMode == "proxy" {
|
||||
// another DHCP server hands out addresses (and router, DNS); this rack only
|
||||
// answers PXE clients with where to boot from. Proxy DHCP is IPv4 (PXE is).
|
||||
if v6 {
|
||||
continue
|
||||
}
|
||||
_, n, _ := net.ParseCIDR(sn.CIDR)
|
||||
fmt.Fprintf(&b, "\n# %s · %s vlan %d · %s · proxy DHCP (addresses from the site's DHCP server)\n", sn.CIDR, orDash(sn.Fabric), sn.VID, sn.Purpose)
|
||||
// No set: tag on a proxy range. dnsmasq (2.91, still in 2.93) appends a request's
|
||||
// tags to the context's persistent tag list in its proxy path without resetting it:
|
||||
// the next request's stack-allocated interface tag ends up pointing at itself and
|
||||
// dnsmasq loops forever (option_filter, rfc2131.c proxy branch). The PXE services
|
||||
// are limited by the interface tag instead, which is rebuilt for every request.
|
||||
fmt.Fprintf(&b, "dhcp-range=%s,proxy,%s\n", n.IP.String(), sn.netmask())
|
||||
var on []string
|
||||
for _, ri := range rc.Interfaces {
|
||||
for _, ad := range ri.Addrs {
|
||||
if sn.contains(ad) && !hasTag(on, ri.Name) {
|
||||
on = append(on, ri.Name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(on) == 0 {
|
||||
on = []string{""} // relayed: untagged
|
||||
}
|
||||
for _, i := range on {
|
||||
if pxeDone[i] {
|
||||
continue
|
||||
}
|
||||
pxeDone[i] = true
|
||||
t := ""
|
||||
if i != "" {
|
||||
t = "tag:" + i + ","
|
||||
}
|
||||
fmt.Fprintf(&b, "pxe-service=%sx86PC,\"Rollcall\",pxelinux\n", t)
|
||||
fmt.Fprintf(&b, "pxe-service=%sBC_EFI,\"Rollcall\",efi64/syslinux.efi\n", t)
|
||||
fmt.Fprintf(&b, "pxe-service=%sX86-64_EFI,\"Rollcall\",efi64/syslinux.efi\n", t)
|
||||
}
|
||||
continue
|
||||
}
|
||||
fmt.Fprintf(&b, "\n# %s · %s vlan %d · %s\n", sn.CIDR, orDash(sn.Fabric), sn.VID, sn.Purpose)
|
||||
if !v6 {
|
||||
// boot files per served subnet: dhcp-boot matching a proxy subnet's clients as
|
||||
// well as its pxe-service sends dnsmasq 2.91 into a busy loop (lab, 2026-10-29)
|
||||
fmt.Fprintf(&b, "dhcp-boot=tag:%s,tag:bios,pxelinux.0\n", tag)
|
||||
fmt.Fprintf(&b, "dhcp-boot=tag:%s,tag:efi64,efi64/syslinux.efi\n", tag)
|
||||
}
|
||||
if sn.Domain != "" {
|
||||
fmt.Fprintf(&b, "domain=%s,%s\n", sn.Domain, sn.CIDR)
|
||||
}
|
||||
@@ -1378,3 +1449,12 @@ func dnsmasqConfig(rc RackController, subnets []Subnet, dnsDir string) (string,
|
||||
}
|
||||
|
||||
var errSubnetUnknown = errors.New("subnet not found")
|
||||
|
||||
func hasStaticRange(sn Subnet) bool {
|
||||
for _, r := range sn.Ranges {
|
||||
if r.Kind == "static" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -289,3 +289,69 @@ func TestStaleJobs(t *testing.T) {
|
||||
t.Errorf("stale jobs: %+v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProxyDHCP(t *testing.T) {
|
||||
sn := labSubnet()
|
||||
sn.DHCPMode = "proxy"
|
||||
sn.Ranges = nil
|
||||
rc := RackController{Name: "r1", Interfaces: []RackIface{{Name: "eth1", Addrs: []string{"10.0.0.2/24"}}}}
|
||||
conf, ok := dnsmasqConfig(rc, []Subnet{sn}, "/d")
|
||||
if !ok {
|
||||
t.Fatal("no config")
|
||||
}
|
||||
for _, want := range []string{"interface=eth1", "dhcp-range=10.0.0.0,proxy,255.255.255.0", `pxe-service=tag:eth1,x86PC,"Rollcall",pxelinux`, `pxe-service=tag:eth1,X86-64_EFI,"Rollcall",efi64/syslinux.efi`} {
|
||||
if !strings.Contains(conf, want) {
|
||||
t.Errorf("proxy config lacks %s:\n%s", want, conf)
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"option:router", "option:dns-server", "dhcp-authoritative", ",12h", "set:s0,10.0.0.0,proxy"} {
|
||||
if strings.Contains(conf, bad) {
|
||||
t.Errorf("proxy config has %q (the site's server owns that):\n%s", bad, conf)
|
||||
}
|
||||
}
|
||||
// machines keep DHCP from the site's server unless a static range is set aside
|
||||
if l := defaultLayout(&sn, []Subnet{sn}); l.Interfaces[0].Links[0].Mode != "dhcp" {
|
||||
t.Errorf("proxy subnet without a static range: %+v", l.Interfaces[0].Links)
|
||||
}
|
||||
sn.Ranges = []IPRange{{Kind: "static", Start: "10.0.0.50", End: "10.0.0.99"}}
|
||||
if l := defaultLayout(&sn, []Subnet{sn}); l.Interfaces[0].Links[0].Mode != "auto" {
|
||||
t.Errorf("proxy subnet with a static range: %+v", l.Interfaces[0].Links)
|
||||
}
|
||||
if dhcpMode(true, "") != "serve" || dhcpMode(false, "proxy") != "" || dhcpMode(true, "proxy") != "proxy" {
|
||||
t.Error("dhcpMode")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDhcpBootOnlyForServedSubnets(t *testing.T) {
|
||||
serve := labSubnet()
|
||||
proxy := labSubnet()
|
||||
proxy.CIDR, proxy.Gateway, proxy.DHCPMode, proxy.Ranges = "10.9.0.0/24", "", "proxy", nil
|
||||
rc := RackController{Name: "r1", Interfaces: []RackIface{{Name: "eth1", Addrs: []string{"10.0.0.2/24"}}, {Name: "eth2", Addrs: []string{"10.9.0.2/24"}}}}
|
||||
conf, _ := dnsmasqConfig(rc, []Subnet{serve, proxy}, "/d")
|
||||
if !strings.Contains(conf, "dhcp-boot=tag:s0,tag:bios,pxelinux.0") || !strings.Contains(conf, "dhcp-boot=tag:s0,tag:efi64,efi64/syslinux.efi") {
|
||||
t.Errorf("served subnet lacks its boot files:\n%s", conf)
|
||||
}
|
||||
for _, l := range strings.Split(conf, "\n") {
|
||||
if strings.HasPrefix(l, "dhcp-boot=") && !strings.HasPrefix(l, "dhcp-boot=tag:s0,") {
|
||||
t.Errorf("dhcp-boot not limited to the served subnet: %s", l)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(conf, "pxe-service=tag:eth2,x86PC") || strings.Contains(conf, "set:s1,10.9.0.0,proxy") {
|
||||
t.Errorf("proxy subnet lacks pxe-service:\n%s", conf)
|
||||
}
|
||||
}
|
||||
|
||||
func TestProxyOnlyConfigHasNoArchTags(t *testing.T) {
|
||||
sn := labSubnet()
|
||||
sn.DHCPMode, sn.Ranges = "proxy", nil
|
||||
rc := RackController{Name: "r1", Interfaces: []RackIface{{Name: "eth1", Addrs: []string{"10.0.0.2/24"}}}}
|
||||
conf, _ := dnsmasqConfig(rc, []Subnet{sn}, "/d")
|
||||
if strings.Contains(conf, "dhcp-match=") || strings.Contains(conf, "dhcp-boot=") {
|
||||
t.Errorf("proxy-only config has arch tags or dhcp-boot:\n%s", conf)
|
||||
}
|
||||
serve := labSubnet()
|
||||
conf, _ = dnsmasqConfig(rc, []Subnet{serve}, "/d")
|
||||
if !strings.Contains(conf, "dhcp-match=set:bios,option:client-arch,0") {
|
||||
t.Errorf("served subnet lacks arch tags:\n%s", conf)
|
||||
}
|
||||
}
|
||||
@@ -507,6 +507,13 @@ func (a *App) hostsFiles(ctx context.Context, rack string, subnets []Subnet) (st
|
||||
fmt.Fprintf(&hosts, "%s %s\n", ip, name)
|
||||
}
|
||||
}
|
||||
// proxy subnets: the site's DHCP server hands out the addresses, so no dhcp-host
|
||||
// lines for them (they would do nothing, and the "known" tag they give a client
|
||||
// helps dnsmasq 2.91 build a cyclic tag list in its PXE proxy code: busy loop)
|
||||
proxied := func(ip string) bool {
|
||||
sn := subnetFor(subnets, ip)
|
||||
return sn != nil && sn.DHCPMode == "proxy"
|
||||
}
|
||||
sort.Slice(ms, func(i, j int) bool { return deref(ms[i].NodeName) < deref(ms[j].NodeName) })
|
||||
for _, m := range ms {
|
||||
if m.NodeName == nil || len(m.MACs) == 0 {
|
||||
@@ -522,7 +529,7 @@ func (a *App) hostsFiles(ctx context.Context, rack string, subnets []Subnet) (st
|
||||
}
|
||||
if mgmt != "" {
|
||||
line(mgmt, name)
|
||||
if mine && familyOf(mgmt+"/32") == 4 {
|
||||
if mine && familyOf(mgmt+"/32") == 4 && !proxied(mgmt) {
|
||||
fmt.Fprintf(&dhcp, "%s,%s,%s\n", m.MACs[0], mgmt, name)
|
||||
}
|
||||
}
|
||||
@@ -534,7 +541,7 @@ func (a *App) hostsFiles(ctx context.Context, rack string, subnets []Subnet) (st
|
||||
switch {
|
||||
case x.Iface == mgmtIface && mgmtIface != "":
|
||||
line(x.IP, name)
|
||||
if v6 && mine {
|
||||
if v6 && mine && !proxied(x.IP) {
|
||||
fmt.Fprintf(&dhcp, "%s,[%s],%s\n", m.MACs[0], x.IP, name)
|
||||
}
|
||||
case x.Iface != "":
|
||||
|
||||
@@ -166,6 +166,7 @@ CREATE TABLE IF NOT EXISTS rack_controllers (
|
||||
ALTER TABLE rack_controllers ADD COLUMN IF NOT EXISTS cert_serial text;
|
||||
ALTER TABLE rack_controllers ADD COLUMN IF NOT EXISTS cert_expires timestamptz;
|
||||
ALTER TABLE rack_controllers ADD COLUMN IF NOT EXISTS arch text;
|
||||
ALTER TABLE vlans ADD COLUMN IF NOT EXISTS dhcp_mode text NOT NULL DEFAULT '';
|
||||
CREATE TABLE IF NOT EXISTS join_tokens (
|
||||
hash text PRIMARY KEY,
|
||||
name text,
|
||||
|
||||
@@ -696,7 +696,7 @@ function pageNetwork() {
|
||||
const sizeTxt = s.size ? s.size : '∞';
|
||||
const ranges = (s.ranges || []).map(r => `<span class="pill tag" title="${esc(r.comment || '')}">${esc(r.kind)} ${esc(r.start)}–${esc(r.end.split('.').pop())} <a href="#" data-rmrange="${r.id}" title="remove">×</a></span>`).join(' ');
|
||||
return [`<b class="mono">${esc(s.cidr)}</b>${s.name ? `<br><span class="muted">${esc(s.name)}</span>` : ''}`, esc(s.purpose), `${esc(s.fabric || '—')} / ${s.vid || 'untagged'}${s.mtu && s.mtu !== 1500 ? ` · mtu ${s.mtu}` : ''}`,
|
||||
s.dhcp ? `<span class="ok">on</span> · ${esc(s.primary_rack)}${s.secondary_rack ? ' + ' + esc(s.secondary_rack) : ''}` : '<span class="muted">off</span>', `<span class="mono">${esc(s.gateway || '—')}</span>`,
|
||||
s.dhcp ? `<span class="ok">${s.dhcp_mode === 'proxy' ? 'proxy' : 'on'}</span> · ${esc(s.primary_rack)}${s.secondary_rack ? ' + ' + esc(s.secondary_rack) : ''}` : '<span class="muted">off</span>', `<span class="mono">${esc(s.gateway || '—')}</span>`,
|
||||
`<span class="mono">${esc((s.dns || []).join(', '))}</span>${s.domain ? `<br><span class="muted">${esc(s.domain)}</span>` : ''}`, `<div class="bar" title="${s.used}/${s.size}"><span style="width:${pct}%"></span></div><span class="muted">${s.used} / ${sizeTxt}</span>${s.family === 6 ? ' <span class="pill tag">IPv6</span>' : ''}`,
|
||||
ranges + ` <button class="btn sm" data-addrange="${esc(s.cidr)}">+ range</button>`, `<button class="btn sm" data-ips="${esc(s.cidr)}">addresses</button> <button class="btn sm" data-editsubnet="${esc(s.cidr)}">edit</button> <button class="btn sm danger" data-rmsubnet="${esc(s.cidr)}">delete</button>`];
|
||||
}));
|
||||
@@ -723,7 +723,7 @@ async function editSubnet(s) {
|
||||
{ name: 'purpose', label: 'Purpose', type: 'select', value: s ? s.purpose : 'provisioning', options: ['provisioning', 'mgmt', 'bmc', 'storage', 'ib', 'other'].map(x => [x, x]) },
|
||||
{ name: 'gateway', label: 'Gateway', value: s ? s.gateway : '' }, { name: 'dns', label: 'DNS servers', value: s ? (s.dns || []).join(',') : '', placeholder: 'comma separated' },
|
||||
{ name: 'domain', label: 'Domain', value: s ? s.domain : '' }, { name: 'mtu', label: 'VLAN MTU', value: s ? String(s.mtu || 1500) : '1500' },
|
||||
{ name: 'dhcp', label: 'Serve DHCP/PXE', type: 'checkbox', value: s ? s.dhcp : false },
|
||||
{ name: 'dhcp', label: 'DHCP / PXE', type: 'select', value: s ? (s.dhcp ? (s.dhcp_mode || 'serve') : 'off') : 'off', options: [['off', 'off: another network, or later'], ['serve', 'serve: this rack hands out addresses and PXE'], ['proxy', 'proxy: the site\'s DHCP server hands out addresses; this rack only answers PXE']] },
|
||||
{ name: 'rack', label: 'Rack controller', type: 'select', value: s ? s.primary_rack : '', options: S.controllers.map(c => [c.name, `${c.name}${c.online ? '' : ' (offline)'}`]) },
|
||||
], 'Ranges are added afterwards: dynamic (PXE pool), reserved (never handed out), static (where machines get addresses).', 'Save');
|
||||
if (!v) return;
|
||||
@@ -732,7 +732,7 @@ async function editSubnet(s) {
|
||||
if (fabric && !S.fabrics.some(f => f.name === fabric)) await api('POST', '/api/v1/fabrics', { name: fabric });
|
||||
const vid = Number(v.vid) || 0;
|
||||
await api('POST', '/api/v1/subnets', { cidr: v.cidr.trim(), name: v.name, fabric, vid, purpose: v.purpose, gateway: v.gateway.trim(), dns: v.dns.split(',').map(x => x.trim()).filter(Boolean), domain: v.domain.trim() });
|
||||
if (fabric) await api('POST', '/api/v1/vlans', { fabric, vid, mtu: Number(v.mtu) || 1500, dhcp: v.dhcp, primary_rack: v.dhcp ? v.rack : '' });
|
||||
if (fabric) await api('POST', '/api/v1/vlans', { fabric, vid, mtu: Number(v.mtu) || 1500, dhcp: v.dhcp !== 'off', dhcp_mode: v.dhcp === 'proxy' ? 'proxy' : '', primary_rack: v.dhcp !== 'off' ? v.rack : '' });
|
||||
toast(`subnet ${v.cidr} saved`); refresh(true);
|
||||
} catch (e) { toast(e.message, true); }
|
||||
}
|
||||
|
||||
@@ -130,7 +130,7 @@ function topoSVG(opt = {}) {
|
||||
o.push(`<line x1="${M + 250}" y1="${y}" x2="${W - M}" y2="${y}"/>`);
|
||||
o.push(`<text class="t1" x="${M}" y="${y - 4}">${esc((g.fabric || 'no fabric') + ' · ' + (g.vid ? 'vlan ' + g.vid : 'untagged'))}</text>`);
|
||||
if (!compact) o.push(`<text class="t2" x="${M}" y="${y + 11}">${esc([...v4, ...v6].map(s => s.cidr).join(' '))}</text>`);
|
||||
o.push(`<text class="t3" x="${W - M}" y="${y - 5}" text-anchor="end">${esc(g.purpose)}${g.dhcp ? ' · DHCP/PXE' : ''}${g.mtu && g.mtu !== 1500 ? ' · mtu ' + g.mtu : ''} · ${used} addr</text>`);
|
||||
o.push(`<text class="t3" x="${W - M}" y="${y - 5}" text-anchor="end">${esc(g.purpose)}${g.dhcp ? (g.subnets.some(s => s.dhcp_mode === 'proxy') ? ' · proxy DHCP (PXE only)' : ' · DHCP/PXE') : ''}${g.mtu && g.mtu !== 1500 ? ' · mtu ' + g.mtu : ''} · ${used} addr</text>`);
|
||||
o.push('</g>');
|
||||
});
|
||||
// region + external IPAM
|
||||
|
||||
+22
-8
@@ -25,7 +25,9 @@
|
||||
# --nat UPLINK this host routes for them: forward + masquerade out of UPLINK
|
||||
# (default with --iface and no --gateway: the default route's interface)
|
||||
# --domain D DNS domain handed to machines (default rollcall.internal)
|
||||
# --no-dhcp model the network but leave DHCP to the existing server
|
||||
# --no-dhcp model the network but leave DHCP to the existing server (no PXE from here)
|
||||
# --proxy-dhcp the site's DHCP server keeps handing out addresses; this host only answers
|
||||
# PXE clients with where to boot from (no ranges, no NAT)
|
||||
# --ssh-key K SSH public key (or a file of them) for root on deployed machines; repeatable.
|
||||
# Default: the keys of the user running sudo, then root's.
|
||||
# --os NAME import one OS image now (e.g. debian-13); a nightly timer syncs the catalog
|
||||
@@ -46,6 +48,7 @@ while [ $# -gt 0 ]; do
|
||||
--os) OS=$2; shift ;;
|
||||
--ssh-key) SSHKEYS+=("$2"); shift ;;
|
||||
--no-dhcp) DHCP=0 ;;
|
||||
--proxy-dhcp) DHCP=2 ;;
|
||||
--yes|-y) YES=1 ;;
|
||||
-h|--help) sed -n '2,31p' "$0"; exit 0 ;;
|
||||
*) echo "unknown option $1 (--help)" >&2; exit 2 ;;
|
||||
@@ -70,9 +73,10 @@ if [ -n "$IFACE" ]; then
|
||||
[ -n "$CIDR_ADDR" ] || die "$IFACE has no IPv4 address; give it its address on the provisioning network first"
|
||||
read -r ADDR NET PREFIX FIRST <<<"$(python3 -c 'import ipaddress,sys;i=ipaddress.ip_interface(sys.argv[1]);print(i.ip,i.network,i.network.prefixlen,i.network.network_address)' "$CIDR_ADDR")"
|
||||
[ "$PREFIX" -le 24 ] || die "$NET is smaller than a /24; model it by hand (rollcall subnet add / range add)"
|
||||
if [ -z "$GATEWAY" ]; then GATEWAY=$ADDR; NAT=${NAT:-$UPLINK}; fi
|
||||
case $DHCP in 1) DHCPTXT="served here" ;; 2) DHCPTXT="proxy (the site's server hands out addresses)" ;; *) DHCPTXT=off ;; esac
|
||||
if [ -z "$GATEWAY" ] && [ $DHCP != 2 ]; then GATEWAY=$ADDR; NAT=${NAT:-$UPLINK}; fi
|
||||
[ -n "$NAT" ] && [ "$NAT" = "$IFACE" ] && die "--nat must be the uplink, not $IFACE"
|
||||
echo " provisioning $IFACE $CIDR_ADDR (subnet $NET), machines' gateway $GATEWAY${NAT:+ via NAT out of $NAT}, DHCP $([ $DHCP = 1 ] && echo on || echo off)"
|
||||
echo " provisioning $IFACE $CIDR_ADDR (subnet $NET), machines' gateway ${GATEWAY:-from the site DHCP server}${NAT:+ via NAT out of $NAT}, DHCP $DHCPTXT"
|
||||
else
|
||||
[ -n "$UPLINK" ] || die "no default route: give this host network access first"
|
||||
ADDR=$(addr_of "$UPLINK" | cut -d/ -f1)
|
||||
@@ -271,17 +275,27 @@ else
|
||||
FAB=provisioning
|
||||
rollcall fabric list 2>/dev/null | awk 'NR>1{print $1}' | grep -qx "$FAB" || rollcall fabric add "$FAB" >/dev/null
|
||||
if ! rollcall subnet list 2>/dev/null | awk 'NR>1{print $1}' | grep -qx "$NET"; then
|
||||
rollcall subnet add "$NET" -fabric "$FAB" -vid 0 -purpose provisioning -gateway "$GATEWAY" -dns "$ADDR" -domain "$DOMAIN" -name provisioning >/dev/null
|
||||
rollcall range add "$NET" reserved "$(off 1)" "$(off 20)" >/dev/null
|
||||
rollcall range add "$NET" static "$(off 50)" "$(off 99)" >/dev/null
|
||||
rollcall range add "$NET" dynamic "$(off 100)" "$(off 200)" >/dev/null
|
||||
ok "subnet $NET: reserved $(off 1)-$(off 20), static $(off 50)-$(off 99), dynamic $(off 100)-$(off 200)"
|
||||
if [ $DHCP = 2 ]; then
|
||||
# the site's DHCP server owns the addresses: no ranges (machines keep DHCP);
|
||||
# set a static range aside later (rollcall range add) to give them fixed ones
|
||||
rollcall subnet add "$NET" -fabric "$FAB" -vid 0 -purpose provisioning ${GATEWAY:+-gateway "$GATEWAY"} -domain "$DOMAIN" -name provisioning >/dev/null
|
||||
ok "subnet $NET (addresses from the site's DHCP server)"
|
||||
else
|
||||
rollcall subnet add "$NET" -fabric "$FAB" -vid 0 -purpose provisioning -gateway "$GATEWAY" -dns "$ADDR" -domain "$DOMAIN" -name provisioning >/dev/null
|
||||
rollcall range add "$NET" reserved "$(off 1)" "$(off 20)" >/dev/null
|
||||
rollcall range add "$NET" static "$(off 50)" "$(off 99)" >/dev/null
|
||||
rollcall range add "$NET" dynamic "$(off 100)" "$(off 200)" >/dev/null
|
||||
ok "subnet $NET: reserved $(off 1)-$(off 20), static $(off 50)-$(off 99), dynamic $(off 100)-$(off 200)"
|
||||
fi
|
||||
else
|
||||
ok "subnet $NET exists, kept"
|
||||
fi
|
||||
if [ $DHCP = 1 ]; then
|
||||
rollcall vlan add "$FAB" 0 -dhcp -rack "$RACK" >/dev/null
|
||||
ok "DHCP/PXE on $IFACE"
|
||||
elif [ $DHCP = 2 ]; then
|
||||
rollcall vlan add "$FAB" 0 -proxy-dhcp -rack "$RACK" >/dev/null
|
||||
ok "proxy DHCP on $IFACE: PXE answers only; addresses from the site's DHCP server"
|
||||
fi
|
||||
if [ -n "$NAT" ]; then
|
||||
echo "net.ipv4.ip_forward=1" > /etc/sysctl.d/60-rollcall-forward.conf
|
||||
|
||||
Reference in new issue
Block a user