Stop rebuilding the image every night when nothing has changed
All checks were successful
build-image / build (push) Successful in 3s
All checks were successful
build-image / build (push) Successful in 3s
The daily workflow already had a gate, but it could never say no: both of its version comparisons were guaranteed to differ. The image installs samba from trixie-security (2:4.22.10+dfsg-0+deb13u2) while decide-build.sh read only trixie/main (...u1), so the versions never matched. And image-info recorded the kernel as the ABI string from /boot/vmlinuz-* (6.12.107+deb13-amd64), which is a package-name suffix, while the script compared it against the archive's package version (6.12.107-1) — two formats that cannot match by construction. Either one alone forced a build; together they produced a fresh multi-hundred-megabyte release every morning for nothing. Merge trixie, trixie-updates and trixie-security before picking the newest version, and record kernel_version in image-info so there is something comparable on both sides. Compare with dpkg instead of sort -V, which mishandles epochs and tildes, and with "gt" instead of "!=" so an image that is somehow ahead of the archive cannot spin in a rebuild loop. Verified against the live archive and a stubbed release API: archive == image now yields build=no, and the samba, kernel, age, first-build and image-ahead-of-archive branches all decide correctly. Releases published before this change carry no kernel_version, so the kernel check is skipped for them and says so on stderr rather than passing silently. The next build closes that gap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -9,12 +9,27 @@
|
||||
#
|
||||
# A rebuild happens when any of these is true:
|
||||
#
|
||||
# 1. samba in the Debian suite differs from the published image
|
||||
# 2. the kernel package differs — kernel security fixes do not bump samba
|
||||
# 1. samba in the Debian archive is newer than the published image
|
||||
# 2. the kernel package is newer — kernel security fixes do not bump samba
|
||||
# 3. the newest release is older than max-age-days — most Debian security
|
||||
# updates touch neither, so without a floor an image could sit unchanged
|
||||
# for months while its openssl and glibc went stale
|
||||
#
|
||||
# Two things this gets wrong if you are not careful, both of which made it
|
||||
# rebuild every single night:
|
||||
#
|
||||
# * The image installs from trixie, trixie-updates AND trixie-security.
|
||||
# Reading only trixie/main reports samba 2:4.22.10+dfsg-0+deb13u1 while the
|
||||
# image contains ...u2 from security, so the versions never match.
|
||||
# * /boot/vmlinuz-* yields an ABI string such as 6.12.107+deb13-amd64, which
|
||||
# is a package-name suffix, not a version. Comparing it against the
|
||||
# archive's 6.12.107-1 never matches either. Images now also record
|
||||
# kernel_version, which is directly comparable.
|
||||
#
|
||||
# Versions are compared with dpkg rather than sort -V, which mishandles epochs
|
||||
# and tildes, and with "gt" rather than "!=" so an image that is somehow ahead
|
||||
# of the archive does not trigger an endless rebuild loop.
|
||||
#
|
||||
# Requires GITEA_API and GITEA_TOKEN in the environment.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/../.."
|
||||
@@ -26,13 +41,39 @@ MAX_AGE_DAYS="${1:-30}"
|
||||
SUITE=$(make -s print-var VAR=DEBIAN_SERIES)
|
||||
ARCH=$(make -s print-var VAR=ARCH)
|
||||
|
||||
# Newest version of each package we care about, in one pass over the index.
|
||||
PKGS=$(curl -fsS "http://deb.debian.org/debian/dists/${SUITE}/main/binary-${ARCH}/Packages.gz" | gunzip)
|
||||
newest() {
|
||||
printf '%s\n' "$PKGS" \
|
||||
| awk -v want="$1" '$1=="Package:" && $2==want {p=1; next} p && $1=="Version:" {print $2; p=0}' \
|
||||
| sort -V | tail -1
|
||||
TMP=$(mktemp -d); trap 'rm -rf "$TMP"' EXIT
|
||||
INDEX="$TMP/packages"
|
||||
|
||||
# $1 url, $2 "required"|"optional". Appends the decompressed index to $INDEX.
|
||||
fetch_index() {
|
||||
local url=$1 need=$2
|
||||
if curl -fsS "$url" 2>/dev/null | xz -dc >> "$INDEX" 2>/dev/null; then
|
||||
return 0
|
||||
fi
|
||||
if [ "$need" = required ]; then
|
||||
echo "cannot read package index: $url" >&2
|
||||
exit 1
|
||||
fi
|
||||
echo "note: no index at $url (skipping)" >&2
|
||||
}
|
||||
|
||||
: > "$INDEX"
|
||||
fetch_index "http://deb.debian.org/debian/dists/${SUITE}/main/binary-${ARCH}/Packages.xz" required
|
||||
fetch_index "http://deb.debian.org/debian-security/dists/${SUITE}-security/main/binary-${ARCH}/Packages.xz" required
|
||||
fetch_index "http://deb.debian.org/debian/dists/${SUITE}-updates/main/binary-${ARCH}/Packages.xz" optional
|
||||
|
||||
# Highest version of a package across every suite we merged.
|
||||
newest() {
|
||||
local best="" v
|
||||
while read -r v; do
|
||||
[ -n "$v" ] || continue
|
||||
if [ -z "$best" ] || dpkg --compare-versions "$v" gt "$best"; then best=$v; fi
|
||||
done < <(awk -v want="$1" '
|
||||
$1=="Package:" && $2==want {p=1; next}
|
||||
p && $1=="Version:" {print $2; p=0}' "$INDEX")
|
||||
printf '%s\n' "$best"
|
||||
}
|
||||
|
||||
UP_SAMBA=$(newest samba)
|
||||
UP_KERNEL=$(newest "linux-image-${ARCH}")
|
||||
[ -n "$UP_SAMBA" ] || { echo "samba not found in Debian ${SUITE}" >&2; exit 1; }
|
||||
@@ -60,9 +101,10 @@ PREV_SAMBA="" PREV_KERNEL=""
|
||||
if [ -n "$INFO_URL" ]; then
|
||||
INFO=$(curl -fsSL "$INFO_URL" || true)
|
||||
PREV_SAMBA=$(printf '%s' "$INFO" | awk -F= '$1=="samba"{print $2}')
|
||||
PREV_KERNEL=$(printf '%s' "$INFO" | awk -F= '$1=="kernel"{print $2}')
|
||||
PREV_KERNEL=$(printf '%s' "$INFO" | awk -F= '$1=="kernel_version"{print $2}')
|
||||
[ "$PREV_KERNEL" = unknown ] && PREV_KERNEL=""
|
||||
fi
|
||||
echo "published: samba=${PREV_SAMBA:-<none>} kernel=${PREV_KERNEL:-<none>}" >&2
|
||||
echo "published: samba=${PREV_SAMBA:-<none>} kernel_version=${PREV_KERNEL:-<none>}" >&2
|
||||
|
||||
AGE_DAYS=99999
|
||||
if [ -n "$CREATED" ]; then
|
||||
@@ -77,13 +119,18 @@ BUILD=no
|
||||
REASON="up to date"
|
||||
if [ -z "$PREV_SAMBA" ]; then
|
||||
BUILD=yes; REASON="no published image yet"
|
||||
elif [ "$UP_SAMBA" != "$PREV_SAMBA" ]; then
|
||||
elif dpkg --compare-versions "$UP_SAMBA" gt "$PREV_SAMBA"; then
|
||||
BUILD=yes; REASON="samba ${PREV_SAMBA} -> ${UP_SAMBA}"
|
||||
elif [ "$UP_KERNEL" != "$PREV_KERNEL" ]; then
|
||||
elif [ -n "$PREV_KERNEL" ] && dpkg --compare-versions "$UP_KERNEL" gt "$PREV_KERNEL"; then
|
||||
BUILD=yes; REASON="kernel ${PREV_KERNEL} -> ${UP_KERNEL}"
|
||||
elif [ "$AGE_DAYS" -ge "$MAX_AGE_DAYS" ]; then
|
||||
BUILD=yes; REASON="image is ${AGE_DAYS} days old (limit ${MAX_AGE_DAYS}) — picking up Debian updates"
|
||||
fi
|
||||
# Releases published before image-info carried kernel_version cannot be checked
|
||||
# for kernel changes; say so rather than letting the gap pass unnoticed.
|
||||
if [ -n "$PREV_SAMBA" ] && [ -z "$PREV_KERNEL" ]; then
|
||||
echo "note: published image records no kernel_version; kernel check skipped" >&2
|
||||
fi
|
||||
echo "decision: ${BUILD} (${REASON})" >&2
|
||||
|
||||
printf 'build=%s\n' "$BUILD"
|
||||
|
||||
@@ -237,7 +237,12 @@ log "writing /etc/adc-maas/image-info"
|
||||
echo "debian_suite=${DEBIAN_SUITE}"
|
||||
echo "packer_maas_ref=${PACKER_MAAS_REF}"
|
||||
dpkg-query -W -f='${Package}=${Version}\n' samba samba-ad-dc winbind krb5-user 2>/dev/null
|
||||
echo "kernel=$(ls -1 /boot/vmlinuz-* 2>/dev/null | sed 's|.*/vmlinuz-||' | head -1)"
|
||||
kabi=$(ls -1 /boot/vmlinuz-* 2>/dev/null | sed 's|.*/vmlinuz-||' | head -1)
|
||||
echo "kernel=${kabi}"
|
||||
# The line above is the ABI string, which is a package-name suffix and only
|
||||
# changes on an ABI bump. CI needs something it can actually compare against
|
||||
# the archive, so record the kernel package version as well.
|
||||
echo "kernel_version=$(dpkg-query -W -f='${Version}' "linux-image-${kabi}" 2>/dev/null || echo unknown)"
|
||||
} > /etc/adc-maas/image-info
|
||||
cat /etc/adc-maas/image-info
|
||||
|
||||
|
||||
Reference in New Issue
Block a user