Files
ilkerandClaude Opus 5 6dc2826c17 hpcod: the life of an application, carried past the moment it is published
Publishing worked; everything after it did not. Four things were wrong
and each was silent in its own way.

Versions were compared as text, so asking for "gromacs" with 2024.9 and
2024.10 published returned 2024.9. The whole meaning of an unversioned
reference is "the newest one", and it was a wrong answer rather than a
missing one - invisible until somebody noticed they had spent a week on
last year's build.

A recipe could hold one version while the catalogue could hold many, so
an update either destroyed what it replaced or grew a parallel
directory sharing nothing with the first. Recipes now carry a list, and
the single-version form is read into the same shape, so nothing
downstream knows which was written.

An image was named by a mutable tag. Whoever repushes gromacs:2024.4
changed what every rebuilt cluster received, silently, and the record
meant to freeze the cluster froze only its opinion of it. Two hashes
are now recorded because they are checkable in different places: the
manifest digest answers "has this moved" from the control plane, and
the SIF hash is the only one a head can check with no network and no
credential - which matters, because the head is where a wrong image
does harm. Pulling by digest is deliberately not done: whether this
Apptainer accepts an @sha256: reference is not established, and a
provision failing at its last task over untested syntax would be worse
than the problem.

Module and binaries had been in the catalogue since it was written and
were read by nothing - Module was referenced nowhere in the program. So
a tenant was handed /opt/apps/gromacs.sif and left to work out
Apptainer. They now become a wrapper per binary and an Lmod modulefile,
and the answer is `module load gromacs`. Whether to pass --nv is
decided on the node at run time: passing it without a driver fails
outright, and omitting it where there is one leaves the job on the CPU
saying nothing at all, which is the dangerous one.

Four states replace `published:`. A withdrawn entry stays in the
catalogue file, because dropping it would make it indistinguishable
from one that never existed - and that difference is what an operator
needs when a cluster is found still holding it. A pinned reference to
one is refused with the recorded reason instead of "no application",
which would send the user hunting for a typo in a correct name.

The portal re-reads the catalogue when it changes, so publishing an
application no longer needs a restart - the step that is invisible when
forgotten. C20 ships the catalogue, which nothing did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 10:25:54 +02:00

117 lines
3.9 KiB
Go

package catalog
import (
"os"
"path/filepath"
"testing"
"time"
)
func write(t *testing.T, path, body string) {
t.Helper()
if err := os.WriteFile(path, []byte(body), 0o644); err != nil {
t.Fatal(err)
}
// mtime has one-second granularity on some filesystems, and a test that
// rewrote a file within the same second would look unchanged. Moved
// backwards rather than slept forwards, so the test stays fast.
old := time.Now().Add(-2 * time.Second)
if err := os.Chtimes(path, old, old); err != nil {
t.Fatal(err)
}
}
// The whole reason this type exists: publishing an application must not need
// a restart. Three steps already stand between an image and a tenant seeing
// it, and a fourth that is invisible when forgotten is the one that wastes
// an afternoon.
func TestANewEntryAppearsWithoutARestart(t *testing.T) {
p := filepath.Join(t.TempDir(), "apps.yaml")
write(t, p, "apps:\n - name: busybox\n image: oras://r/busybox:musl\n")
w := FromFile(p)
if got := len(w.Current().Apps); got != 1 {
t.Fatalf("started with %d entries, want 1", got)
}
write(t, p, "apps:\n - name: busybox\n image: oras://r/busybox:musl\n"+
" - name: stress-ng\n image: oras://r/stress-ng:0.18.07\n")
got := w.Current()
if len(got.Apps) != 2 {
t.Fatalf("after the file changed there are %d entries, want 2", len(got.Apps))
}
if _, ok := got.Lookup("stress-ng"); !ok {
t.Error("the new application does not resolve by name")
}
}
// A half-written file is a moment, not a state. Emptying the plan page
// because somebody saved broken YAML would be a larger outage than the
// mistake that caused it.
func TestABrokenFileKeepsThePreviousContents(t *testing.T) {
p := filepath.Join(t.TempDir(), "apps.yaml")
write(t, p, "apps:\n - name: busybox\n image: oras://r/busybox:musl\n")
w := FromFile(p)
_ = w.Current()
write(t, p, "apps:\n - name: busybox\n image: [unclosed\n")
if got := len(w.Current().Apps); got != 1 {
t.Fatalf("a broken file left %d entries, want the previous 1", got)
}
// And the mtime was not recorded, so a file fixed a second later is
// picked up rather than ignored until the portal restarts.
write(t, p, "apps:\n - name: busybox\n image: oras://r/busybox:musl\n"+
" - name: blast\n image: oras://r/blast:2.16.0\n")
if got := len(w.Current().Apps); got != 2 {
t.Fatalf("after the file was repaired there are %d entries, want 2", got)
}
}
// A file that is deleted is a complete statement - nothing is published -
// unlike a file that is merely unparseable.
func TestADeletedFilePublishesNothing(t *testing.T) {
p := filepath.Join(t.TempDir(), "apps.yaml")
write(t, p, "apps:\n - name: busybox\n image: oras://r/busybox:musl\n")
w := FromFile(p)
_ = w.Current()
if err := os.Remove(p); err != nil {
t.Fatal(err)
}
if got := len(w.Current().Apps); got != 0 {
t.Errorf("a removed catalogue still offers %d entries", got)
}
}
// An unchanged file must not be re-parsed on every request: the plan page
// renders this on each load, and the portal answers the tenant subdomains too.
func TestAnUnchangedFileIsNotRereadEveryTime(t *testing.T) {
p := filepath.Join(t.TempDir(), "apps.yaml")
write(t, p, "apps:\n - name: busybox\n image: oras://r/busybox:musl\n")
w := FromFile(p)
first := w.Current()
second := w.Current()
if first != second {
t.Error("the catalogue was parsed again although the file had not changed")
}
}
// A portal with no catalogue configured is a site that publishes nothing,
// not an error, and Current must still be safe to call.
func TestNoCatalogueConfigured(t *testing.T) {
if got := len(FromFile("").Current().Apps); got != 0 {
t.Errorf("an unconfigured catalogue offered %d entries", got)
}
if got := len(Static(nil).Current().Apps); got != 0 {
t.Errorf("a static nil catalogue offered %d entries", got)
}
var w *Watcher
if got := len(w.Current().Apps); got != 0 {
t.Errorf("a nil watcher offered %d entries", got)
}
}