Files
ilkerandClaude Opus 5 d67af5d6c3 hpcod: projects, and what a cluster cost
"How much did the protein run use" is a question about a piece of
work, not about c003-15. A tenant asks for clusters over months;
individually each one is an address, some hardware and a lease, and
nothing joined them up.

A project groups them, and is optional throughout. Somebody who wants
one cluster for an afternoon should not have to invent a project to
get it, and requiring one would mean every tenant carrying a
"default" that means nothing. The form does not even ask when there
are none: a field whose only option is "none" is a question with one
answer.

The cost side needed no new columns. The cluster record already said
when it was created and destroyed, and cluster_vm said how many cores
and how much memory it held; what was missing was anything that read
them. cluster_usage is that, and it is a view rather than a table
because a derived number stored is a number that can disagree with
what it was derived from. It works retroactively - every cluster this
estate has ever built now has a line.

Core-hours are reserved, not executed, and the view says so. That is
the honest unit for an on-demand cluster: the hardware was yours and
nobody else could have it, whether or not a job ran. What was
actually executed is a Slurm accounting question and there is no
slurmdbd.

The project is attached inside the allocating transaction, for the
same reason the placement and the applications are, and it is checked
there to be the tenant's own and still open - a cluster filed against
somebody else's project leaks what they are working on. Closing keeps
history and stops it being offered. There is no delete: the reference
from cluster is RESTRICT, because a project removed from under its
clusters leaves their cost attributed to nothing, and afterwards is
when the question gets asked.

A release now writes manifest.json beside home.tgz: the nodes, the
wall time, the core-hours. The database knows all of it and outlives
the cluster, but the tarball can be moved off this host and the
description travels with it. Not fatal if it fails - the data is
already safe by then, and holding hardware because a note could not
be written is the wrong trade.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 13:05:35 +02:00

153 lines
4.7 KiB
Go

package store
import (
"context"
"reflect"
"strings"
"testing"
"time"
)
// The applications are written in the allocating transaction and read back
// by the provisioner. Nothing between those two points understands them, so
// what matters is that the round trip is exact and ordered.
func TestApplicationsSurviveAllocation(t *testing.T) {
s := testStore(t)
ctx := context.Background()
tenant := mustTenant(t, s, "acme")
want := []ClusterApp{
{Name: "gromacs", Image: "oras://reg.example/hpcapps/gromacs:2024.4",
Digest: "sha256:aaa", SIFSHA256: "bbb",
Version: "2024.4", Module: "gromacs", Binaries: []string{"gmx_mpi", "gmx"}},
// Nothing recorded beyond the name and the image, which is every
// application published before the hashes and the run metadata
// existed. It must round-trip as empty rather than as an error.
{Name: "busybox", Image: "docker://docker.io/library/busybox:latest",
Binaries: []string{}},
}
c, err := s.AllocateCluster(ctx, tenant, plainPlacement(), time.Hour, want, nil)
if err != nil {
t.Fatal(err)
}
got, err := s.Apps(ctx, c.SerialID)
if err != nil {
t.Fatal(err)
}
if len(got) != len(want) {
t.Fatalf("read back %d applications, want %d: %+v", len(got), len(want), got)
}
for i := range want {
if !reflect.DeepEqual(got[i], want[i]) {
t.Errorf("application %d came back as %+v, want %+v", i, got[i], want[i])
}
}
}
// A cluster that asked for nothing reads back as nothing rather than as an
// error. Asking for a bare scheduler is a normal request.
func TestAClusterWithNoApplicationsReadsBackEmpty(t *testing.T) {
s := testStore(t)
ctx := context.Background()
tenant := mustTenant(t, s, "acme")
c, err := s.AllocateCluster(ctx, tenant, plainPlacement(), 0, nil, nil)
if err != nil {
t.Fatal(err)
}
got, err := s.Apps(ctx, c.SerialID)
if err != nil {
t.Fatalf("an empty application list was an error: %v", err)
}
if len(got) != 0 {
t.Errorf("applications appeared from nowhere: %+v", got)
}
}
// The same application twice means the caller resolved it twice. Collapsing
// it silently would hide a catalogue that has two entries under one name.
func TestTheSameApplicationTwiceIsRefused(t *testing.T) {
s := testStore(t)
ctx := context.Background()
tenant := mustTenant(t, s, "acme")
_, err := s.AllocateCluster(ctx, tenant, plainPlacement(), 0, []ClusterApp{
{Name: "gromacs", Image: "oras://reg.example/a:1"},
{Name: "gromacs", Image: "oras://reg.example/b:2"},
}, nil)
if err == nil {
t.Fatal("a duplicate application was accepted")
}
if !strings.Contains(err.Error(), "gromacs") {
t.Errorf("the refusal does not name the application: %v", err)
}
// And the whole allocation rolled back with it: a cluster holding an
// address but no record of what it is for is the state this change
// exists to prevent.
live, err := s.Live(ctx)
if err != nil {
t.Fatal(err)
}
if len(live) != 0 {
t.Errorf("the refused allocation left %d cluster(s) behind", len(live))
}
}
// The question the name index on cluster_app exists for. Nothing asked it
// until hpcod apps was written, which is why a withdrawal could not name who
// it affected and the registry had no rule about what it may delete.
func TestAppUsageNamesWhoIsHoldingWhat(t *testing.T) {
s := testStore(t)
ctx := context.Background()
tenant := mustTenant(t, s, "acme")
first, err := s.AllocateCluster(ctx, tenant, plainPlacement(), time.Hour, []ClusterApp{
{Name: "gromacs", Image: "oras://r/g:2024.4", Version: "2024.4", Digest: "sha256:aa"},
{Name: "busybox", Image: "oras://r/b:musl", Version: "musl"},
}, nil)
if err != nil {
t.Fatal(err)
}
if _, err := s.AllocateCluster(ctx, tenant, plainPlacement(), time.Hour, []ClusterApp{
{Name: "gromacs", Image: "oras://r/g:2024.4", Version: "2024.4", Digest: "sha256:aa"},
}, nil); err != nil {
t.Fatal(err)
}
use, err := s.AppUsage(ctx, "gromacs")
if err != nil {
t.Fatal(err)
}
if len(use) != 2 {
t.Fatalf("gromacs is held by %d clusters, want 2: %+v", len(use), use)
}
if use[0].TenantName != "acme" || use[0].Digest != "sha256:aa" {
t.Errorf("the hold does not carry who or what: %+v", use[0])
}
// An empty name is every application, which is what the retention rule
// needs: the whole set of images no live cluster may lose.
all, err := s.AppUsage(ctx, "")
if err != nil {
t.Fatal(err)
}
if len(all) != 3 {
t.Errorf("every hold is %d rows, want 3: %+v", len(all), all)
}
// A released cluster gives its hold back. Counting destroyed clusters
// would pin every tag that was ever published, for ever.
if err := s.Release(ctx, first.SerialID); err != nil {
t.Fatal(err)
}
after, err := s.AppUsage(ctx, "gromacs")
if err != nil {
t.Fatal(err)
}
if len(after) != 1 {
t.Errorf("after releasing one cluster gromacs is held by %d, want 1", len(after))
}
}