Files
ilkerandClaude Opus 5 6dc2826c17 hpcod: the life of an application, carried past the moment it is published
Publishing worked; everything after it did not. Four things were wrong
and each was silent in its own way.

Versions were compared as text, so asking for "gromacs" with 2024.9 and
2024.10 published returned 2024.9. The whole meaning of an unversioned
reference is "the newest one", and it was a wrong answer rather than a
missing one - invisible until somebody noticed they had spent a week on
last year's build.

A recipe could hold one version while the catalogue could hold many, so
an update either destroyed what it replaced or grew a parallel
directory sharing nothing with the first. Recipes now carry a list, and
the single-version form is read into the same shape, so nothing
downstream knows which was written.

An image was named by a mutable tag. Whoever repushes gromacs:2024.4
changed what every rebuilt cluster received, silently, and the record
meant to freeze the cluster froze only its opinion of it. Two hashes
are now recorded because they are checkable in different places: the
manifest digest answers "has this moved" from the control plane, and
the SIF hash is the only one a head can check with no network and no
credential - which matters, because the head is where a wrong image
does harm. Pulling by digest is deliberately not done: whether this
Apptainer accepts an @sha256: reference is not established, and a
provision failing at its last task over untested syntax would be worse
than the problem.

Module and binaries had been in the catalogue since it was written and
were read by nothing - Module was referenced nowhere in the program. So
a tenant was handed /opt/apps/gromacs.sif and left to work out
Apptainer. They now become a wrapper per binary and an Lmod modulefile,
and the answer is `module load gromacs`. Whether to pass --nv is
decided on the node at run time: passing it without a driver fails
outright, and omitting it where there is one leaves the job on the CPU
saying nothing at all, which is the dangerous one.

Four states replace `published:`. A withdrawn entry stays in the
catalogue file, because dropping it would make it indistinguishable
from one that never existed - and that difference is what an operator
needs when a cluster is found still holding it. A pinned reference to
one is refused with the recorded reason instead of "no application",
which would send the user hunting for a typo in a correct name.

The portal re-reads the catalogue when it changes, so publishing an
application no longer needs a restart - the step that is invisible when
forgotten. C20 ships the catalogue, which nothing did.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-22 10:25:54 +02:00

149 lines
5.3 KiB
Go

package store
import (
"context"
"fmt"
"github.com/jackc/pgx/v5"
)
// ClusterApp is one application a cluster was built to hold, as the head
// node needs it: a name to call the SIF and a URI to pull it from.
//
// Deliberately not catalog.App. What the catalogue knows - which capability
// an application implies, how large it is, whether it is signed - is decided
// before allocation and has no meaning afterwards. Carrying the whole record
// would mean the provisioner imported the catalogue, and the head would be
// one import away from having an opinion about what an application is.
type ClusterApp struct {
Name string `json:"name"`
Image string `json:"image"`
// Digest is what the image's tag pointed at when this cluster was
// allocated. Recorded rather than pulled by, so a head node that ends up
// with something else can say so instead of running it.
Digest string `json:"digest,omitempty"`
// SIFSHA256 is the hash of the file the head should end up with. Empty
// for an application published before it was recorded, which the head
// treats as "cannot check" rather than "checked and fine".
SIFSHA256 string `json:"sif_sha256,omitempty"`
// Version, Module and Binaries are what the head needs to make the
// application runnable: a modulefile called <module>/<version> that puts
// a wrapper for each binary on PATH.
Version string `json:"version,omitempty"`
Module string `json:"module,omitempty"`
Binaries []string `json:"binaries,omitempty"`
}
// recordApps writes the resolved application list in the allocating
// transaction. A duplicate name is refused rather than deduplicated: two
// requests for the same application mean the caller resolved something
// twice, and silently collapsing it would hide that.
func recordApps(ctx context.Context, tx pgx.Tx, serialID int64, apps []ClusterApp) error {
for i, a := range apps {
if a.Name == "" || a.Image == "" {
return fmt.Errorf("application %d needs both a name and an image", i)
}
_, err := tx.Exec(ctx, `
INSERT INTO cluster_app
(cluster_serial_id, ordinal, name, image, digest, sif_sha256,
version, module, binaries)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)`,
serialID, i, a.Name, a.Image, a.Digest, a.SIFSHA256,
a.Version, a.Module, a.Binaries)
if isUniqueViolationOn(err, "cluster_app_name_unique") {
return fmt.Errorf("application %q asked for twice", a.Name)
}
if err != nil {
return err
}
}
return nil
}
// Apps returns what a cluster was allocated with, in the order the catalogue
// resolved them.
//
// A separate call rather than a field filled by every cluster read, which
// mirrors VMs: the list is wanted at exactly one point - rendering the
// variables the tenant module is built from - and loading it for every row
// of a portal listing would be a query per cluster for a column nobody
// displays.
func (s *Store) Apps(ctx context.Context, serialID int64) ([]ClusterApp, error) {
rows, err := s.pool.Query(ctx, `
SELECT name, image, digest, sif_sha256, version, module, binaries
FROM cluster_app
WHERE cluster_serial_id = $1 ORDER BY ordinal`, serialID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []ClusterApp
for rows.Next() {
var a ClusterApp
if err := rows.Scan(&a.Name, &a.Image, &a.Digest, &a.SIFSHA256,
&a.Version, &a.Module, &a.Binaries); err != nil {
return nil, err
}
out = append(out, a)
}
return out, rows.Err()
}
// AppUse is one live cluster's hold on one application.
type AppUse struct {
SerialID int64
AccountingName string
TenantName string
State string
Name string
Version string
Image string
Digest string
SIFSHA256 string
}
// AppUsage answers the question the name index on cluster_app exists for, and
// which nothing asked until now: who is running this.
//
// It is what makes a withdrawal more than a note to oneself. A CVE in GROMACS
// is not addressed by taking it out of the catalogue - that only stops the
// next cluster - and until this existed there was no way to find the clusters
// already holding it short of reading every record.
//
// It is also the retention rule for the registry. A tag that appears here is
// referenced by a cluster that may still be rebuilt, so deleting it would
// mean that rebuild produced a different cluster. Not a retention window: a
// constraint, checkable in one query.
//
// An empty name means every application. Destroyed clusters are excluded:
// their hold is over, and including them would make every tag look pinned
// for ever.
func (s *Store) AppUsage(ctx context.Context, name string) ([]AppUse, error) {
rows, err := s.pool.Query(ctx, `
SELECT c.serial_id, c.accounting_name, t.name, c.state,
a.name, a.version, a.image, a.digest, a.sif_sha256
FROM cluster_app a
JOIN cluster c ON c.serial_id = a.cluster_serial_id
JOIN tenant t ON t.id = c.tenant_id
WHERE c.destroyed_at IS NULL
AND ($1 = '' OR a.name = $1)
ORDER BY a.name, a.version, c.cluster_id`, name)
if err != nil {
return nil, err
}
defer rows.Close()
var out []AppUse
for rows.Next() {
var u AppUse
if err := rows.Scan(&u.SerialID, &u.AccountingName, &u.TenantName, &u.State,
&u.Name, &u.Version, &u.Image, &u.Digest, &u.SIFSHA256); err != nil {
return nil, err
}
out = append(out, u)
}
return out, rows.Err()
}