Slurm's jobs in llmgw's accounting: projects as accounts, jobs priced
CI / test (push) Failing after 7m24s

With slurm.users llmgw, each project is a Slurm account, its organization's
and team's above it, with the project's members and its team's and
organization's on it; slurm.conf holds a job to its user's accounts. The
controller's agent keeps them with sacctmgr, only what it made, and reports
the jobs that ended in the last day; the leader records each once, its
GPU-hours (core-hours without cards) priced at auth.gpu_hour_cost into its
project's usage as model slurm/PARTITION.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-15 11:54:01 +01:00
1 parent 8547528fb7
commit fcda4d3f85
22 files changed
+1390 -76

No files matched your search

+10 -1
View File
@@ -2107,7 +2107,16 @@ lent, partition`); the leader tells each named node's agent (root) what to be, t
controller ~20 s later, sbatch ran on n1 as ada. Disabled in AD: suspended in 56 s, the Linux user
gone in 89 s, ssh refused, home kept. Enabled: back in 32 s, same UID, key kept. Taken out of
ml-team: out of the team in 11 s; put back: in again in 56 s.
- P6, Slurm's jobs in llmgw's accounting (written 2026-11-12, not started). The tenancy tree to Slurm:
- P6, done 2026-11-15 but for budgets: each project a Slurm account (org and team accounts above it,
ids keyed by kind: the first install's org, team and project share the id "default"), its members and its
team's and org's on it; AccountingStorageEnforce=associations; the controller's agent keeps them with
sacctmgr (only what it made; a user's default account moved before an association goes, the user taken out
when none is left) and reports the jobs that ended in the last day; the leader records each once
(slurm_jobs, keyed cluster/id) with a usage row, model slurm/PARTITION, priced at gpu_hour_cost or
core_hour_cost. Tried on the VMs: accounts made, ayse's job on her project's account ran and another
project's was refused, both jobs in deneme's usage at 345 and 0 micro-units. Left: budgets (a job's cost
is not counted against a budget, which binds keys; GrpTRESMins from a project's budget would hold its jobs).
- P6 as written 2026-11-12: The tenancy tree to Slurm:
each project an account (`sacctmgr add account`, parent its team and organization), each member a user
of it, kept by the leader from the store; `sbatch --account=PROJECT`. The records back: the leader reads
`sacct --json` for jobs ended since the last read (through the controller's agent) into a