Slurm's accounting: slurmdbd and a MariaDB on the controller
CI / test (push) Successful in 10m57s

Since 23.11 Slurm runs badly without accounting storage: the main
scheduler passes every job by as InvalidAccount. slurmdbd takes MySQL or
MariaDB only, so the controller's agent installs MariaDB beside it,
answering that machine alone, makes the database and its user (the
password made from the munge key, never on a command line), runs
slurmdbd and adds the cluster, all before slurmctld. A MariaDB that was
on the machine before keeps its settings. slurm.conf accounts the jobs
there, with what each used from its cgroup and the cards, and the
priority is multifactor again. Port 6819 joins Slurm's in the firewall.
ctl slurm status and the console say whether slurmdbd runs; the install
screen, server init --slurm and the guide say what is put where.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-12 07:59:08 +01:00
1 parent 02774769a7
commit 748f777a56
20 files changed
+425 -37

No files matched your search

+10 -3
View File
@@ -177,14 +177,21 @@ init again to serve on it.
choices screen, runs a Slurm cluster beside llmgw: slurmctld on this
machine's node, slurmd on every node, those that join later too
(`slurm.all_nodes`). The agents install munge and Slurm (22.05 or newer)
from the machines' own package sources. No node is Slurm's at first: each is
from the machines' own package sources, and on this machine slurmdbd, which
keeps the jobs' records (`sacct`), in a MariaDB installed beside it that
answers this machine alone. Slurm keeps its records in MySQL or MariaDB
only, not in the cluster's PostgreSQL; the database's password is made from
the cluster's munge key, so nothing new is stored. A MariaDB the machine
already had keeps its settings and only gets the database `slurm_acct_db`
and its user. Back it up with the machine (`mariadb-dump slurm_acct_db`);
records older than a year are purged. No node is Slurm's at first: each is
drained in Slurm and serves models until `llmgw ctl slurm lend NODE` gives it
to Slurm — its models move off first — and `llmgw ctl slurm reclaim NODE`
takes it back once its jobs end. Jobs are submitted on this machine with
`sbatch` and `srun`; `llmgw ctl slurm status` shows where each node is. It
needs this machine to be a node, and its hostname to be a name Slurm takes.
On a cluster installed without it, `llmgw ctl slurm setup --controller NODE
--all-nodes` does the same later. Slurm's ports (6817–6818, and srun's 60001–63000) answer the cluster's
--all-nodes` does the same later. Slurm's ports (6817–6819, and srun's 60001–63000) answer the cluster's
networks only, as llmgw's own do; a firewall of the site's own between the
nodes must let them through.
@@ -3160,7 +3167,7 @@ loaded at boot by `llmgw-firewall.service`):
| Machine | Ports, for the cluster only |
| --- | --- |
| control plane | 7443 (agents), metrics when not on loopback, 8081 (its mirror), engines and 9100 when it is a node, 2379/2380 when it holds the store's third etcd vote |
| node | engine ports (`control.port_range`, 20000–20999), 9100; with a part in Slurm also 6817–6818 (slurmctld, slurmd) and 60001–63000 (srun's `SrunPortRange`) |
| node | engine ports (`control.port_range`, 20000–20999), 9100; with a part in Slurm also 6817–6819 (slurmctld, slurmd, slurmdbd) and 60001–63000 (srun's `SrunPortRange`) |
| store member | 5432, 8008, 2379, 2380, 9100 |
| mirror | 8081, 8082, 9100 |