Slurm's shared files and users: one path everywhere, accounts as Linux users
CI / test (push) Failing after 7m1s

slurm.shared mounts the site's NFS export, has an llmgw node export the path
over NFS to the Slurm machines alone (2049 in the firewall), or looks for a
file system the site mounts. slurm.users "llmgw" makes each active account a
Linux user on every Slurm machine: login from its e-mail, a UID from 200000
kept in slurm_users and never reused, its home on the shared path, its SSH
keys from the portal's Profile (or ctl slurm users keys) read by sshd from
/etc/llmgw/ssh-keys. server init --slurm chooses /shared from the first
machine and llmgw's users; --slurm-shared and --slurm-users change that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-12 21:34:34 +01:00
1 parent b9d20e7bba
commit d8ce573bdb
45 files changed
+2505 -79

No files matched your search

+20 -1
View File
@@ -195,6 +195,25 @@ On a cluster installed without it, `llmgw ctl slurm setup --controller NODE
networks only, as llmgw's own do; a firewall of the site's own between the
nodes must let them through.
**Slurm's shared files and users.** Jobs need the same files on every
machine: homes, datasets, output, checkpoints. With `--slurm` this machine
exports `/shared` over NFS to every Slurm machine and they mount it
(`slurm.shared`); NFS's port 2049 answers the cluster's networks only, the
export the Slurm machines alone. That is one machine, not highly available
and no faster than its disk and network: enough for a few nodes. A site with
storage of its own uses it instead — `--slurm-shared HOST:/EXPORT` at
install, or `llmgw ctl slurm shared --path /shared --nfs HOST:/EXPORT`
later; Lustre, BeeGFS or GPFS the site mounts itself, and `llmgw ctl slurm
shared --path /lustre/hpc` only has llmgw look that it is there. Every
active llmgw account is a Linux user on every Slurm machine
(`slurm.users: llmgw`): its login from its e-mail address, a UID from
200000 up that is never given to anyone else, its home under
`/shared/home`. People add their SSH public keys in the portal's
**Profile** and log in to this machine to run `sbatch`; an account suspended
or deleted loses its user, its files stay. A site whose users come from its
directory (sssd against Active Directory or LDAP) passes `--slurm-users
site`, or later `llmgw ctl slurm users off`.
**Secure by default.** What init writes is the hardened configuration, not a
demonstration one:
@@ -3167,7 +3186,7 @@ loaded at boot by `llmgw-firewall.service`):
| Machine | Ports, for the cluster only |
| --- | --- |
| control plane | 7443 (agents), metrics when not on loopback, 8081 (its mirror), engines and 9100 when it is a node, 2379/2380 when it holds the store's third etcd vote |
| node | engine ports (`control.port_range`, 20000–20999), 9100; with a part in Slurm also 6817–6819 (slurmctld, slurmd, slurmdbd) and 60001–63000 (srun's `SrunPortRange`) |
| node | engine ports (`control.port_range`, 20000–20999), 9100; with a part in Slurm also 6817–6819 (slurmctld, slurmd, slurmdbd) and 60001–63000 (srun's `SrunPortRange`); exporting Slurm's shared files, 2049 (NFS) |
| store member | 5432, 8008, 2379, 2380, 9100 |
| mirror | 8081, 8082, 9100 |