Slurm's shared files and users: one path everywhere, accounts as Linux users
CI / test (push) Failing after 7m1s
CI / test (push) Failing after 7m1s
slurm.shared mounts the site's NFS export, has an llmgw node export the path over NFS to the Slurm machines alone (2049 in the firewall), or looks for a file system the site mounts. slurm.users "llmgw" makes each active account a Linux user on every Slurm machine: login from its e-mail, a UID from 200000 kept in slurm_users and never reused, its home on the shared path, its SSH keys from the portal's Profile (or ctl slurm users keys) read by sshd from /etc/llmgw/ssh-keys. server init --slurm chooses /shared from the first machine and llmgw's users; --slurm-shared and --slurm-users change that. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b9d20e7bba
commit
d8ce573bdb
45 files changed
+2505
-79
No files matched your search
+20
-1
@@ -195,6 +195,25 @@ On a cluster installed without it, `llmgw ctl slurm setup --controller NODE
|
||||
networks only, as llmgw's own do; a firewall of the site's own between the
|
||||
nodes must let them through.
|
||||
|
||||
**Slurm's shared files and users.** Jobs need the same files on every
|
||||
machine: homes, datasets, output, checkpoints. With `--slurm` this machine
|
||||
exports `/shared` over NFS to every Slurm machine and they mount it
|
||||
(`slurm.shared`); NFS's port 2049 answers the cluster's networks only, the
|
||||
export the Slurm machines alone. That is one machine, not highly available
|
||||
and no faster than its disk and network: enough for a few nodes. A site with
|
||||
storage of its own uses it instead — `--slurm-shared HOST:/EXPORT` at
|
||||
install, or `llmgw ctl slurm shared --path /shared --nfs HOST:/EXPORT`
|
||||
later; Lustre, BeeGFS or GPFS the site mounts itself, and `llmgw ctl slurm
|
||||
shared --path /lustre/hpc` only has llmgw look that it is there. Every
|
||||
active llmgw account is a Linux user on every Slurm machine
|
||||
(`slurm.users: llmgw`): its login from its e-mail address, a UID from
|
||||
200000 up that is never given to anyone else, its home under
|
||||
`/shared/home`. People add their SSH public keys in the portal's
|
||||
**Profile** and log in to this machine to run `sbatch`; an account suspended
|
||||
or deleted loses its user, its files stay. A site whose users come from its
|
||||
directory (sssd against Active Directory or LDAP) passes `--slurm-users
|
||||
site`, or later `llmgw ctl slurm users off`.
|
||||
|
||||
**Secure by default.** What init writes is the hardened configuration, not a
|
||||
demonstration one:
|
||||
|
||||
@@ -3167,7 +3186,7 @@ loaded at boot by `llmgw-firewall.service`):
|
||||
| Machine | Ports, for the cluster only |
|
||||
| --- | --- |
|
||||
| control plane | 7443 (agents), metrics when not on loopback, 8081 (its mirror), engines and 9100 when it is a node, 2379/2380 when it holds the store's third etcd vote |
|
||||
| node | engine ports (`control.port_range`, 20000–20999), 9100; with a part in Slurm also 6817–6819 (slurmctld, slurmd, slurmdbd) and 60001–63000 (srun's `SrunPortRange`) |
|
||||
| node | engine ports (`control.port_range`, 20000–20999), 9100; with a part in Slurm also 6817–6819 (slurmctld, slurmd, slurmdbd) and 60001–63000 (srun's `SrunPortRange`); exporting Slurm's shared files, 2049 (NFS) |
|
||||
| store member | 5432, 8008, 2379, 2380, 9100 |
|
||||
| mirror | 8081, 8082, 9100 |
|
||||
|
||||
|
||||
Reference in new issue
Block a user