Commit Graph
5 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 57f0fd2ed0 Slurm's jobs in containers: pyxis and enroot, installed by the agents
CI / test (push) Failing after 6m53s
slurm.containers (ctl slurm containers on) has every Slurm machine's agent
install enroot from NVIDIA's release and build pyxis against its own
Slurm, both pinned and checked against their digests, and on GPU machines
nvidia-container-cli from NVIDIA's repository. enroot keeps each user's
containers under /run/enroot and /var/lib/enroot, needing no login
session, with the shared path in every container. The leader has Slurm
load pyxis only once every machine reports it: plugstack.conf's
"required" fails every job on a machine without the plugin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 21:31:20 +01:00
Ilker ManapandClaude Opus 5.5 4b4306bb6a Slurm users reach a node only through a job of theirs
CI / test (push) Successful in 9m41s
llmgw's Slurm users log in to the controller freely and to another Slurm
machine only while a job of theirs runs there: the agent installs
pam_slurm_adopt and puts it in sshd's account stack for the users it made
alone, so root and the machine's own users are never locked out. Their
session is put in the job's extern step (PrologFlags=Contain), and
pam_systemd, which would move it out, is skipped for them by taking the
session file into sshd's stack with a guard in front of it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 20:30:49 +01:00
Ilker ManapandClaude Opus 5.5 d8ce573bdb Slurm's shared files and users: one path everywhere, accounts as Linux users
CI / test (push) Failing after 7m1s
slurm.shared mounts the site's NFS export, has an llmgw node export the path
over NFS to the Slurm machines alone (2049 in the firewall), or looks for a
file system the site mounts. slurm.users "llmgw" makes each active account a
Linux user on every Slurm machine: login from its e-mail, a UID from 200000
kept in slurm_users and never reused, its home on the shared path, its SSH
keys from the portal's Profile (or ctl slurm users keys) read by sshd from
/etc/llmgw/ssh-keys. server init --slurm chooses /shared from the first
machine and llmgw's users; --slurm-shared and --slurm-users change that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 21:34:34 +01:00
Ilker ManapandClaude Opus 5.5 914985bf9f Slurm in the first install: server init --slurm, and slurm.all_nodes
CI / test (push) Successful in 9m30s
The choices screen has a Slurm part, and --slurm writes slurmctld on this
machine's node and slurmd on every node, those that join later too, into
server.yaml. slurm.all_nodes (ctl slurm setup --all-nodes) takes the nodes
from the leader's inventory. No node is lent at install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:50:35 +01:00
Ilker ManapandClaude Opus 5.5 d439d507d7 Slurm beside llmgw, P1: its settings and the files a controller runs on
slurm.{enabled,cluster_name,controller,nodes,lent,partition} in the cluster's
settings, checked; internal/slurm writes slurm.conf (configless), gres.conf
and cgroup.conf from the nodes' slurmd -C lines and GPUs, derives the munge
key from a cluster secret, and reads scontrol and version output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:20:31 +01:00