Slurm in the first install: server init --slurm, and slurm.all_nodes
CI / test (push) Successful in 9m30s
CI / test (push) Successful in 9m30s
The choices screen has a Slurm part, and --slurm writes slurmctld on this machine's node and slurmd on every node, those that join later too, into server.yaml. slurm.all_nodes (ctl slurm setup --all-nodes) takes the nodes from the leader's inventory. No node is lent at install. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b15944aa1f
commit
914985bf9f
13 files changed
+208
-20
No files matched your search
+14
-1
@@ -166,13 +166,26 @@ is a flag too, and the plan shows the command with them: `--starter qwen3-8b`
|
||||
`--judge-fail open`, `--judge-categories`, `--judge-replicas`,
|
||||
`--judge-budget`, `--language tr`, `--signup-open`, `--default-rpm`,
|
||||
`--default-tpm`, `--max-key-ttl`, `--media` (pictures and 3D: ComfyUI on the
|
||||
nodes, and `presets.auto`), `--evict-after-idle` and `--min-lifetime`. They
|
||||
nodes, and `presets.auto`), `--slurm` (see below), `--evict-after-idle` and `--min-lifetime`. They
|
||||
go into `server.yaml` as init writes it, and become the cluster's first
|
||||
settings when it starts; a `server.yaml` already there is never overwritten,
|
||||
so run again they change nothing — the settings page does. An NVIDIA card with no working
|
||||
driver is reported rather than silently ignored; install the driver and run
|
||||
init again to serve on it.
|
||||
|
||||
**Slurm, for training and batch jobs.** `--slurm`, or the Slurm part on the
|
||||
choices screen, runs a Slurm cluster beside llmgw: slurmctld on this
|
||||
machine's node, slurmd on every node, those that join later too
|
||||
(`slurm.all_nodes`). The agents install munge and Slurm (22.05 or newer)
|
||||
from the machines' own package sources. No node is Slurm's at first: each is
|
||||
drained in Slurm and serves models until `llmgw ctl slurm lend NODE` gives it
|
||||
to Slurm — its models move off first — and `llmgw ctl slurm reclaim NODE`
|
||||
takes it back once its jobs end. Jobs are submitted on this machine with
|
||||
`sbatch` and `srun`; `llmgw ctl slurm status` shows where each node is. It
|
||||
needs this machine to be a node, and its hostname to be a name Slurm takes.
|
||||
On a cluster installed without it, `llmgw ctl slurm setup --controller NODE
|
||||
--all-nodes` does the same later.
|
||||
|
||||
**Secure by default.** What init writes is the hardened configuration, not a
|
||||
demonstration one:
|
||||
|
||||
|
||||
@@ -406,6 +406,7 @@ rather than repeated.
|
||||
| `cluster_name` | `string` | [SlurmSettings](#slurmsettings) |
|
||||
| `controller` | `string` | [SlurmSettings](#slurmsettings) |
|
||||
| `nodes` | `[]string` | [SlurmSettings](#slurmsettings) |
|
||||
| `all_nodes` | `bool` | [SlurmSettings](#slurmsettings) |
|
||||
| `lent` | `[]string` | [SlurmSettings](#slurmsettings) |
|
||||
| `partition` | `string` | [SlurmSettings](#slurmsettings) |
|
||||
| `dir` | `string` | [StrataConfig](#strataconfig) |
|
||||
@@ -3525,6 +3526,13 @@ GPU, and need not be one of Nodes.
|
||||
Nodes are the llmgw nodes that run slurmd. Each stays llmgw's, drained
|
||||
in Slurm, until it is also in Lent.
|
||||
|
||||
### `all_nodes`
|
||||
|
||||
`bool`
|
||||
|
||||
AllNodes runs slurmd on every llmgw node, those that join later too,
|
||||
besides any named in Nodes. Each is llmgw's until it is lent, as any.
|
||||
|
||||
### `lent`
|
||||
|
||||
`[]string`
|
||||
|
||||
Reference in new issue
Block a user