A machine too small for the models it would serve is said so first
CI / test (push) Successful in 8m42s
CI / test (push) Successful in 8m42s
A control plane that is also a node, serving the starter model and two judges, needs about 10.5 GB: 3.5 for itself, 2 for the starter, 2 for each judge, and one more because the scheduler counts whole gigabytes. On 8 GB the install did every step and then had nowhere to run the starter. The setup's checks now fail this machine's memory and name what it would serve, and server init refuses before it installs anything, naming the flags that leave those models off it. quick-install.md said 8 GB; it says 12. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
06baa38858
commit
d110968243
10 files changed
+131
-5
No files matched your search
@@ -8,7 +8,7 @@ explains every choice.
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **A control plane** | One machine, or three for high availability (the store replicated, either one able to take over). Linux, x86-64 or ARM64: Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk each — 8 GB RAM when it is also a node running the starter model and the two judges. |
|
||||
| **A control plane** | One machine, or three for high availability (the store replicated, either one able to take over). Linux, x86-64 or ARM64: Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk each — 12 GB RAM when it is also a node running the starter model and the two judges, which setup checks before it installs anything; four cores or more, or the judges on machines of their own, keep a judgment within its wait. |
|
||||
| **GPU nodes** (optional to start) | The same Linux, with the NVIDIA driver installed. vLLM needs driver 525 or newer and a Turing GPU or newer; a V100 gets an older vLLM. |
|
||||
| **root** | on the control plane. On the other two of a three-machine setup, `sudo` (its password is asked for if it needs one); on nodes, `sudo` without a password. |
|
||||
| **SSH** | from the control plane to the other machines. The setup screens take a key, the ssh agent or a password; `node add` uses ssh keys. A machine nothing may log into joins with a code instead (`llmgw join`). |
|
||||
|
||||
Reference in new issue
Block a user