Docs: one way in — quick-install starts at the setup screens, quickstart becomes development, install.md no longer says engines are yours to install

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 17:32:54 +02:00
1 parent 4979e08d64
commit 6abd510628
5 files changed
+103 -72

No files matched your search

+33 -18
View File
@@ -8,10 +8,10 @@ explains every choice.
| | |
|---|---|
| **One control plane machine** | Linux, x86-64 or ARM64. Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk. |
| **GPU nodes** (optional to start) | The same Linux, with the NVIDIA driver installed. vLLM needs driver 580 or newer and a Turing GPU or newer (not V100). |
| **root** | on the control plane, and `sudo` without a password on every node. |
| **SSH** | from the control plane to each node, as a user with passwordless `sudo` there. |
| **A control plane** | One machine, or three for high availability (the store replicated, either one able to take over). Linux, x86-64 or ARM64: Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk each. |
| **GPU nodes** (optional to start) | The same Linux, with the NVIDIA driver installed. vLLM needs driver 525 or newer and a Turing GPU or newer; a V100 gets an older vLLM. |
| **root** | on the control plane. On the other two of a three-machine setup, `sudo` (its password is asked for if it needs one); on nodes, `sudo` without a password. |
| **SSH** | from the control plane to the other machines. The setup screens take a key, the ssh agent or a password; `node add` uses ssh keys. A machine nothing may log into joins with a code instead (`llmgw join`). |
| **Network** | nodes reach the control plane on 7443 and 8081; users reach it on 8080. The machines need internet access to download engines and models. |
You do not install llama.cpp, vLLM, Python or CUDA. The nodes do that
@@ -35,26 +35,38 @@ install it.
## 2. Set up the control plane
```
sudo llmgw server init
sudo llmgw
```
It asks for your email and a few choices, then sets up everything on this
machine, in a few minutes. In a script, give them as flags instead:
`sudo llmgw server init --admin you@example.com`.
With nothing installed, `llmgw` opens the setup screens. They ask whether
this is **one machine or three** (high availability), and for three, the
other two: address, user, and an ssh key, the ssh agent or a password. They
check every machine before installing anything — system, disk, clock, ports,
internet, whether the machines reach one another — show the plan, and
install:
- the service, its certificates and its configuration;
- your administrator account. **The password is printed once. Copy it now.**
- the service, its certificates and its configuration, and a firewall for
its internal ports;
- with three machines, the store replicated across them (PostgreSQL under
Patroni);
- your administrator account. **The password is shown once. Copy it now.**
- a model mirror, which downloads model files once for the whole cluster;
- this machine as a first node, and a small model (Qwen2.5 0.5B) on it.
Run again later, `sudo llmgw` is the console: the cluster's state and
everything you can do to it ([console.md](console.md)). Every step below is
also there, under **Do**.
**In a script**, the same for one machine:
```
sudo llmgw server init --admin you@example.com
```
If nodes will reach this machine by a name or address it does not know itself
(a DNS name, a NAT address), add `--host` for each:
```
sudo llmgw server init --admin you@example.com --host llm.example.com
```
It is safe to run again. It keeps what exists and finishes what is missing.
`--host llm.example.com`. It is safe to run again: it keeps what exists and
finishes what is missing.
**The mirror on its own machine**, when it wants a larger disk than the
control plane has: first on that machine, with llmgw installed as in step 1,
@@ -75,7 +87,7 @@ the account page.
## 3. Add the GPU nodes
On the control plane:
In the console, **Do → Add a node**. Or on the control plane:
```
sudo llmgw node add ubuntu@10.0.0.21
@@ -197,6 +209,9 @@ through the proxy. There is an nginx configuration too:
## Next
- [install.md](install.md): the same steps with every option explained.
- [install.md](install.md): the same steps with every option explained, and
running the cluster after: monitoring, notifications, single sign-on,
backups, upgrades.
- [console.md](console.md): everything the console does.
- [fabrics.md](fabrics.md): InfiniBand and NVLink.
- [onboarding.md](onboarding.md): accounts, projects, budgets and keys.