A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days

server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 17:45:31 +02:00
1 parent 6abd510628
commit ddb0472f9b
9 files changed
+284 -37

No files matched your search

+5 -2
View File
@@ -8,7 +8,7 @@ explains every choice.
| | |
|---|---|
| **A control plane** | One machine, or three for high availability (the store replicated, either one able to take over). Linux, x86-64 or ARM64: Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk each. |
| **A control plane** | One machine, or three for high availability (the store replicated, either one able to take over). Linux, x86-64 or ARM64: Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk each — 6 GB RAM when it is also a node running the starter model and the judge. |
| **GPU nodes** (optional to start) | The same Linux, with the NVIDIA driver installed. vLLM needs driver 525 or newer and a Turing GPU or newer; a V100 gets an older vLLM. |
| **root** | on the control plane. On the other two of a three-machine setup, `sudo` (its password is asked for if it needs one); on nodes, `sudo` without a password. |
| **SSH** | from the control plane to the other machines. The setup screens take a key, the ssh agent or a password; `node add` uses ssh keys. A machine nothing may log into joins with a code instead (`llmgw join`). |
@@ -51,7 +51,10 @@ install:
Patroni);
- your administrator account. **The password is shown once. Copy it now.**
- a model mirror, which downloads model files once for the whole cluster;
- this machine as a first node, and a small model (Qwen2.5 0.5B) on it.
- this machine as a first node, and a small model (Qwen2.5 0.5B) on it;
- guardrails on every request, with a judge model (Llama Guard 3 1B) on a
CPU, and API keys that expire within 90 days. What is hardened, and how
to loosen it: [install.md](install.md#2-the-control-plane).
Run again later, `sudo llmgw` is the console: the cluster's state and
everything you can do to it ([console.md](console.md)). Every step below is