The baseline fails closed with two judges, and a judge not answering is a notification

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 19:19:49 +02:00
1 parent 07af03aa93
commit c553f2041a
11 files changed
+67 -22

No files matched your search

+3 -3
View File
@@ -8,7 +8,7 @@ explains every choice.
| | |
|---|---|
| **A control plane** | One machine, or three for high availability (the store replicated, either one able to take over). Linux, x86-64 or ARM64: Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk each — 6 GB RAM when it is also a node running the starter model and the judge. |
| **A control plane** | One machine, or three for high availability (the store replicated, either one able to take over). Linux, x86-64 or ARM64: Debian 12, Ubuntu 22.04+ or RHEL 10. No GPU needed; 2 GB RAM, 20 GB disk each — 8 GB RAM when it is also a node running the starter model and the two judges. |
| **GPU nodes** (optional to start) | The same Linux, with the NVIDIA driver installed. vLLM needs driver 525 or newer and a Turing GPU or newer; a V100 gets an older vLLM. |
| **root** | on the control plane. On the other two of a three-machine setup, `sudo` (its password is asked for if it needs one); on nodes, `sudo` without a password. |
| **SSH** | from the control plane to the other machines. The setup screens take a key, the ssh agent or a password; `node add` uses ssh keys. A machine nothing may log into joins with a code instead (`llmgw join`). |
@@ -52,8 +52,8 @@ install:
- your administrator account. **The password is shown once. Copy it now.**
- a model mirror, which downloads model files once for the whole cluster;
- this machine as a first node, and a small model (Qwen2.5 0.5B) on it;
- guardrails on every request, with a judge model (Llama Guard 3 1B) on a
CPU, and API keys that expire within 90 days. What is hardened, and how
- guardrails on every request, with a judge model (Llama Guard 3 1B, two
replicas) on CPUs — a request it cannot check is refused, not let through — and API keys that expire within 90 days. What is hardened, and how
to loosen it: [install.md](install.md#2-the-control-plane).
Run again later, `sudo llmgw` is the console: the cluster's state and