A hardened install by default: baseline guardrails with Llama Guard 3 1B judging on a CPU, keys at most 90 days

server init (and the setup screens) write guard.default [baseline] —
secrets refused, cards masked, injection recorded, the judge refusing
S1-S4, S7, S9-S12 in prompts and answers, fail open after 5s — fetch the
judge into the mirror and run it on a node's CPU. --no-guard-model leaves
it out. The closing text names what init cannot do: TLS on the client
port and backup recipients.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 17:45:31 +02:00
1 parent 6abd510628
commit ddb0472f9b
9 files changed
+284 -37

No files matched your search

+34 -2
View File
@@ -117,7 +117,8 @@ That is the whole step. In order, it:
3. issues the control plane's certificate for this machine's hostname and
every address it has, and copies `ca.pem` beside it
4. writes `/etc/llmgw/server.yaml`: mutual TLS, authentication, default limits
and a price, and a control plane with an empty catalog
and a price, keys that expire after at most 90 days, the baseline
guardrails policy, and a control plane with an empty catalog
5. creates the store, the `default` project, and your account, already approved
and promoted to administrator — the password is generated and **printed
once**
@@ -129,8 +130,39 @@ That is the whole step. In order, it:
engine
9. fetches Qwen2.5 0.5B into the mirror and serves it on this node, so the
first thing anyone opens `/chat` to find is a model that answers
10. fetches Llama Guard 3 1B, the guardrails' judge, and runs it on the CPU
of a node with room — this one, or the first that joins
11. closes llmgw's internal ports to everything but the cluster's networks
`--no-mirror`, `--no-node` and `--no-model` leave the last three out. An NVIDIA
`--no-mirror`, `--no-node` and `--no-model` leave out 7 to 9,
`--no-guard-model` 10, and `--no-firewall` 11.
**Secure by default.** What init writes is the hardened configuration, not a
demonstration one:
| | |
|---|---|
| Agents | mutual TLS with the cluster's own authority; the agent port refuses plaintext |
| Admin API | loopback only, behind a token; pprof off |
| Store | between machines, TLS everywhere (PostgreSQL, etcd, Patroni) |
| Firewall | llmgw's internal ports answer the cluster's networks only |
| Accounts | signup closed; administrators create accounts or approve them |
| Keys | at most 90 days (`auth.max_key_ttl`), shorter per part of the tree with `llmgw keys ttl` |
| Requests | every key limited (60 a minute, 200 000 tokens a minute) and priced, so budgets bind |
| Guardrails | the `baseline` policy on every request: secrets in prompts and answers refused, card numbers masked, injection attempts recorded, and the judge refusing violence, crime, weapons, child abuse, privacy violations, hate, self-harm and sexual content, in prompts and answers |
The judge runs on a CPU, about half a second for a short text, and reads a
streamed answer every 600 characters. When it is slower than 5 seconds or
not running, the request is served and recorded as flagged rather than
refused (`fail: open`), so a judge that is down does not take the gateway
with it; `fail: closed` turns that round. Defamation, specialized advice,
intellectual property and elections are left out of what it refuses: they
catch too much ordinary work. [guardrails.md](guardrails.md) has the rest.
Two things are left to you because they need something only you have: a
certificate for the client port — port 8080 is plain HTTP; put the proxy in
[deploy/proxy/](../deploy/proxy/) in front — and the keys backups are
encrypted to (`backup_recipients`, see [Backing up](#backing-up)). An NVIDIA
card with no working driver is reported rather than silently ignored; install
the driver and run init again to serve on it.