Answer the two questions asked before a gateway is allowed near anything
CI / test (push) Successful in 1m34s

Neither had a page. "We have GPUs, now what" was left to the reader, and what
llmgw stores was scattered across four documents and a changelog — which is
the same as not being written down when somebody is doing a security review.

docs/data.md describes the behaviour and leaves the legal mapping to whoever
holds the obligation. Every claim in it was checked against the source, and it
names where to look for each: usage rows carry no text field, keys are stored
hashed, conversations are off until a named project turns them on and cannot be
turned on without a retention window.

It also states the egress exactly, which took reading every outbound call to
say honestly: nothing on the serving path leaves the cluster, and the two
things that can reach outside — fetching weights, and ctl plan reading a model's
architecture — are both started by an operator. Weight fetching points wherever
you tell it, including an internal mirror.

docs/choosing-models.md is the opinion llmgw itself does not have. Three jobs
rather than one model, with the third — embeddings — being the one that gets
forgotten and the one that decides whether the deployment can answer questions
about your company or only questions in general.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-27 08:16:08 +02:00
1 parent f2f375d824
commit b7a97468e0
4 files changed
+289

No files matched your search

+4
View File
@@ -786,6 +786,10 @@ requests. On a slow model, set `timeouts.generation` higher.
## What to read next
- `docs/choosing-models.md` — which models to run and how many, if the answer
to "we have GPUs, now what" is not already obvious
- `docs/data.md` — what is stored and what leaves the machine, for the review
that happens before this is allowed near anyone's documents
- `docs/reference.md` — every configuration key, generated from the source, so
it describes the version you are running rather than the one somebody last
wrote about