Answer the two questions asked before a gateway is allowed near anything
CI / test (push) Successful in 1m34s
CI / test (push) Successful in 1m34s
Neither had a page. "We have GPUs, now what" was left to the reader, and what llmgw stores was scattered across four documents and a changelog — which is the same as not being written down when somebody is doing a security review. docs/data.md describes the behaviour and leaves the legal mapping to whoever holds the obligation. Every claim in it was checked against the source, and it names where to look for each: usage rows carry no text field, keys are stored hashed, conversations are off until a named project turns them on and cannot be turned on without a retention window. It also states the egress exactly, which took reading every outbound call to say honestly: nothing on the serving path leaves the cluster, and the two things that can reach outside — fetching weights, and ctl plan reading a model's architecture — are both started by an operator. Weight fetching points wherever you tell it, including an internal mirror. docs/choosing-models.md is the opinion llmgw itself does not have. Three jobs rather than one model, with the third — embeddings — being the one that gets forgotten and the one that decides whether the deployment can answer questions about your company or only questions in general. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f2f375d824
commit
b7a97468e0
4 files changed
+289
No files matched your search
@@ -786,6 +786,10 @@ requests. On a slow model, set `timeouts.generation` higher.
|
||||
|
||||
## What to read next
|
||||
|
||||
- `docs/choosing-models.md` — which models to run and how many, if the answer
|
||||
to "we have GPUs, now what" is not already obvious
|
||||
- `docs/data.md` — what is stored and what leaves the machine, for the review
|
||||
that happens before this is allowed near anyone's documents
|
||||
- `docs/reference.md` — every configuration key, generated from the source, so
|
||||
it describes the version you are running rather than the one somebody last
|
||||
wrote about
|
||||
|
||||
Reference in new issue
Block a user