Chat requests answered from knowledge bases

A request naming collections in X-Llmgw-Knowledge has its last question
embedded on the caller's account, the best passages found and added to the
system message, numbered, with an instruction to cite them and to say when
they do not hold the answer; x-llmgw-sources says which were given. Done
before the guardrails judge the request, never served from the answer cache,
and a collection the key may not read is answered as one that does not
exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 15:27:50 +02:00
1 parent 65116064d2
commit 496625824b
10 files changed
+653 -28

No files matched your search

+14 -1
View File
@@ -946,7 +946,7 @@ PostgreSQL tests of store, router and adminapi share LLMGW_TEST_POSTGRES and
the store's TRUNCATE each test; run them with `go test -p 1` or one package
at a time, or a router test loses its rows mid-run.
### F3. Retrieval on the request path (M)
### F3. Retrieval on the request path (M) — done 2026-10-08 (branch `rag`), but defaults and cache_salt
- `llmgw.knowledge: [collections]` in the body (stripped before the
engine), or default collections per project/key.
- Allowed set from the caller's chain (§7.9.2), cached like limits; grants
@@ -966,6 +966,19 @@ at a time, or a router test loses its rows mid-run.
(including via cache), that a personal key without a project is refused,
metrics (retrieval latency, chunks returned), docs.
Built (`internal/router/rag.go`): `X-Llmgw-Knowledge` (header, as the
other llmgw extensions are, not a body field), `X-Llmgw-Knowledge-K`;
collections from the key's chain (kbEffective, shared with the kb
endpoints), 404 for one not readable; the question embedded through the
ordinary path on the caller's principal; SearchChunks; passages trimmed to
the model's context; added to the one leading system message (or a new
one); `x-llmgw-sources`; before guardInput; never cached; metrics. Left:
default collections per project or key (none yet: a client must name them);
`cache_salt` per organization (not sent: llama.cpp may refuse an unknown
field, and the router does not know the engine before choosing the
replica); the chat playground has no collection picker; returned passages
are not audited per request (the header and the kept request record them).
### F4. Adapter training (L, after F1–F3)
A run-to-completion job placed by the control plane like a replica and
supervised by the agent like an engine, in a Python environment the agent