Chat requests answered from knowledge bases
A request naming collections in X-Llmgw-Knowledge has its last question embedded on the caller's account, the best passages found and added to the system message, numbered, with an instruction to cite them and to say when they do not hold the answer; x-llmgw-sources says which were given. Done before the guardrails judge the request, never served from the answer cache, and a collection the key may not read is answered as one that does not exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
65116064d2
commit
496625824b
10 files changed
+653
-28
No files matched your search
+14
-1
@@ -946,7 +946,7 @@ PostgreSQL tests of store, router and adminapi share LLMGW_TEST_POSTGRES and
|
||||
the store's TRUNCATE each test; run them with `go test -p 1` or one package
|
||||
at a time, or a router test loses its rows mid-run.
|
||||
|
||||
### F3. Retrieval on the request path (M)
|
||||
### F3. Retrieval on the request path (M) — done 2026-10-08 (branch `rag`), but defaults and cache_salt
|
||||
- `llmgw.knowledge: [collections]` in the body (stripped before the
|
||||
engine), or default collections per project/key.
|
||||
- Allowed set from the caller's chain (§7.9.2), cached like limits; grants
|
||||
@@ -966,6 +966,19 @@ at a time, or a router test loses its rows mid-run.
|
||||
(including via cache), that a personal key without a project is refused,
|
||||
metrics (retrieval latency, chunks returned), docs.
|
||||
|
||||
Built (`internal/router/rag.go`): `X-Llmgw-Knowledge` (header, as the
|
||||
other llmgw extensions are, not a body field), `X-Llmgw-Knowledge-K`;
|
||||
collections from the key's chain (kbEffective, shared with the kb
|
||||
endpoints), 404 for one not readable; the question embedded through the
|
||||
ordinary path on the caller's principal; SearchChunks; passages trimmed to
|
||||
the model's context; added to the one leading system message (or a new
|
||||
one); `x-llmgw-sources`; before guardInput; never cached; metrics. Left:
|
||||
default collections per project or key (none yet: a client must name them);
|
||||
`cache_salt` per organization (not sent: llama.cpp may refuse an unknown
|
||||
field, and the router does not know the engine before choosing the
|
||||
replica); the chat playground has no collection picker; returned passages
|
||||
are not audited per request (the header and the kept request record them).
|
||||
|
||||
### F4. Adapter training (L, after F1–F3)
|
||||
A run-to-completion job placed by the control plane like a replica and
|
||||
supervised by the agent like an engine, in a Python environment the agent
|
||||
|
||||
Reference in new issue
Block a user