Chat requests answered from knowledge bases

A request naming collections in X-Llmgw-Knowledge has its last question
embedded on the caller's account, the best passages found and added to the
system message, numbered, with an instruction to cite them and to say when
they do not hold the answer; x-llmgw-sources says which were given. Done
before the guardrails judge the request, never served from the answer cache,
and a collection the key may not read is answered as one that does not
exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 15:27:50 +02:00
1 parent 65116064d2
commit 496625824b
10 files changed
+653 -28

No files matched your search

+5
View File
@@ -251,6 +251,11 @@ func reusable(req *parsedRequest, anyAnswer bool) bool {
if req.noStore || req.ThreadID != "" || req.ContentType != "" || !cacheEndpoints[req.Path] {
return false
}
// An answer from knowledge bases is the caller's own: two keys of one
// project can read different collections.
if req.Header.Get(knowledgeHeader) != "" {
return false
}
if v := strings.ToLower(strings.TrimSpace(req.Header.Get(cacheHeader))); v == "no" || v == "off" || v == "bypass" {
return false
}