Chat requests answered from knowledge bases
A request naming collections in X-Llmgw-Knowledge has its last question embedded on the caller's account, the best passages found and added to the system message, numbered, with an instruction to cite them and to say when they do not hold the answer; x-llmgw-sources says which were given. Done before the guardrails judge the request, never served from the answer cache, and a collection the key may not read is answered as one that does not exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
65116064d2
commit
496625824b
10 files changed
+653
-28
No files matched your search
@@ -251,6 +251,11 @@ func reusable(req *parsedRequest, anyAnswer bool) bool {
|
||||
if req.noStore || req.ThreadID != "" || req.ContentType != "" || !cacheEndpoints[req.Path] {
|
||||
return false
|
||||
}
|
||||
// An answer from knowledge bases is the caller's own: two keys of one
|
||||
// project can read different collections.
|
||||
if req.Header.Get(knowledgeHeader) != "" {
|
||||
return false
|
||||
}
|
||||
if v := strings.ToLower(strings.TrimSpace(req.Header.Get(cacheHeader))); v == "no" || v == "off" || v == "bypass" {
|
||||
return false
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user