On an organization, team, project or key, the narrowest winning: llmgw ctl
kb default, POST /api/v1/kb/defaults, and the console's Knowledge screen. A
request's own X-Llmgw-Knowledge overrides it and none turns it off. A default
never refuses a request: a collection the caller cannot read is left out, and
a failed search answers without passages. Removing a collection takes it out
of every default.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A request naming collections in X-Llmgw-Knowledge has its last question
embedded on the caller's account, the best passages found and added to the
system message, numbered, with an instruction to cite them and to say when
they do not hold the answer; x-llmgw-sources says which were given. Done
before the guardrails judge the request, never served from the answer cache,
and a collection the key may not read is answered as one that does not
exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
llmgw keys cache --project P --ttl 1h [--any] | --off, and Cache answers… on
the console's project menu. The same request — project, endpoint, canonical
model and the body without stream, stream_options, user, metadata, store and
service_tier, keys sorted — is answered from the gateway's memory for the
TTL. Only reproducible answers (temperature 0 or a seed; embeddings, rerank,
score) unless --any; only a whole 200 from the model asked for, not a
fallback's, a canary's, or one a guardrail changed.
Looked up after the limits and the input guardrails; the output policy
judges a reused answer; no_store and stored conversations bypass it.
Streamed answers are kept put back together and streamed back as chunks.
x-llmgw-cache: hit|miss|bypass, Age on a hit, Cache-Control: no-cache or
x-llmgw-cache: no to bypass. A hit is recorded with no tokens and no cost,
marked cache_hit (migration 0027) and shown by ctl request. An LRU bounded
by router.cache.max_bytes and max_entry_bytes; llmgw_cache_requests_total,
llmgw_cache_bytes, llmgw_cache_entries.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
