## CHANGELOG ### Reusing answers - **A response cache, per project, off until asked for.** `llmgw keys cache --project P --ttl 1h` has the same request — same project, endpoint, model by its catalog name, and body with `stream`, `stream_options`, `user`, `metadata`, `store` and `service_tier` left out and its keys in any order — answered from the gateway's memory instead of the model, for the TTL. `--off` stops it; without either it says how it stands. The console has it as **Cache answers…** on a project. - **Only answers that come out the same are reused:** chat and completions asked at temperature 0 or with a seed, and embeddings, rerank and score. `--any` reuses sampled answers too. Only a whole 200 from the model asked for is kept — not an error, a fallback's or a canary's answer, one a guardrail changed or stopped, or one over `router.cache.max_entry_bytes`. - **Streams both ways:** a streamed answer is kept put back together into one completion (tool calls and reasoning included), and a kept answer is streamed as chunks — with the usage chunk if the client asked — to a client that streams. `/v1/messages` and `/v1/responses` get it too. - **Headers:** `x-llmgw-cache: hit|miss|bypass` on every answer of a project that reuses answers, `Age` on a hit. `Cache-Control: no-cache` or `no-store`, or `x-llmgw-cache: no`, asks the model anew. - **Policies still apply:** limits and the input guardrails run before the cache is looked in, the output policy judges a reused answer as it would a new one, and a request under `no_store` or continuing a stored conversation is never served from it nor kept in it. A project's answers are never looked up by another project's keys. - **A hit costs nothing:** recorded as a 200 with no tokens and no cost, marked `cache_hit` on the usage record (migration 0027), and shown by `llmgw ctl request`; it counts towards the requests-per-minute limit only. - **In memory only:** `router.cache.max_bytes` (128 MB) for all projects together in each gateway process, least recently used dropped first; nothing is written to the store. `llmgw_cache_requests_total{model,result}`, `llmgw_cache_bytes` and `llmgw_cache_entries`. ## BACKLOG **Done (branch `features-c2`):** exact-match answer cache per project (`keys cache --ttl/--any/--off`, console **Cache answers…**, `projects` columns, 30-second setting cache), reproducible answers only unless `--any`, LRU in memory bounded by `router.cache.max_bytes` and `max_entry_bytes`, streamed answers kept and replayed as chunks, `x-llmgw-cache` and `Cache-Control: no-cache`, guardrails before a hit and the output policy on it, `no_store` and threads bypass, a hit recorded as `cache_hit` with no tokens and no cost (migration 0027), metrics, docs. Left: semantic (embedding-similarity) caching through the embeddings models; a cache shared across control planes (each process keeps its own, deliberately: a shared one would write every answer to the store); a hit still waits its turn in admission, because the guardrails judge the request after admission and the key is the request they let through; logprobs in a streamed answer leave it unkept; `users forget` does not reach into the cache (answers age out by the TTL); no listing or flushing of what is cached (`--off` stops it being served at once, the memory is freed by TTL and LRU).