Ilker ManapandClaude Opus 5.5
3a3db8cd20
CI / test (push) Successful in 8m52s
llmgw keys cache --similar 0.95 also gives a chat question the answer to one like it: the last user message embedded by router.cache.embedding_model (asked on the gateway's account, as a judge is, within 2 s), compared by cosine with the questions answered in the same conversation — system prompt, earlier turns, tools and sampling the same. The exact cache is asked first; an exact hit embeds nothing. Said as x-llmgw-cache: similar with x-llmgw-cache-similarity, charged as a hit. The vectors live with their answers in the gateway's memory, counted in max_bytes and evicted with them, not in a pgvector table as BACKLOG I1 planned: the cache keeps answer text out of the store on purpose. A question over 2,000 characters is reused only when the same. Migration 0051 holds the setting. BACKLOG I1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
7 lines
415 B
SQL
7 lines
415 B
SQL
-- Ported from the SQLite migration of the same number.
|
|
-- Answers to similar questions, for a project that asks: the cosine
|
|
-- similarity a question must reach to another's to be given its answer.
|
|
-- 0 is off, and only the same request is answered again. The questions and
|
|
-- answers are held in each gateway's memory, never here.
|
|
ALTER TABLE projects ADD COLUMN cache_similar DOUBLE PRECISION NOT NULL DEFAULT 0;
|