Ilker ManapandClaude Opus 5.5 3a3db8cd20 Answers to similar questions, not only the same one ...

llmgw keys cache --similar 0.95 also gives a chat question the answer to
one like it: the last user message embedded by router.cache.embedding_model
(asked on the gateway's account, as a judge is, within 2 s), compared by
cosine with the questions answered in the same conversation — system
prompt, earlier turns, tools and sampling the same. The exact cache is
asked first; an exact hit embeds nothing. Said as x-llmgw-cache: similar
with x-llmgw-cache-similarity, charged as a hit.

The vectors live with their answers in the gateway's memory, counted in
max_bytes and evicted with them, not in a pgvector table as BACKLOG I1
planned: the cache keeps answer text out of the store on purpose. A
question over 2,000 characters is reused only when the same. Migration
0051 holds the setting. BACKLOG I1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

2026-10-25 17:47:11 +01:00

..

migrations

Answers to similar questions, not only the same one

2026-10-25 17:47:11 +01:00

answercache_test.go

Answers to similar questions, not only the same one

2026-10-25 17:47:11 +01:00

answercache.go

Answers to similar questions, not only the same one

2026-10-25 17:47:11 +01:00

audit_test.go

Staff roles (viewer, operator, billing) and the console's Audit view

2026-10-04 16:30:56 +02:00

audit.go

Audit log: every change through the admin API and the administration pages, with ctl audit

2026-10-04 16:25:56 +02:00

batches_test.go

Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key

2026-10-04 23:21:44 +02:00

batches.go

Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it

2026-10-04 21:00:01 +02:00

capacity_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

capacity.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

claims_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

claims.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

clusterstate.go

The firewall setting reaches the mirror machine and the store's members

2026-10-09 08:53:17 +02:00

controlplanes.go

Add a standby control plane with server add, and draw the screens as windows

2026-10-03 22:29:12 +02:00

copy_test.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

copy.go

The store keeps the company's MCP servers, and which tools a scope may call

2026-10-20 10:36:08 +02:00

dialect_test.go

Name people and projects in usage reports, and total a usage legend

2026-09-28 22:37:21 +02:00

dialect.go

Name people and projects in usage reports, and total a usage legend

2026-09-28 22:37:21 +02:00

directory.go

People sign in with their Active Directory account

2026-10-18 10:59:50 +02:00

eval_test.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

eval.go

Quality suites, and new weights held to one: ctl eval, model update/promote --eval, model gate

2026-10-05 00:04:43 +02:00

export_test.go

The usage sums and exports count pictures

2026-10-10 10:36:19 +02:00

export.go

A ComfyUI model may make meshes: from a picture, with its licence inside

2026-10-10 18:30:52 +02:00

facility_test.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

facility.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

federation_test.go

Federation: another llmgw cluster as a provider, its models as PEER/MODEL

2026-10-05 01:21:30 +02:00

federation.go

Federation: another llmgw cluster as a provider, its models as PEER/MODEL

2026-10-05 01:21:30 +02:00

forget_test.go

Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster

2026-10-04 23:27:31 +02:00

forget.go

Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster

2026-10-04 23:27:31 +02:00

guard_test.go

The /admin/guard page, a Guardrails link, and the policies in force on the profile page

2026-10-04 15:15:38 +02:00

guard.go

Guardrails turn off and on; a judge's refusal says which hazard; ctl guard judge ask

2026-10-06 12:41:08 +02:00

hierarchy_test.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

hierarchy.go

Knowledge bases in the store: collections, grants and chunks kept apart per tenant

2026-10-08 12:47:57 +02:00

jobs_test.go

Jobs run side by side, one per model at a time

2026-10-14 10:06:20 +02:00

jobs.go

Jobs run side by side, one per model at a time

2026-10-14 10:06:20 +02:00

keys_test.go

Add the store: tenancy, keys and usage on SQLite

2026-09-16 08:37:40 +02:00

keys.go

Add the store: tenancy, keys and usage on SQLite

2026-09-16 08:37:40 +02:00

keyttl_test.go

Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing

2026-10-04 16:45:05 +02:00

keyttl.go

Key lifecycle: lifetime limits by scope, rotation with overlap, kinds and expiry in the listing

2026-10-04 16:45:05 +02:00

keyusage_test.go

Name people and projects in usage reports, and total a usage legend

2026-09-28 22:37:21 +02:00

knowledge_test.go

Knowledge bases at scale: halfvec, a text index per collection, ef_search raised to the candidates wanted

2026-10-08 22:08:29 +02:00

knowledge.go

Knowledge bases at scale: halfvec, a text index per collection, ef_search raised to the candidates wanted

2026-10-08 22:08:29 +02:00

latency_test.go

Show each team's first-token and latency percentiles, errors and refusals

2026-10-03 19:50:16 +02:00

latency.go

Show each team's first-token and latency percentiles, errors and refusals

2026-10-03 19:50:16 +02:00

leadership_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

leadership.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

licences_test.go

A person leaves licences out, and no key of theirs reaches a model under one

2026-10-11 16:10:22 +02:00

licences.go

A person leaves licences out, and no key of theirs reaches a model under one

2026-10-11 16:10:22 +02:00

limits_owner_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

mcp_test.go

The store keeps the company's MCP servers, and which tools a scope may call

2026-10-20 10:36:08 +02:00

mcp.go

The company's MCP tools under the guardrails, a rate and a breaker

2026-10-20 23:00:22 +02:00

memberships_test.go

Refuse the keys of an account that is not active

2026-10-04 10:15:32 +02:00

memberships.go

A person leaves licences out, and no key of theirs reaches a model under one

2026-10-11 16:10:22 +02:00

memory_test.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

memory.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

migrate0010_test.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

migrate.go

Let a person belong to many places, or to none

2026-09-28 19:03:46 +02:00

models.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

nodeengines_test.go

Give each GPU node a vLLM build it can run, chosen on the cluster page

2026-10-01 12:39:14 +02:00

nodeengines.go

Give each GPU node a vLLM build it can run, chosen on the cluster page

2026-10-01 12:39:14 +02:00

notify_test.go

Two control planes send a notification once

2026-10-10 13:36:08 +02:00

notify.go

Two control planes send a notification once

2026-10-10 13:36:08 +02:00

open_test.go

Run the store on PostgreSQL, chosen by what the address looks like

2026-09-25 16:18:10 +02:00

open.go

Secrets in Vault: env: and vault: references wherever a secret file is named, and the store's own secrets sealed under a master key — Vault transit or a local key

2026-10-04 23:21:44 +02:00

pghosts.go

Set up a replicated store with server init --patroni, and watch it

2026-10-03 20:57:41 +02:00

postgres_failfast_test.go

A lost store machine is given up on in seconds, and a new leader waits for its judge

2026-10-15 20:33:13 +02:00

postgres_memberships_test.go

Give an organization's managers a page of their own

2026-09-28 19:23:12 +02:00

postgres_test.go

Merge C24 federation: another llmgw cluster as a provider

2026-10-05 01:29:35 +02:00

postgres.go

A lost store machine is given up on in seconds, and a new leader waits for its judge

2026-10-15 20:33:13 +02:00

presets_test.go

The leader adds the picture and 3D presets a node fits, unless told not to

2026-10-13 20:48:40 +02:00

presets.go

The leader adds the picture and 3D presets a node fits, unless told not to

2026-10-13 20:48:40 +02:00

quotas_test.go

Guarantee and limit capacity per part of the tenancy tree

2026-09-30 18:15:38 +02:00

quotas.go

Guarantee and limit capacity per part of the tenancy tree

2026-09-30 18:15:38 +02:00

requestlog.go

Forgetting a person and keeping requests in the cluster: users forget, keys residency --in-cluster

2026-10-04 23:27:31 +02:00

resets_test.go

A forgotten password is set through a one-time link

2026-10-07 13:20:11 +02:00

resets.go

A forgotten password is set through a one-time link

2026-10-07 13:20:11 +02:00

revoked_test.go

Revoke a removed node's certificate, and require --api-client with a mirror's mutual TLS

2026-10-04 10:34:35 +02:00

revoked.go

Revoke a removed node's certificate, and require --api-client with a mirror's mutual TLS

2026-10-04 10:34:35 +02:00

rollup_test.go

Read usage from sums kept as it is written, not from every request

2026-10-03 19:43:10 +02:00

rollup.go

A ComfyUI model may make meshes: from a picture, with its licence inside

2026-10-10 18:30:52 +02:00

seal.go

Onboard users with one-time links instead of pasted keys

2026-09-16 09:11:28 +02:00

secrets_test.go

Sign-in is set up from the console, and a directory tried before it is kept

2026-10-19 12:55:27 +02:00

secrets.go

Sign-in is set up from the console, and a directory tried before it is kept

2026-10-19 12:55:27 +02:00

selfusage_test.go

Show a person their own use and what is left of their budgets

2026-09-28 22:51:13 +02:00

selfusage.go

Read usage from sums kept as it is written, not from every request

2026-10-03 19:43:10 +02:00

sessions_test.go

End a session unused for a day, and purge dead sessions and claims hourly

2026-10-04 10:50:56 +02:00

sessions.go

Staff roles (viewer, operator, billing) and the console's Audit view

2026-10-04 16:30:56 +02:00

settings_test.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

settings.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

sqlite.go

The store keeps the company's MCP servers, and which tools a scope may call

2026-10-20 10:36:08 +02:00

store_test.go

Add a standby control plane with server add, and draw the screens as windows

2026-10-03 22:29:12 +02:00

store.go

A cached prompt costs the cached rate

2026-10-25 17:15:48 +01:00

tenancy_test.go

Let an admin build the tenancy tree, instead of sending them to a shell

2026-09-27 12:36:13 +02:00

tenancy.go

The store keeps the company's MCP servers, and which tools a scope may call

2026-10-20 10:36:08 +02:00

threads_test.go

Give projects somewhere to keep a conversation, off unless they ask

2026-09-18 11:05:34 +02:00

threads.go

Stop calling the PostgreSQL store SQLite, and stop guessing where it is

2026-09-27 12:20:40 +02:00

train.go

Trained adapters published, private to their owners like the knowledge bases they come from

2026-10-08 21:30:50 +02:00

unique_test.go

Recognise a duplicate sign-up on PostgreSQL, and say what to do

2026-10-01 12:15:37 +02:00

usagescope_test.go

Show what a group spent, and fix the reports that never ran on PostgreSQL

2026-09-27 12:26:14 +02:00

usagescope.go

Read usage from sums kept as it is written, not from every request

2026-10-03 19:43:10 +02:00

usageseries_test.go

Show usage over time, per organization, team, project and person

2026-09-28 08:11:31 +02:00

usageseries.go

Show each team's first-token and latency percentiles, errors and refusals

2026-10-03 19:50:16 +02:00

users.go

People sign in with their Active Directory account

2026-10-18 10:59:50 +02:00

usertimeout_linux.go

A lost store machine is given up on in seconds, and a new leader waits for its judge

2026-10-15 20:33:13 +02:00

usertimeout_other.go

A lost store machine is given up on in seconds, and a new leader waits for its judge

2026-10-15 20:33:13 +02:00