Merge C17 response cache

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 23:51:19 +02:00
commit f48f55a866
27 files changed
+2066 -11

No files matched your search

+29
View File
@@ -47,6 +47,8 @@ rather than repeated.
| `max_lines` | `int` | [Batch](#batch) |
| `concurrency` | `int` | [Batch](#batch) |
| `retain_days` | `int` | [Batch](#batch) |
| `max_bytes` | `int64` | [Cache](#cache) |
| `max_entry_bytes` | `int64` | [Cache](#cache) |
| `model` | `string` | [Canary](#canary) |
| `percent` | `int` | [Canary](#canary) |
| `min_requests` | `int` | [Canary](#canary) |
@@ -205,6 +207,7 @@ rather than repeated.
| `tag_labels` | `[]string` | [Router](#router) |
| `max_body_bytes` | `int64` | [Router](#router) |
| `batch` | `Batch` | [Router](#router) |
| `cache` | `Cache` | [Router](#router) |
| `trusted_proxies` | `[]string` | [Router](#router) |
| `timeouts` | `Timeouts` | [Router](#router) |
| `scoring` | `Scoring` | [Router](#router) |
@@ -571,6 +574,22 @@ every interactive request.
RetainDays is how long a batch's answer and error files are kept;
30 unless set. Uploaded files are kept until deleted.
## Cache
### `max_bytes`
`int64`
MaxBytes bounds every project's answers together; 128 MB unless set.
Past it, the answers used longest ago go first.
### `max_entry_bytes`
`int64`
MaxEntryBytes bounds one answer; a longer one is not kept. 1 MB
unless set.
## Canary
### `model`
@@ -1704,6 +1723,16 @@ another. Needs the store.
Takes a [Batch](#batch) block.
### `cache`
`Cache`
Cache bounds the answers kept for the projects that reuse them
(llmgw keys cache). Which projects do, and for how long, is theirs
to set; this is only how much memory it may take.
Takes a [Cache](#cache) block.
### `trusted_proxies`
`[]string`