Requests kept and replayed: keys requests per project with retention and a share, ctl request --body and --replay

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 22:51:21 +02:00
1 parent 841ee21c12
commit 32ff1dbd6f
22 files changed
+757 -10

No files matched your search

+23
View File
@@ -1130,6 +1130,29 @@ curl http://gateway:8080/v1/chat/completions \
100 values of each apart and the rest as `other`: a tag is the client's
to set, and every value is a series.
## Keeping requests, and replaying them
To follow a request up — what exactly was sent, what came back — or to try
real traffic on another model, a project can keep its requests:
```
llmgw keys requests --project Acme/ml/train --retain-hours 72 # every one
llmgw keys requests --project Acme/ml/train --retain-hours 168 --sample 0.05
llmgw ctl request req_3f2a… --body # what was sent and answered
llmgw ctl request req_3f2a… --replay qwen2.5-14b-instruct # the same, to another model
llmgw keys requests --project Acme/ml/train --off
```
- **What is kept:** the request as the model was sent it — after the
guardrails' redaction, with any stored conversation spliced in — and the
answer as the client got it, up to 128 KB of it. A share (`--sample`) is
the same requests every time for the same ids.
- **Replay** sends it as the gateway itself: past the limits, the budget and
the guardrails, which judged it the first time, and charged to nobody.
- **Never** for a project under a guardrails policy with `no_store`. Kept
requests past their hours, or of a project turned off, are removed hourly.
The console has it as **Keep requests…** on a project.
## Batches
Work that can wait — a night's classification, embeddings for a document