Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 21:00:01 +02:00
1 parent 889be70b8f
commit 841ee21c12
19 files changed
+1775 -10

No files matched your search

+44
View File
@@ -43,6 +43,10 @@ rather than repeated.
| `batch_size` | `int` | [Auth](#auth) |
| `buffer_size` | `int` | [Auth](#auth) |
| `usage_retention` | `time.Duration` | [Auth](#auth) |
| `max_file_bytes` | `int64` | [Batch](#batch) |
| `max_lines` | `int` | [Batch](#batch) |
| `concurrency` | `int` | [Batch](#batch) |
| `retain_days` | `int` | [Batch](#batch) |
| `model` | `string` | [Canary](#canary) |
| `percent` | `int` | [Canary](#canary) |
| `min_requests` | `int` | [Canary](#canary) |
@@ -198,6 +202,7 @@ rather than repeated.
| `tenant_label` | `string` | [Router](#router) |
| `tag_labels` | `[]string` | [Router](#router) |
| `max_body_bytes` | `int64` | [Router](#router) |
| `batch` | `Batch` | [Router](#router) |
| `trusted_proxies` | `[]string` | [Router](#router) |
| `timeouts` | `Timeouts` | [Router](#router) |
| `scoring` | `Scoring` | [Router](#router) |
@@ -517,6 +522,35 @@ five-minute sums kept beside the records, which are not pruned, so this
bounds the table that grows by a row a request without changing any
figure a page shows. At least a day when set.
## Batch
### `max_file_bytes`
`int64`
MaxFileBytes bounds one uploaded file; 100 MB unless set.
### `max_lines`
`int`
MaxLines bounds the requests in one batch; 50 000 unless set.
### `concurrency`
`int`
Concurrency is how many of one batch's requests are in flight at
once; 4 unless set. They queue in the lowest admission band, behind
every interactive request.
### `retain_days`
`int`
RetainDays is how long a batch's answer and error files are kept;
30 unless set. Uploaded files are kept until deleted.
## Canary
### `model`
@@ -1622,6 +1656,16 @@ a series. The first 100 values of each are counted apart, the rest as
MaxBodyBytes bounds the request body the router will read.
### `batch`
`Batch`
Batch is the Files and Batch APIs (/v1/files, /v1/batches): a file of
requests served at the lowest priority, its answers written to
another. Needs the store.
Takes a [Batch](#batch) block.
### `trusted_proxies`
`[]string`