Batches: OpenAI's Files and Batch APIs, served at the lowest priority under a store lease, each line a request of the key that made it
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
889be70b8f
commit
841ee21c12
19 files changed
+1775
-10
No files matched your search
@@ -43,6 +43,10 @@ rather than repeated.
|
||||
| `batch_size` | `int` | [Auth](#auth) |
|
||||
| `buffer_size` | `int` | [Auth](#auth) |
|
||||
| `usage_retention` | `time.Duration` | [Auth](#auth) |
|
||||
| `max_file_bytes` | `int64` | [Batch](#batch) |
|
||||
| `max_lines` | `int` | [Batch](#batch) |
|
||||
| `concurrency` | `int` | [Batch](#batch) |
|
||||
| `retain_days` | `int` | [Batch](#batch) |
|
||||
| `model` | `string` | [Canary](#canary) |
|
||||
| `percent` | `int` | [Canary](#canary) |
|
||||
| `min_requests` | `int` | [Canary](#canary) |
|
||||
@@ -198,6 +202,7 @@ rather than repeated.
|
||||
| `tenant_label` | `string` | [Router](#router) |
|
||||
| `tag_labels` | `[]string` | [Router](#router) |
|
||||
| `max_body_bytes` | `int64` | [Router](#router) |
|
||||
| `batch` | `Batch` | [Router](#router) |
|
||||
| `trusted_proxies` | `[]string` | [Router](#router) |
|
||||
| `timeouts` | `Timeouts` | [Router](#router) |
|
||||
| `scoring` | `Scoring` | [Router](#router) |
|
||||
@@ -517,6 +522,35 @@ five-minute sums kept beside the records, which are not pruned, so this
|
||||
bounds the table that grows by a row a request without changing any
|
||||
figure a page shows. At least a day when set.
|
||||
|
||||
## Batch
|
||||
|
||||
### `max_file_bytes`
|
||||
|
||||
`int64`
|
||||
|
||||
MaxFileBytes bounds one uploaded file; 100 MB unless set.
|
||||
|
||||
### `max_lines`
|
||||
|
||||
`int`
|
||||
|
||||
MaxLines bounds the requests in one batch; 50 000 unless set.
|
||||
|
||||
### `concurrency`
|
||||
|
||||
`int`
|
||||
|
||||
Concurrency is how many of one batch's requests are in flight at
|
||||
once; 4 unless set. They queue in the lowest admission band, behind
|
||||
every interactive request.
|
||||
|
||||
### `retain_days`
|
||||
|
||||
`int`
|
||||
|
||||
RetainDays is how long a batch's answer and error files are kept;
|
||||
30 unless set. Uploaded files are kept until deleted.
|
||||
|
||||
## Canary
|
||||
|
||||
### `model`
|
||||
@@ -1622,6 +1656,16 @@ a series. The first 100 values of each are counted apart, the rest as
|
||||
|
||||
MaxBodyBytes bounds the request body the router will read.
|
||||
|
||||
### `batch`
|
||||
|
||||
`Batch`
|
||||
|
||||
Batch is the Files and Batch APIs (/v1/files, /v1/batches): a file of
|
||||
requests served at the lowest priority, its answers written to
|
||||
another. Needs the store.
|
||||
|
||||
Takes a [Batch](#batch) block.
|
||||
|
||||
### `trusted_proxies`
|
||||
|
||||
`[]string`
|
||||
|
||||
Reference in new issue
Block a user