A cached prompt costs the cached rate

cached_per_1k in a price charges the prompt tokens the answer counts as
cached_tokens at their own rate; zero keeps charging them as prompt tokens.
The count reaches the cost and not a column: CostMicros holds what it
changed. Cached beyond the prompt is clamped, an engine's miscount rather
than a refund. BACKLOG L5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-25 17:15:48 +01:00
1 parent 73e1bb6733
commit ed0d1ec0ad
12 files changed
+85 -11

No files matched your search

+5 -1
View File
@@ -391,7 +391,11 @@ by itself. Three things are worth knowing before you edit it:
- `default_rpm`, `default_tpm` and `default_price` are what make limits and
budgets mean anything. Without a price every request costs zero and a budget
never binds; without the defaults, a project created without limits is
metered by nothing at all.
metered by nothing at all. A price may set `cached_per_1k` beside
`prompt_per_1k`: the prompt tokens the engine or provider read from its
cache (the answer's `cached_tokens`) are charged at it, the rest at the
prompt rate. A provider that bills cached reads at a tenth is
`{prompt_per_1k: 2500, cached_per_1k: 250, completion_per_1k: 10000}`.
- `control.enabled: true` is what makes this a control plane while its catalog
is empty. Without it, a file with no models is refused as having nothing to
serve.
+19
View File
@@ -424,6 +424,7 @@ rather than repeated.
| `secret_file` | `string` | [Webhook](#webhook) |
| `prompt_per_1k` | `int64` | [accounting.Price](#accountingprice) |
| `completion_per_1k` | `int64` | [accounting.Price](#accountingprice) |
| `cached_per_1k` | `int64` | [accounting.Price](#accountingprice) |
| `per_image` | `int64` | [accounting.Price](#accountingprice) |
| `per_mesh` | `int64` | [accounting.Price](#accountingprice) |
| `name` | `string` | [admission.Band](#admissionband) |
@@ -462,6 +463,7 @@ rather than repeated.
| `queue_ttl` | `time.Duration` | [config.Band](#configband) |
| `prompt_per_1k` | `int64` | [config.Price](#configprice) |
| `completion_per_1k` | `int64` | [config.Price](#configprice) |
| `cached_per_1k` | `int64` | [config.Price](#configprice) |
| `per_image` | `int64` | [config.Price](#configprice) |
| `per_mesh` | `int64` | [config.Price](#configprice) |
| `disabled` | `[]string` | [engines.Config](#enginesconfig) |
@@ -3600,6 +3602,15 @@ is sha256=<hex HMAC of the body>.
`int64`
### `cached_per_1k`
`int64`
CachedPer1k is what a prompt token costs that the engine or provider
read from its cache instead of computing: the part of prompt_tokens
the answer counts as cached_tokens. Zero charges those at
PromptPer1k, as before there was a rate for them.
### `per_image`
`int64`
@@ -3874,6 +3885,14 @@ had rather wait behind people than be turned away.
`int64`
### `cached_per_1k`
`int64`
CachedPer1k is the cost of a prompt token the engine or provider read
from its cache (the answer's cached_tokens, counted within the
prompt's). Zero charges them at prompt_per_1k.
### `per_image`
`int64`