auth.pricing takes per_image, charged on top of a prompt's tokens; a model
priced only per picture is not free, so a spent budget still stops it.
Each usage row keeps how many pictures were made (migration 0041), and
ctl request and the admin API's request view show them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A team was given a budget of a million, the quota page showed it, and after
sixty requests it read zero percent used. Not because the counting was wrong:
cost comes from a price table, no model had a price, and every request
therefore cost nothing. The number was a limit in appearance and bound on
nothing.
An empty price table is the ordinary case — self-hosted models have no
per-token price, which the schema already says. Showing somebody a budget
without mentioning it is not.
The page says so, and says it precisely: rate and token limits are unaffected,
because they count requests and tokens rather than money. A warning that read
as "no quota works here" would be worse than the silence it replaces — those
two were enforcing perfectly well, as twenty served and ten refused had just
shown.
AnythingPriced is on the accountant rather than the price table, because what a
caller is asking is whether the budgets it is about to display mean anything,
and the accountant is what would enforce them. A model priced at zero is not
priced; a fallback price prices everything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The second half of phase 5. A key now has a cost, a quota and a bill.
Rate limits use a sliding window rather than a fixed one. A fixed window lets
a caller send its whole minute's allowance in the last second of one window
and again in the first second of the next, so the observed rate is twice the
configured limit exactly when the system is least able to take it. Checking
and charging are one operation, or two concurrent requests both see room and
both take it.
Token limits are checked against an estimate, because the real count only
exists once the request has run; the difference is charged afterwards, so a
caller who consistently underestimates is still bounded, one window late. A
request with no max_tokens gets a modest assumption rather than zero —
assuming zero would let an unbounded request through every token limit.
Limits are checked before admission, not after: a request over its quota
should be refused rather than queued, where it would occupy a band's capacity
to no purpose. All refusals are 429, but a spent budget carries no Retry-After
because waiting does not help until the period rolls over, and saying
otherwise turns one client into a retry loop.
Usage goes to a buffered channel and is written in batches. Record never
blocks: if the buffer fills — a database stalled long enough to lose eight
thousand records — records are dropped and counted, because losing accounting
is bad and turning a stalled database into an outage is worse. A clean
shutdown drains what is queued; those are requests that were already served.
Failures are logged at error, since silently losing billing data is the kind
of thing discovered at the end of a month.
Requests are accounted whatever their outcome. A request that used a GPU and
then failed still used it, and a key that could spend nothing by failing would
be a way around a budget.
Budgets are counted in memory and restored from the store at startup, so a
restart does not hand every key a fresh allowance. The budget period travels
with the reservation rather than being re-derived, because a monthly budget
credited to a daily counter is a budget that never binds.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>