Ilker ManapandClaude Opus 5
832d0a652c
A team was given a budget of a million, the quota page showed it, and after sixty requests it read zero percent used. Not because the counting was wrong: cost comes from a price table, no model had a price, and every request therefore cost nothing. The number was a limit in appearance and bound on nothing. An empty price table is the ordinary case — self-hosted models have no per-token price, which the schema already says. Showing somebody a budget without mentioning it is not. The page says so, and says it precisely: rate and token limits are unaffected, because they count requests and tokens rather than money. A warning that read as "no quota works here" would be worse than the silence it replaces — those two were enforcing perfectly well, as twenty served and ten refused had just shown. AnythingPriced is on the accountant rather than the price table, because what a caller is asking is whether the budgets it is about to display mean anything, and the accountant is what would enforce them. A model priced at zero is not priced; a fallback price prices everything. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
98 lines
3.0 KiB
Go
98 lines
3.0 KiB
Go
// Package accounting turns finished requests into spend, enforces what a key
|
|
// is allowed, and gets both into the store without touching the request path
|
|
// (plan §7.9.3, §7.9.4).
|
|
package accounting
|
|
|
|
import (
|
|
"strings"
|
|
"sync"
|
|
)
|
|
|
|
// Price is what one model costs, in micro-units per thousand tokens.
|
|
//
|
|
// Micro-units rather than a float because spend is accumulated by repeated
|
|
// addition, and repeated addition of floats drifts. What a unit means is the
|
|
// operator's: self-hosted models have no per-token price, so it may be
|
|
// GPU-seconds, a notional rate, or 1000 everywhere — which turns a budget into
|
|
// a token quota.
|
|
type Price struct {
|
|
PromptPer1k int64 `koanf:"prompt_per_1k"`
|
|
CompletionPer1k int64 `koanf:"completion_per_1k"`
|
|
}
|
|
|
|
// Zero reports whether this model spends nothing.
|
|
//
|
|
// It matters beyond arithmetic: a zero-cost model stays available to a key
|
|
// whose budget is spent, which is what makes the plan's budget fallback work
|
|
// (§7.7).
|
|
func (p Price) Zero() bool { return p.PromptPer1k == 0 && p.CompletionPer1k == 0 }
|
|
|
|
// Cost computes what a request spent, rounding up.
|
|
//
|
|
// Up rather than to nearest: rounding down would make a stream of tiny requests
|
|
// free, which is exactly the shape of traffic a quota exists to bound.
|
|
func (p Price) Cost(promptTokens, completionTokens int64) int64 {
|
|
return divUp(promptTokens*p.PromptPer1k, 1000) + divUp(completionTokens*p.CompletionPer1k, 1000)
|
|
}
|
|
|
|
func divUp(n, d int64) int64 {
|
|
if n <= 0 {
|
|
return 0
|
|
}
|
|
return (n + d - 1) / d
|
|
}
|
|
|
|
// Pricing resolves a model name to its price.
|
|
type Pricing struct {
|
|
mu sync.RWMutex
|
|
byModel map[string]Price
|
|
fallback Price
|
|
}
|
|
|
|
// NewPricing builds a price table. Names are matched case-insensitively,
|
|
// because a model name reaches us from whatever the client typed.
|
|
func NewPricing(byModel map[string]Price, fallback Price) *Pricing {
|
|
table := make(map[string]Price, len(byModel))
|
|
for name, p := range byModel {
|
|
table[strings.ToLower(name)] = p
|
|
}
|
|
return &Pricing{byModel: table, fallback: fallback}
|
|
}
|
|
|
|
// For returns a model's price, or the fallback.
|
|
func (p *Pricing) For(model string) Price {
|
|
p.mu.RLock()
|
|
defer p.mu.RUnlock()
|
|
if v, ok := p.byModel[strings.ToLower(model)]; ok {
|
|
return v
|
|
}
|
|
return p.fallback
|
|
}
|
|
|
|
// AnythingPriced reports whether any model costs anything.
|
|
//
|
|
// A budget is spend per period, and spend is computed from a price table. With
|
|
// no prices every request costs nothing, so a budget written on a team is a
|
|
// number that can never move — displayed as a limit, reported as zero percent
|
|
// used, and binding on nothing.
|
|
//
|
|
// Self-hosted models have no per-token price, so an empty table is the ordinary
|
|
// case rather than a misconfiguration. What is not ordinary is showing somebody
|
|
// a budget without saying so.
|
|
func (p *Pricing) AnythingPriced() bool {
|
|
if p == nil {
|
|
return false
|
|
}
|
|
p.mu.RLock()
|
|
defer p.mu.RUnlock()
|
|
if !p.fallback.Zero() {
|
|
return true
|
|
}
|
|
for _, price := range p.byModel {
|
|
if !price.Zero() {
|
|
return true
|
|
}
|
|
}
|
|
return false
|
|
}
|