Prices from what the cluster costs: ctl prices suggest
CI / test (push) Successful in 9m51s

auth.gpu_hour_cost (and core_hour_cost) is what an hour of a GPU (a core)
costs the operator. The leader keeps, by the hour, the GPU- and core-hours
each model's replicas hold (held_1h); llmgw ctl prices suggest spreads a
model's over the tokens it served, an output token counted as four input
ones and a cached prompt token as a tenth of one, and --apply writes the
prices into auth.pricing. Idle hours are in the cost, so a model held and
little used comes out dear; one held and billed nothing is shown with what
it held.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-15 06:54:55 +01:00
1 parent b8d3c084c5
commit 8547528fb7
22 files changed
+834 -1

No files matched your search

+2
View File
@@ -39,6 +39,8 @@ var Reference = []Block{
{Key: "personal", Type: "Personal", Doc: "Personal is what a person may do on their own, outside any project: make\npersonal keys and use the chat page personally, within these limits.", Nested: "config.Personal"},
{Key: "pricing", Type: "map[string]Price", Doc: "Pricing is what each model costs, for budgets. A model absent from the\ntable costs whatever DefaultPrice says, which is nothing unless set.", Nested: ""},
{Key: "default_price", Type: "Price", Doc: "", Nested: "config.Price"},
{Key: "gpu_hour_cost", Type: "float64", Doc: "GPUHourCost is what an hour of one GPU costs the operator, in the\nunits budgets are in: the hardware written off over its years, its\npower and room, and the people who run it, over the GPUs and the\nhours of a month. With CoreHourCost for a CPU core, it is what\n`llmgw ctl prices suggest` prices each model from: the hours its\nreplicas held, idle ones too, over the tokens it served.", Nested: ""},
{Key: "core_hour_cost", Type: "float64", Doc: "", Nested: ""},
{Key: "flush_interval", Type: "time.Duration", Doc: "FlushInterval and BatchSize decide when usage records are written.", Nested: ""},
{Key: "batch_size", Type: "int", Doc: "", Nested: ""},
{Key: "buffer_size", Type: "int", Doc: "BufferSize bounds the queue. Beyond it records are dropped rather than\nblocking a request: losing accounting is bad, turning a stalled database\ninto an outage is worse.", Nested: ""},