auth.gpu_hour_cost (and core_hour_cost) is what an hour of a GPU (a core) costs the operator. The leader keeps, by the hour, the GPU- and core-hours each model's replicas hold (held_1h); llmgw ctl prices suggest spreads a model's over the tokens it served, an output token counted as four input ones and a cached prompt token as a tenth of one, and --apply writes the prices into auth.pricing. Idle hours are in the cost, so a model held and little used comes out dear; one held and billed nothing is shown with what it held. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b8d3c084c5
commit
8547528fb7
22 files changed
+834
-1
No files matched your search
@@ -43,6 +43,8 @@ rather than repeated.
|
||||
| `personal` | `Personal` | [Auth](#auth) |
|
||||
| `pricing` | `map[string]Price` | [Auth](#auth) |
|
||||
| `default_price` | `Price` | [Auth](#auth) |
|
||||
| `gpu_hour_cost` | `float64` | [Auth](#auth) |
|
||||
| `core_hour_cost` | `float64` | [Auth](#auth) |
|
||||
| `flush_interval` | `time.Duration` | [Auth](#auth) |
|
||||
| `batch_size` | `int` | [Auth](#auth) |
|
||||
| `buffer_size` | `int` | [Auth](#auth) |
|
||||
@@ -722,6 +724,21 @@ table costs whatever DefaultPrice says, which is nothing unless set.
|
||||
|
||||
Takes a [config.Price](#configprice) block.
|
||||
|
||||
### `gpu_hour_cost`
|
||||
|
||||
`float64`
|
||||
|
||||
GPUHourCost is what an hour of one GPU costs the operator, in the
|
||||
units budgets are in: the hardware written off over its years, its
|
||||
power and room, and the people who run it, over the GPUs and the
|
||||
hours of a month. With CoreHourCost for a CPU core, it is what
|
||||
`llmgw ctl prices suggest` prices each model from: the hours its
|
||||
replicas held, idle ones too, over the tokens it served.
|
||||
|
||||
### `core_hour_cost`
|
||||
|
||||
`float64`
|
||||
|
||||
### `flush_interval`
|
||||
|
||||
`time.Duration`
|
||||
|
||||
Reference in new issue
Block a user