auth.gpu_hour_cost (and core_hour_cost) is what an hour of a GPU (a core) costs the operator. The leader keeps, by the hour, the GPU- and core-hours each model's replicas hold (held_1h); llmgw ctl prices suggest spreads a model's over the tokens it served, an output token counted as four input ones and a cached prompt token as a tenth of one, and --apply writes the prices into auth.pricing. Idle hours are in the cost, so a model held and little used comes out dear; one held and billed nothing is shown with what it held. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b8d3c084c5
commit
8547528fb7
22 files changed
+834
-1
No files matched your search
@@ -100,6 +100,10 @@ type API struct {
|
||||
ctl *control.Controller
|
||||
// slurmUsers are the Linux users made from the accounts; nil without a store.
|
||||
slurmUsers SlurmUsers
|
||||
// prices are suggested from what each model held and served; nil
|
||||
// without a store.
|
||||
prices PriceStore
|
||||
priceCfg func() PriceConfig
|
||||
// webhooks counts the settings' notify.webhooks; nil does not say.
|
||||
webhooks func() int
|
||||
}
|
||||
@@ -143,6 +147,7 @@ func (a *API) Register(mux Mux) {
|
||||
a.registerTenancy(mux)
|
||||
a.registerTrain(mux)
|
||||
a.registerSlurm(mux)
|
||||
a.registerPrices(mux)
|
||||
a.registerGuard(mux)
|
||||
a.registerSettings(mux)
|
||||
a.registerAuth(mux)
|
||||
|
||||
Reference in new issue
Block a user