auth.gpu_hour_cost (and core_hour_cost) is what an hour of a GPU (a core) costs the operator. The leader keeps, by the hour, the GPU- and core-hours each model's replicas hold (held_1h); llmgw ctl prices suggest spreads a model's over the tokens it served, an output token counted as four input ones and a cached prompt token as a tenth of one, and --apply writes the prices into auth.pricing. Idle hours are in the cost, so a model held and little used comes out dear; one held and billed nothing is shown with what it held. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
b8d3c084c5
commit
8547528fb7
22 files changed
+834
-1
No files matched your search
@@ -408,6 +408,13 @@ var Routes = []Route{
|
||||
"all away. The machines have them within a minute.",
|
||||
Request: SlurmKeysRequest{}, Response: slurmsvc.User{}},
|
||||
|
||||
{Method: "GET", Path: "/api/v1/prices/suggest", Summary: "Each model's price, from what it held of the cluster",
|
||||
Description: "days (30 if not given) is the window. A model's cost is the GPU- and core-hours its replicas held, idle " +
|
||||
"ones too, at auth.gpu_hour_cost and auth.core_hour_cost; spread over its tokens, an output token four input " +
|
||||
"ones and a cached prompt token a tenth of one, it is price, in micro-units a thousand tokens as " +
|
||||
"auth.pricing takes them. current is the price in force. 409 with no cost set.",
|
||||
Response: PriceSuggestions{}},
|
||||
|
||||
{Method: "POST", Path: "/api/v1/train/jobs", Summary: "Queue a LoRA adapter's training on a node",
|
||||
Description: "The node trains when its agent is there, nothing else trains on it, and a card has gpu_memory_mib free; " +
|
||||
"the pairs are NDJSON of {\"q\", \"a\"}. An adapter learns how answers sound far better than what they say.",
|
||||
|
||||
Reference in new issue
Block a user