auth.gpu_hour_cost (and core_hour_cost) is what an hour of a GPU (a core) costs the operator. The leader keeps, by the hour, the GPU- and core-hours each model's replicas hold (held_1h); llmgw ctl prices suggest spreads a model's over the tokens it served, an output token counted as four input ones and a cached prompt token as a tenth of one, and --apply writes the prices into auth.pricing. Idle hours are in the cost, so a model held and little used comes out dear; one held and billed nothing is shown with what it held. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed
1 parent
b8d3c084c5
commit
8547528fb7
22 files changed
+834
-1
No files matched your search
@@ -95,6 +95,9 @@
|
||||
{
|
||||
"name": "presets"
|
||||
},
|
||||
{
|
||||
"name": "prices"
|
||||
},
|
||||
{
|
||||
"name": "projects"
|
||||
},
|
||||
@@ -4396,6 +4399,34 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/prices/suggest": {
|
||||
"get": {
|
||||
"operationId": "getPricesSuggest",
|
||||
"summary": "Each model's price, from what it held of the cluster",
|
||||
"description": "days (30 if not given) is the window. A model's cost is the GPU- and core-hours its replicas held, idle ones too, at auth.gpu_hour_cost and auth.core_hour_cost; spread over its tokens, an output token four input ones and a cached prompt token a tenth of one, it is price, in micro-units a thousand tokens as auth.pricing takes them. current is the price in force. 409 with no cost set.",
|
||||
"tags": [
|
||||
"prices"
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"description": "OK",
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/PriceSuggestions"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"401": {
|
||||
"$ref": "#/components/responses/Unauthorized"
|
||||
},
|
||||
"default": {
|
||||
"$ref": "#/components/responses/Error"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"/api/v1/projects": {
|
||||
"post": {
|
||||
"operationId": "postProjects",
|
||||
@@ -8846,6 +8877,73 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"PriceSuggestion": {
|
||||
"type": "object",
|
||||
"description": "PriceSuggestion is one model's.",
|
||||
"properties": {
|
||||
"model": {
|
||||
"type": "string"
|
||||
},
|
||||
"gpu_hours": {
|
||||
"type": "number"
|
||||
},
|
||||
"core_hours": {
|
||||
"type": "number"
|
||||
},
|
||||
"cost": {
|
||||
"type": "number",
|
||||
"description": "Cost is what it held cost, in budget units."
|
||||
},
|
||||
"prompt_tokens": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"completion_tokens": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"price": {
|
||||
"$ref": "#/components/schemas/accounting.Price",
|
||||
"description": "Price is the suggestion; zero with a Note saying why there is none."
|
||||
},
|
||||
"note": {
|
||||
"type": "string"
|
||||
},
|
||||
"current": {
|
||||
"$ref": "#/components/schemas/accounting.Price"
|
||||
}
|
||||
}
|
||||
},
|
||||
"PriceSuggestions": {
|
||||
"type": "object",
|
||||
"description": "PriceSuggestions is a model's price from its cost, for each model.",
|
||||
"properties": {
|
||||
"days": {
|
||||
"type": "integer",
|
||||
"description": "Days is the window; RecordedFrom the first hour held is known for, later than the window's start on a cluster that began keeping it since, which makes the window shorter than asked."
|
||||
},
|
||||
"since": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"recorded_from": {
|
||||
"type": "string",
|
||||
"format": "date-time"
|
||||
},
|
||||
"gpu_hour_cost": {
|
||||
"type": "number"
|
||||
},
|
||||
"core_hour_cost": {
|
||||
"type": "number"
|
||||
},
|
||||
"models": {
|
||||
"type": "array",
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/PriceSuggestion"
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"ProfileReplicasRequest": {
|
||||
"type": "object",
|
||||
"description": "ProfileReplicasRequest counts a model's replicas per profile: profile name to replicas, and CPU, for a llama.cpp model with no profile on CPUs, the one to add.",
|
||||
@@ -10007,6 +10105,35 @@
|
||||
}
|
||||
}
|
||||
},
|
||||
"accounting.Price": {
|
||||
"type": "object",
|
||||
"description": "Price is what one model costs, in micro-units per thousand tokens.\n\nMicro-units rather than a float because spend is accumulated by repeated addition, and repeated addition of floats drifts. What a unit means is the operator's: self-hosted models have no per-token price, so it may be GPU-seconds, a notional rate, or 1000 everywhere — which turns a budget into a token quota.",
|
||||
"properties": {
|
||||
"PromptPer1k": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"CompletionPer1k": {
|
||||
"type": "integer",
|
||||
"format": "int64"
|
||||
},
|
||||
"CachedPer1k": {
|
||||
"type": "integer",
|
||||
"format": "int64",
|
||||
"description": "CachedPer1k is what a prompt token costs that the engine or provider read from its cache instead of computing: the part of prompt_tokens the answer counts as cached_tokens. Zero charges those at PromptPer1k, as before there was a rate for them."
|
||||
},
|
||||
"PerImage": {
|
||||
"type": "integer",
|
||||
"format": "int64",
|
||||
"description": "PerImage is what one picture costs, for a model that makes them: a picture's cost is the card's time, which its prompt's few tokens do not measure."
|
||||
},
|
||||
"PerMesh": {
|
||||
"type": "integer",
|
||||
"format": "int64",
|
||||
"description": "PerMesh is what one 3D model costs: minutes of a card, which no token count measures."
|
||||
}
|
||||
}
|
||||
},
|
||||
"accounting.Rates": {
|
||||
"type": "object",
|
||||
"description": "Rates are an accountant's two limiters' counts.",
|
||||
|
||||
Reference in new issue
Block a user