Scale and place by what a replica was measured to serve
Release / build (push) Successful in 1m8s
CI / test (push) Successful in 4m24s
Release / notes (push) Successful in 3s

Stored ctl bench results reach the reconciler: requests per replica past
the measured SLO concurrency scale a model up, a replica is taken back only
if the rest stay under 70% of theirs, and among profiles of equal priority
the most measured tokens per GPU or core is tried first. ctl model target
sizes a model for a throughput against the room there is.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-30 18:21:08 +02:00
1 parent 9dc1b62fdc
commit 78e41460f9
16 files changed
+503 -15

No files matched your search

+1 -1
View File
@@ -137,7 +137,7 @@ func newCtlCmd() *cobra.Command {
// apply to it; it lives here because this is where operators look.
newBenchCmd(client, &asJSON),
newStatusCmd(client, &asJSON),
newModelCmd(client),
newModelCmd(client, &asJSON),
newScaleCmd(client),
newRestartCmd(client),
newRetryCmd(client),