Scale and place by what a replica was measured to serve

Stored ctl bench results reach the reconciler: requests per replica past
the measured SLO concurrency scale a model up, a replica is taken back only
if the rest stay under 70% of theirs, and among profiles of equal priority
the most measured tokens per GPU or core is tried first. ctl model target
sizes a model for a throughput against the room there is.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-30 18:21:08 +02:00
1 parent 9dc1b62fdc
commit 78e41460f9
16 files changed
+503 -15

No files matched your search

+3
View File
@@ -488,6 +488,9 @@ func startControlPlane(ctx context.Context, cfg config.Server, cat *catalog.Live
// The tenancy tree's capacity quotas: what each part is
// guaranteed and may hold.
ctl.Reconciler().SetQuotas(control.QuotaPolicy(st, log))
// And what each model was measured to serve, for scaling and for
// choosing between profiles.
ctl.Reconciler().SetMeasurements(control.Measurements(st, log))
log.Info("competing to be the control plane that places work",
"owner", lead.Owner())
}