Guarantee and limit capacity per part of the tenancy tree

A model has an owner (org, team or project). Quotas guarantee a scope
GPUs, cores and memory and may limit it; what a scope holds is summed from
its models' replicas and those beneath it. A replica within its owner's
guarantee is not taken for another scope; what is held beyond a guarantee
is borrowed and taken back for a scope within its own. A limit stops a
model before it starts, and headroom stops there too. ctl quota and ctl
model owner set them; metrics and a CapacityAtLimit alert report them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-30 18:15:38 +02:00
1 parent ac24e204a6
commit 9dc1b62fdc
28 files changed
+1475 -16

No files matched your search

+5
View File
@@ -226,6 +226,8 @@ func newServerCmd() *cobra.Command {
// And what each group of the tenancy tree spent, which is
// the other half of a quota being reviewable.
api.SetUsage(st)
// Capacity quotas, set and read over this API.
api.SetQuotas(st)
}
api.Register(adminMux)
@@ -483,6 +485,9 @@ func startControlPlane(ctx context.Context, cfg config.Server, cat *catalog.Live
if st := h.Store(); st != nil {
lead = control.NewLeadership(st, cfg.Control.LeaseTTL, log)
ctl.SetLeadership(lead)
// The tenancy tree's capacity quotas: what each part is
// guaranteed and may hold.
ctl.Reconciler().SetQuotas(control.QuotaPolicy(st, log))
log.Info("competing to be the control plane that places work",
"owner", lead.Owner())
}