Guarantee and limit capacity per part of the tenancy tree
A model has an owner (org, team or project). Quotas guarantee a scope GPUs, cores and memory and may limit it; what a scope holds is summed from its models' replicas and those beneath it. A replica within its owner's guarantee is not taken for another scope; what is held beyond a guarantee is borrowed and taken back for a scope within its own. A limit stops a model before it starts, and headroom stops there too. ctl quota and ctl model owner set them; metrics and a CapacityAtLimit alert report them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
ac24e204a6
commit
9dc1b62fdc
28 files changed
+1475
-16
No files matched your search
@@ -226,6 +226,8 @@ func newServerCmd() *cobra.Command {
|
||||
// And what each group of the tenancy tree spent, which is
|
||||
// the other half of a quota being reviewable.
|
||||
api.SetUsage(st)
|
||||
// Capacity quotas, set and read over this API.
|
||||
api.SetQuotas(st)
|
||||
}
|
||||
api.Register(adminMux)
|
||||
|
||||
@@ -483,6 +485,9 @@ func startControlPlane(ctx context.Context, cfg config.Server, cat *catalog.Live
|
||||
if st := h.Store(); st != nil {
|
||||
lead = control.NewLeadership(st, cfg.Control.LeaseTTL, log)
|
||||
ctl.SetLeadership(lead)
|
||||
// The tenancy tree's capacity quotas: what each part is
|
||||
// guaranteed and may hold.
|
||||
ctl.Reconciler().SetQuotas(control.QuotaPolicy(st, log))
|
||||
log.Info("competing to be the control plane that places work",
|
||||
"owner", lead.Owner())
|
||||
}
|
||||
|
||||
Reference in new issue
Block a user