Guarantee and limit capacity per part of the tenancy tree
A model has an owner (org, team or project). Quotas guarantee a scope GPUs, cores and memory and may limit it; what a scope holds is summed from its models' replicas and those beneath it. A replica within its owner's guarantee is not taken for another scope; what is held beyond a guarantee is borrowed and taken back for a scope within its own. A limit stops a model before it starts, and headroom stops there too. ctl quota and ctl model owner set them; metrics and a CapacityAtLimit alert report them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
ac24e204a6
commit
9dc1b62fdc
28 files changed
+1475
-16
No files matched your search
@@ -0,0 +1,19 @@
|
||||
-- Capacity a part of the tenancy tree is guaranteed, and the most it may hold.
|
||||
--
|
||||
-- Beside limits rather than in it: limits are counted on the request path, per
|
||||
-- key, in requests and tokens; these are counted by the reconciler, per model,
|
||||
-- in the GPUs, cores and memory its replicas hold. A NULL is a resource this
|
||||
-- row does not govern: guaranteed nothing, or limited by nothing.
|
||||
CREATE TABLE capacity_quotas (
|
||||
scope_type TEXT NOT NULL,
|
||||
scope_id TEXT NOT NULL,
|
||||
gpus INTEGER,
|
||||
cpu_cores INTEGER,
|
||||
memory_gb INTEGER,
|
||||
limit_gpus INTEGER,
|
||||
limit_cpu_cores INTEGER,
|
||||
limit_memory_gb INTEGER,
|
||||
updated_at BIGINT NOT NULL,
|
||||
updated_by TEXT NOT NULL,
|
||||
PRIMARY KEY (scope_type, scope_id)
|
||||
);
|
||||
Reference in new issue
Block a user