Guarantee and limit capacity per part of the tenancy tree
A model has an owner (org, team or project). Quotas guarantee a scope GPUs, cores and memory and may limit it; what a scope holds is summed from its models' replicas and those beneath it. A replica within its owner's guarantee is not taken for another scope; what is held beyond a guarantee is borrowed and taken back for a scope within its own. A limit stops a model before it starts, and headroom stops there too. ctl quota and ctl model owner set them; metrics and a CapacityAtLimit alert report them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
ac24e204a6
commit
9dc1b62fdc
28 files changed
+1475
-16
No files matched your search
@@ -95,6 +95,17 @@ groups:
|
||||
the cluster is too small for what is asked of it at these classes.
|
||||
`llmgw ctl models` shows who it gave way to.
|
||||
|
||||
- alert: CapacityAtLimit
|
||||
expr: llmgw_capacity_holds >= on(scope, resource) llmgw_capacity_limit
|
||||
for: 15m
|
||||
labels: { severity: info }
|
||||
annotations:
|
||||
summary: "{{ $labels.scope }} holds all the {{ $labels.resource }} it may"
|
||||
description: >-
|
||||
Its models get no further replicas, under load or not, until
|
||||
something of theirs stops or the limit is raised: `llmgw ctl quota
|
||||
list`, `llmgw ctl quota set`.
|
||||
|
||||
- alert: EngineInstallFailed
|
||||
expr: llmgw_node_engine_install{state="failed"} == 1
|
||||
for: 15m
|
||||
|
||||
Reference in new issue
Block a user