There was already a cpu_overcommit multiplier, but a multiplier is not a
policy. Nothing said whether a site was meant to be strict, a ratio left in
the file kept inflating capacity indefinitely, and the resulting number was
reported on its own - "56 cores" being either a machine or a decision, with
no way to tell which.
Now: oversubscribe is the switch, off by default, and the ratios say how
much. Separating them means turning the policy off does not mean editing
the numbers back to 1.0 and losing what they were tuned to. Every
combination that has more than one meaning is refused at startup - a ratio
above 1 with the switch off, the switch on with nothing to multiply - so a
config that reads as strict cannot quietly be anything else.
Memory joins CPU, because it had no ratio at all and enabling only CPU
would have moved the refusal from one resource to the other. They are not
equally safe and the comments say so: oversubscribed cores make jobs slow,
oversubscribed memory makes them die.
The inventory now carries the hardware alongside the schedulable figures,
and both the CLI and the plan output say when a node is oversubscribed and
by how much. A held device is finally printed too - discovery learned who
owns each card in the previous commit, and nothing displayed it.
Enabled on hv01 at cpu 6.0, mem 1.3, chosen by measurement. 28 cores with
153 vCPU already committed by 36 running guests, load average 2.3: the box
is committed, not busy, so 6.0 sits just above what is already promised.
Guests claim 618 GiB of 503 and ballooning absorbs it, with 118 GiB
genuinely available, so 1.3 promises about 36 GiB more than today and stays
inside what the host can actually produce.
Measured both ways, because a switch tested in one direction is untested:
off 28 cores (0 free), 504 GiB (0 free) request rejected
on 168 cores (15 free), 655 GiB (37 free) PLACED, compute1 gets
gpu 0000:01:00.0 (Quadro K620)
The live config was the contradictory case - ratio 2.0, no switch - and
startup refused it by name before it was fixed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
53 lines
2.0 KiB
Plaintext
53 lines
2.0 KiB
Plaintext
# Backing clouds the portal can place tenants on.
|
|
#
|
|
# "kind" selects the provider implementation. Only "proxmox" exists today;
|
|
# OpenStack and others plug in here without anything above this file changing.
|
|
#
|
|
# Secrets belong in the environment, not in this file. Any setting may be
|
|
# written as "env:NAME" to read it from $NAME at startup.
|
|
|
|
sites:
|
|
- id: hv01
|
|
name: Lab cluster (Stockholm)
|
|
kind: proxmox
|
|
settings:
|
|
endpoint: https://10.0.0.10:8006
|
|
token: env:HV01_TOKEN # USER@REALM!TOKENID=UUID
|
|
insecure_tls: "true" # lab CA is self-signed
|
|
timeout: 20s
|
|
# Oversubscription is off by default: a site offers exactly the
|
|
# hardware it has. An HPC tenant handed oversubscribed cores simply
|
|
# runs slowly, and one handed memory the host cannot produce is
|
|
# killed, so a site has to opt in.
|
|
#
|
|
# The switch and the ratios are separate so that turning the policy
|
|
# off does not mean editing the numbers back to 1.0 and losing what
|
|
# they were tuned to. A ratio above 1.0 while the switch is off is
|
|
# refused at startup rather than ignored, because a config that reads
|
|
# as strict must not quietly inflate the site.
|
|
#
|
|
# Pick the numbers from measurement, not habit: what the guests on
|
|
# this host already commit, and how much of it they actually use.
|
|
# Memory is the one to be careful with - CPU oversubscription makes
|
|
# jobs slow, memory oversubscription makes them die.
|
|
oversubscribe: "false"
|
|
# cpu_overcommit: "6.0"
|
|
# mem_overcommit: "1.3"
|
|
|
|
# - id: pve2
|
|
# name: GPU cluster
|
|
# kind: proxmox
|
|
# settings:
|
|
# endpoint: https://10.0.0.20:8006
|
|
# token: env:PVE2_TOKEN
|
|
|
|
# Not implemented yet - shown so the shape of a second provider is clear.
|
|
# - id: os-region-1
|
|
# name: OpenStack region 1
|
|
# kind: openstack
|
|
# settings:
|
|
# auth_url: https://keystone.example.com/v3
|
|
# application_credential_id: env:OS_AC_ID
|
|
# application_credential_secret: env:OS_AC_SECRET
|
|
# region: RegionOne
|