Scaling by the clock: schedule windows with their own min and max, prewarm, control.timezone

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-04 23:16:56 +02:00
1 parent 74429f1c00
commit bce54f0941
14 files changed
+382 -9

No files matched your search

+20
View File
@@ -924,6 +924,26 @@ Models added at runtime only. One declared in `server.yaml` is read again on
every start, so a number set here would hold until the next restart and then
revert — change it in the file instead.
Between `min` and `max` a model scales with its load: requests waiting at
every replica add one, a quiet spell takes it back. For load that comes by the
clock, a schedule says so ahead of time:
```yaml
scaling:
min: 1
max: 4
prewarm: 30m # start a window's replicas this long before it opens
schedule:
- {days: [weekdays], from: "08:00", to: "18:00", min: 3, max: 6}
- {from: "22:00", to: "02:00", min: 1} # past midnight
```
Times are in `control.timezone` (`Europe/Istanbul`; empty is the control
plane's own). The first window that holds wins; outside every window, `min`
and `max` apply. An on-demand model (`min: 0`) with a window is kept loaded
through it, and loaded again on the first request after. Days are `mon` …
`sun`, `weekdays` or `weekends`.
How much traffic each replica is given is the profile's `weight`, and guessing
it is how a cluster ends up with one saturated replica beside two idle ones.
Measure it instead: