Ilker ManapandClaude Opus 5.5 1e7086e13e How long an on-demand model holds a card is a live setting ...

On one card shared by flux and sf3d, a mesh asked for after a picture
waited out the picture model's five minutes of protection and two of idle.
on_demand.evict_after_idle and on_demand.min_lifetime set those for the
cluster, in force on the leader as they change; unset keeps 2m and 5m.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-14 08:58:38 +02:00

..

adopt.go

An adopted on-demand replica counts as used when adopted

2026-10-11 08:55:33 +02:00

demand.go

llmgw stop, start and restart act on the whole system

2026-10-07 18:01:45 +02:00

evict.go

How long an on-demand model holds a card is a live setting

2026-10-14 08:58:38 +02:00

facility_test.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

fetch_test.go

Carry a download's progress to where somebody is looking at it

2026-09-27 07:49:08 +02:00

hardware.go

A profile may name the card it needs rather than its class, and fit any site

2026-10-11 16:02:32 +02:00

headroom_test.go

Reserve CPU cores and require memory for CPU models

2026-09-30 17:43:13 +02:00

headroom.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

health.go

How long an on-demand model holds a card is a live setting

2026-10-14 08:58:38 +02:00

leadership_test.go

A standby that takes over waits for the fleet before placing

2026-10-07 13:40:28 +02:00

measured_test.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00

measured.go

A profile may name the card it needs rather than its class, and fit any site

2026-10-11 16:02:32 +02:00

notify_test.go

Notifications: webhooks for a parked model, a lost node, a spent budget and new weights; Alertmanager receiver templates

2026-10-04 16:58:11 +02:00

preempt_test.go

Name only the replicas that would make room in headroom

2026-09-30 23:01:40 +02:00

preempt.go

How long an on-demand model holds a card is a live setting

2026-10-14 08:58:38 +02:00

quotas_test.go

Guarantee and limit capacity per part of the tenancy tree

2026-09-30 18:15:38 +02:00

quotas.go

Guarantee and limit capacity per part of the tenancy tree

2026-09-30 18:15:38 +02:00

reconciler_test.go

How long an on-demand model holds a card is a live setting

2026-10-14 08:58:38 +02:00

reconciler.go

How long an on-demand model holds a card is a live setting

2026-10-14 08:58:38 +02:00

replica.go

A replica says what its engine computes on, and llama.cpp's word on the CPU refuses a GPU placement

2026-10-07 13:02:17 +02:00

restart.go

A replica restarted in place keeps the room it frees for its replacement

2026-10-08 11:42:55 +02:00

schedule_test.go

Scaling by the clock: schedule windows with their own min and max, prewarm, control.timezone

2026-10-04 23:16:56 +02:00

spread_test.go

Pin down where CPU replicas land, and where they deliberately do not

2026-09-16 09:57:12 +02:00

stuck_test.go

Stop an idle model to make room for one somebody asked for

2026-09-17 17:51:07 +02:00

trace_test.go

Tracing beyond the router: reconcile passes, placement and a replica's start traced down to the node

2026-10-05 00:03:17 +02:00

trace.go

Tracing beyond the router: reconcile passes, placement and a replica's start traced down to the node

2026-10-05 00:03:17 +02:00