On one card shared by flux and sf3d, a mesh asked for after a picture waited out the picture model's five minutes of protection and two of idle. on_demand.evict_after_idle and on_demand.min_lifetime set those for the cluster, in force on the leader as they change; unset keeps 2m and 5m. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
19 lines
816 B
Go
19 lines
816 B
Go
package config
|
|
|
|
import "time"
|
|
|
|
// OnDemandSettings is how models that load when asked for take turns on the
|
|
// cards: on a site with fewer cards than such models, the wait between one
|
|
// and the next.
|
|
type OnDemandSettings struct {
|
|
// EvictAfterIdle is how long an on-demand replica must go unasked-for
|
|
// before a model that wants its card may have it. Zero keeps 2m.
|
|
EvictAfterIdle time.Duration `koanf:"evict_after_idle"`
|
|
// MinLifetime is how long a replica that has just come up is safe from
|
|
// that, so two models do not take turns paying for loads. Zero keeps
|
|
// 5m. On one card shared by a picture model and a 3D model, 1m each
|
|
// brings the wait between a picture and its mesh to about a minute.
|
|
// Both are in force on the leader within seconds.
|
|
MinLifetime time.Duration `koanf:"min_lifetime"`
|
|
}
|