Ilker ManapandClaude Opus 5.5 fa6e68c8e8 A model put first is not evicted, idle, for a lower class ...

The VLM is to come first on the one card. Its class makes it take the card
from flux, triposr or sf3d when asked for, but idle eviction ignored class:
a 3D job two minutes into a lull would have taken the card from it. Idle
eviction offers only replicas of no higher class than the model wanting
room, and the wait such a model is told runs to the holder's idle timeout.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-13 10:15:14 +02:00

..

adminapi

The client API described in OpenAPI, and both APIs in Swagger UI

2026-10-10 20:22:48 +02:00

agentsvc

LoRA adapters trained on the nodes: llmgw ctl train

2026-10-08 20:25:57 +02:00

catalog

A custom node's compiled packages are built for the card that runs them

2026-10-11 17:23:05 +02:00

fabrics

Watch each fabric's subnet managers, and keep a fabric one through a failover

2026-09-29 07:47:03 +02:00

fabrictest

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

facility

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

federation

Federation: another llmgw cluster as a provider, its models as PEER/MODEL

2026-10-05 01:21:30 +02:00

fleetmetrics

MIG: a card in MIG mode is placed as its instances, each a GPU of class <card>-mig-<profile>

2026-10-05 00:01:14 +02:00

inventory

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

modelserve

Qwen3Guard is the judge, and judges on a GPU and on CPUs are asked in turn

2026-10-08 10:22:47 +02:00

modelsvc

A profile may load weights of its own

2026-10-10 17:15:38 +02:00

nvlink

Find multi-node NVLink domains, place across them, and show them as racks

2026-09-29 09:07:59 +02:00

placement

Nodes show their engines' versions and GPUs' capability; a profile can ask for an engine version

2026-10-07 13:06:57 +02:00

quota

Guarantee and limit capacity per part of the tenancy tree

2026-09-30 18:15:38 +02:00

reconciler

A model put first is not evicted, idle, for a lower class

2026-10-13 10:15:14 +02:00

topology

Map every InfiniBand fabric, draw it, and place replicas by it

2026-09-29 08:46:23 +02:00

train

LoRA adapters trained on the nodes: llmgw ctl train

2026-10-08 20:25:57 +02:00

control_test.go

Give every engine a key only the router holds

2026-09-30 00:38:29 +02:00

control.go

A profile may name the card it needs rather than its class, and fit any site

2026-10-11 16:02:32 +02:00

enginechoice.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

external_test.go

Serve a model from an external OpenAI-compatible provider

2026-09-30 23:29:01 +02:00

facility_test.go

Facility-aware placement: racks, PDUs, holds and power caps

2026-10-05 01:17:49 +02:00

facility.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

leadership_test.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

leadership.go

The cluster's settings live in the store, and every change takes effect

2026-10-05 15:25:23 +02:00

quotas.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00