Ilker ManapandClaude Opus 5.5 7cb9d4fa01 Busy is not unfit: rank files by the hardware the cluster has, and let a model wait for its cards ...

Ranking what was free first made every file come out on the CPU on a cluster
whose one card was in use. The plan now ranks by hardware alone, free now
only breaking a tie; a model served onto cards that are busy is added and
waits for them, the job ending as waiting rather than failed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-04 01:17:32 +02:00

..

adminapi

Plan a repository's files against the fleet, and serve the one that suits it the way it fits

2026-10-04 01:03:48 +02:00

agentsvc

Add a standby control plane with server add, and draw the screens as windows

2026-10-03 22:29:12 +02:00

catalog

Track the registry's new weights for served models, and move a model to them or back

2026-10-04 00:12:20 +02:00

fabrics

Watch each fabric's subnet managers, and keep a fabric one through a failover

2026-09-29 07:47:03 +02:00

fabrictest

Measure two nodes' fabric rail by rail: llmgw ctl fabric-test

2026-09-29 21:29:23 +02:00

fleetmetrics

Read what wears a GPU out: XIDs, remapped rows, PCIe and NVLink errors

2026-10-01 16:02:07 +02:00

inventory

Read what wears a GPU out: XIDs, remapped rows, PCIe and NVLink errors

2026-10-01 16:02:07 +02:00

modelserve

Busy is not unfit: rank files by the hardware the cluster has, and let a model wait for its cards

2026-10-04 01:17:32 +02:00

modelsvc

Track the registry's new weights for served models, and move a model to them or back

2026-10-04 00:12:20 +02:00

nvlink

Find multi-node NVLink domains, place across them, and show them as racks

2026-09-29 09:07:59 +02:00

placement

Reserve CPU cores and require memory for CPU models

2026-09-30 17:43:13 +02:00

quota

Guarantee and limit capacity per part of the tenancy tree

2026-09-30 18:15:38 +02:00

reconciler

Restart in place at once when only the old replica is in the way

2026-10-01 12:11:37 +02:00

topology

Map every InfiniBand fabric, draw it, and place replicas by it

2026-09-29 08:46:23 +02:00

control_test.go

Give every engine a key only the router holds

2026-09-30 00:38:29 +02:00

control.go

Map the fleet's GPUs, and name the cards held and idle

2026-10-01 15:54:58 +02:00

enginechoice.go

Give each GPU node a vLLM build it can run, chosen on the cluster page

2026-10-01 12:39:14 +02:00

external_test.go

Serve a model from an external OpenAI-compatible provider

2026-09-30 23:29:01 +02:00

leadership.go

Stop a second control plane from placing work behind the first's back

2026-09-25 18:24:06 +02:00

quotas.go

Scale and place by what a replica was measured to serve

2026-09-30 18:21:08 +02:00