Ilker ManapandClaude Opus 5.5 dc483d8c68 Choose each engine's build from the node's hardware, for every GPU generation ...

Each engine has a list of builds, best first, from what its project
publishes and how each was compiled: llama.cpp CUDA 13.3, CUDA 12.8
(driver 570 for the GPUs it ships as PTX), then Vulkan, never the CPU
build on an NVIDIA node; vLLM CUDA 13, then CUDA 12.9, and for GPUs
older than every build of the version asked for, the newest built for
them (V100: 0.19.1). The node takes the first its hardware meets and
that installs and computes on the GPU, moving on when one does not, and
reports the version it runs. GPU processes are read from nvidia-smi's
full report, so Vulkan engines count as on their GPU.

Refs #2.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

2026-10-01 14:35:58 +02:00

..

engine.go

Decide engines per node, and let a node join without one it cannot run

2026-10-01 12:07:25 +02:00

engines_test.go

Launch multi-node vLLM its own way by default, not through Ray

2026-09-29 09:13:59 +02:00

llamacpp_test.go

Place llama.cpp across several GPUs of a node with gpus_per_node

2026-10-01 11:56:34 +02:00

llamacpp.go

Choose each engine's build from the node's hardware, for every GPU generation

2026-10-01 14:35:58 +02:00

probe_test.go

Stop placing engines on nodes that do not have them

2026-09-27 16:11:55 +02:00

probe.go

Install vLLM on every GPU node that can run it, as llama.cpp is

2026-09-29 13:15:54 +02:00

sglang.go

Give every engine a key only the router holds

2026-09-30 00:38:29 +02:00

sim_test.go

Onboard a node with a script, and find two examples that never worked

2026-09-16 12:28:36 +02:00

sim.go

Onboard a node with a script, and find two examples that never worked

2026-09-16 12:28:36 +02:00

unsafeenv.go

Stop a browser page, or a model definition, from running code on the nodes

2026-09-30 00:29:52 +02:00

vllm.go

Release v0.10.4: CPU threads, tracing, and the rest of the review

2026-09-30 04:00:35 +02:00