4 Commits

Include renames

Author	SHA1	Message	Date
Ilker ManapandClaude Opus 5.5	dc483d8c68	Choose each engine's build from the node's hardware, for every GPU generation ... Each engine has a list of builds, best first, from what its project publishes and how each was compiled: llama.cpp CUDA 13.3, CUDA 12.8 (driver 570 for the GPUs it ships as PTX), then Vulkan, never the CPU build on an NVIDIA node; vLLM CUDA 13, then CUDA 12.9, and for GPUs older than every build of the version asked for, the newest built for them (V100: 0.19.1). The node takes the first its hardware meets and that installs and computes on the GPU, moving on when one does not, and reports the version it runs. GPU processes are read from nvidia-smi's full report, so Vulkan engines count as on their GPU. Refs #2. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 14:35:58 +02:00
Ilker ManapandClaude Opus 5.5	86e9a1cb83	Give V100s the last vLLM built for them, not a build without their kernels ... Read from vLLM's release pipeline: every x86 build from 0.20.0 has kernels for compute 7.5 and up, the CUDA 12.9 one included, and 0.19.1 is the last built for 7.0. PyPI's build is CUDA 13 from 0.20, not 0.26. A V100 asked for a newer vLLM is refused, naming 0.19.1 for engine_overrides; the CUDA 12.9 fallback is for older drivers on Turing and newer. The cu129 wheel is found among the release's assets, since its manylinux tag differs between versions. Refs #2. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 14:04:45 +02:00
Ilker ManapandClaude Opus 5.5	4f280a3546	Give each GPU node a vLLM build it can run, chosen on the cluster page ... A node that cannot run the CUDA 13 build installs the same version's CUDA 12.9 build from vLLM's GitHub release, with PyTorch's cu129 torch; the install computes on the GPU before it counts. The build can be chosen per node — auto, cu13, cu129, none — on the cluster page, with ctl engine-build, or over the admin API; the control plane keeps the choice and tells the node at once. Refs #2. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 12:39:14 +02:00
Ilker ManapandClaude Opus 5.5	4c5a2fd6da	Install vLLM on every GPU node that can run it, as llama.cpp is ... vLLM was installed by hand and named in each agent's configuration. The agent now installs it: a Python 3.12 environment built by uv, which is itself a GitHub release checked against its checksum, from PyPI resolved as it stood a day after that vLLM was published and installed with --require-hashes. The resolved list is kept in the environment. The environment is built relocatable under a scratch name and renamed into place once it runs, like an archive. The default is 0.30.0, whose flags were checked against the ones llmgw builds. It is built for CUDA 13, so it needs driver 580 and a Turing GPU or newer, and 25 GB free while it installs; a node lacking one says which, and a node with no GPU skips it without being shown as a fault. gcc comes with it because Triton compiles its launcher at the first model load. SGLang installs the same way when the control plane names a version. llmgw doctor reports whether vLLM can be installed on the node. Checked by installing 0.30.0 this way on a test node: 7.7 GB, torch 2.13 for CUDA 13.0, importing cleanly; the node has no GPU, so vLLM was not started. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-29 13:15:54 +02:00