llama.cpp's CUDA runtime is unpacked beside the backend, and CUDA 13 follows the build
CI / test (push) Successful in 12m42s

The cudart archive unpacks into a directory of its own, while
libggml-cuda.so finds its libraries only through $ORIGIN, so the runtime
the release carries was never loaded: CUDA builds worked only on nodes
with a CUDA toolkit of the same major version installed. On cuda01
(CUDA 13.3 toolkit) b12606's CUDA 12.8 build saw no GPU and the install
fell through to Vulkan.

The CUDA 13 build's name also moved with llama.cpp's CI: 13.3 up to
b11070, 13.4 from b11071, 13.5 from b12170. A newer version asked for
the 13.3 archive, which does not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-21 13:39:55 +01:00
1 parent 32d4d9c3af
commit 0ffd633bf6
5 files changed
+101 -10

No files matched your search

+2 -2
View File
@@ -787,7 +787,7 @@ Nothing to do: the agent installs them once the node joins, under
| Engine | On | How |
|---|---|---|
| llama.cpp | every node | the release build for its processor and GPU — CUDA 13.3, CUDA 12.8 (x64), or Vulkan for GPUs and drivers the CUDA builds do not cover; the CPU build only where there is no NVIDIA GPU — checked against GitHub's checksum and seen to list the GPU |
| llama.cpp | every node | the release build for its processor and GPU — CUDA 13 (13.3, 13.4 or 13.5, whichever the build was published for), CUDA 12.8 (x64), or Vulkan for GPUs and drivers the CUDA builds do not cover; the CPU build only where there is no NVIDIA GPU — checked against GitHub's checksum and seen to list the GPU |
| vLLM | nodes with an NVIDIA GPU it can run on | a Python environment built by uv from PyPI, about 8 GB |
| SGLang | only when you ask for it | the same way as vLLM |
| TensorRT-LLM | only nodes where you installed it | not installed by llmgw: see [TensorRT-LLM, which you bring](#tensorrt-llm-which-you-bring) |
@@ -3312,7 +3312,7 @@ That carries:
`--version` for another), with `checksums.txt` and its signature, checked
against the project's signing key as `ctl upgrade` checks it.
- **llama.cpp** at the version this llmgw installs, every build for each
architecture — CPU, CUDA 13.3, CUDA 12.8 (amd64), Vulkan — each checked
architecture — CPU, CUDA 13 (cuda-13.3 up to b11070, cuda-13.4 to b12169, cuda-13.5 after), CUDA 12.8 (amd64), Vulkan — each checked
against the checksum GitHub publishes, the same check a node does online.
`--variant cpu --variant cuda-13.3` carries only those, for a site whose
hardware you know; `--engine llamacpp=b10991` names the version.