llama.cpp's CUDA runtime is unpacked beside the backend, and CUDA 13 follows the build
CI / test (push) Successful in 12m42s
CI / test (push) Successful in 12m42s
The cudart archive unpacks into a directory of its own, while libggml-cuda.so finds its libraries only through $ORIGIN, so the runtime the release carries was never loaded: CUDA builds worked only on nodes with a CUDA toolkit of the same major version installed. On cuda01 (CUDA 13.3 toolkit) b12606's CUDA 12.8 build saw no GPU and the install fell through to Vulkan. The CUDA 13 build's name also moved with llama.cpp's CI: 13.3 up to b11070, 13.4 from b11071, 13.5 from b12170. A newer version asked for the 13.3 archive, which does not exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
32d4d9c3af
commit
0ffd633bf6
5 files changed
+101
-10
No files matched your search
+2
-2
@@ -787,7 +787,7 @@ Nothing to do: the agent installs them once the node joins, under
|
||||
|
||||
| Engine | On | How |
|
||||
|---|---|---|
|
||||
| llama.cpp | every node | the release build for its processor and GPU — CUDA 13.3, CUDA 12.8 (x64), or Vulkan for GPUs and drivers the CUDA builds do not cover; the CPU build only where there is no NVIDIA GPU — checked against GitHub's checksum and seen to list the GPU |
|
||||
| llama.cpp | every node | the release build for its processor and GPU — CUDA 13 (13.3, 13.4 or 13.5, whichever the build was published for), CUDA 12.8 (x64), or Vulkan for GPUs and drivers the CUDA builds do not cover; the CPU build only where there is no NVIDIA GPU — checked against GitHub's checksum and seen to list the GPU |
|
||||
| vLLM | nodes with an NVIDIA GPU it can run on | a Python environment built by uv from PyPI, about 8 GB |
|
||||
| SGLang | only when you ask for it | the same way as vLLM |
|
||||
| TensorRT-LLM | only nodes where you installed it | not installed by llmgw: see [TensorRT-LLM, which you bring](#tensorrt-llm-which-you-bring) |
|
||||
@@ -3312,7 +3312,7 @@ That carries:
|
||||
`--version` for another), with `checksums.txt` and its signature, checked
|
||||
against the project's signing key as `ctl upgrade` checks it.
|
||||
- **llama.cpp** at the version this llmgw installs, every build for each
|
||||
architecture — CPU, CUDA 13.3, CUDA 12.8 (amd64), Vulkan — each checked
|
||||
architecture — CPU, CUDA 13 (cuda-13.3 up to b11070, cuda-13.4 to b12169, cuda-13.5 after), CUDA 12.8 (amd64), Vulkan — each checked
|
||||
against the checksum GitHub publishes, the same check a node does online.
|
||||
`--variant cpu --variant cuda-13.3` carries only those, for a site whose
|
||||
hardware you know; `--engine llamacpp=b10991` names the version.
|
||||
|
||||
Reference in new issue
Block a user