llama.cpp's CUDA runtime is unpacked beside the backend, and CUDA 13 follows the build
CI / test (push) Successful in 12m42s

The cudart archive unpacks into a directory of its own, while
libggml-cuda.so finds its libraries only through $ORIGIN, so the runtime
the release carries was never loaded: CUDA builds worked only on nodes
with a CUDA toolkit of the same major version installed. On cuda01
(CUDA 13.3 toolkit) b12606's CUDA 12.8 build saw no GPU and the install
fell through to Vulkan.

The CUDA 13 build's name also moved with llama.cpp's CI: 13.3 up to
b11070, 13.4 from b11071, 13.5 from b12170. A newer version asked for
the 13.3 archive, which does not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-11-21 13:39:55 +01:00
1 parent 32d4d9c3af
commit 0ffd633bf6
5 files changed
+101 -10

No files matched your search

+1 -1
View File
@@ -131,7 +131,7 @@ func newBundleCreateCmd() *cobra.Command {
f.StringVar(&releases, "releases", release.DefaultBase, "where llmgw's releases are published")
f.StringArrayVar(&arches, "arch", []string{"amd64"}, "an architecture of the site's machines, amd64 or arm64; repeatable")
f.StringArrayVar(&engines, "engine", nil, "an engine version to carry, engine=version; repeatable (default: llama.cpp at the version this llmgw installs)")
f.StringArrayVar(&variants, "variant", nil, "a build to carry: cpu, cuda-13.3, cuda-12.8, vulkan; repeatable (default: every build)")
f.StringArrayVar(&variants, "variant", nil, "a build to carry: cpu, cuda-13.3 (cuda-13.4 from llama.cpp b11071, cuda-13.5 from b12170), cuda-12.8, vulkan; repeatable (default: every build)")
f.StringArrayVar(&models, "model", nil, "a model file to carry, name/version=hf:owner/repo/file or name/version=https://...; repeatable")
f.BoolVar(&noStarter, "no-starter-models", false, "leave out the starter model and the guardrails' judge that server init serves")
f.StringVar(&registry, "registry", "", "the model registry, when not the public Hugging Face")