engine-check computes a token on the GPU, and finds the installed llama.cpp

Seeing a device is not computing on it. For llama.cpp on a machine with
NVIDIA GPUs, engine-check now loads a small GGUF (--model, or the smallest in
the weights cache) with every layer on the GPU, asks for one token, and
reads from llama.cpp's own output where the layers went: a build that
computes on the CPU fails, naming the fix. With no --bin it checks the build
the agent installed under /opt/llmgw/engines.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-07 13:04:38 +02:00
1 parent 0bd0d881c5
commit e4e15200d6
3 files changed
+217 -1

No files matched your search

+3
View File
@@ -277,6 +277,9 @@ func (l *LlamaCpp) bin() string {
return "llama-server"
}
// Binary is the llama-server this engine runs.
func (l *LlamaCpp) Binary() string { return l.bin() }
// FallsBackToCPU says that llama.cpp keeps running when its GPU backend does
// not load: --n-gpu-layers asks for layers on a GPU it then has none of, and
// the server comes up and answers on the CPU, at a tenth of the speed, with no