Ilker ManapandClaude Opus 5
634f87ba15
CI / test (push) Successful in 1m26s
Proxmox defaults a VM to kvm64, which hides AVX, AVX2, FMA and even SSE4.2 from the guest. llama.cpp then loads libggml-cpu-x64.so — its scalar baseline — instead of the backend built for the host's instruction set, and an inference node is not slightly slower but unusable. Measured on node05, an 8-core VM on a Xeon E5-2680 v4, same weights, same thread count, the only change being --cpu host: Qwen2.5-Coder kvm64 host 1.5B q4_K_M 5.80 39.09 tok/s 3B q4_K_M 2.90 20.51 7B q4_K_M 1.33 9.88 Prompt processing went from 7.6 to 143.8, a factor of nineteen. This cost an afternoon because nothing looks broken. The node registers, the engine starts, the replies are correct; every number is just small, and the conclusion that suggests itself is that CPU inference is hopeless rather than that the VM cannot multiply efficiently. A 14B model at 1.7 tokens a second was nearly written off on that basis. The trade is live migration, which a VM pinned to the host CPU cannot do. For a node whose job is arithmetic that is the right way round, and --cpu overrides it where it is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>