2 Commits

Include renames

Author	SHA1	Message	Date
Ilker ManapandClaude Opus 5	634f87ba15	Provision nodes with the host CPU model ... Proxmox defaults a VM to kvm64, which hides AVX, AVX2, FMA and even SSE4.2 from the guest. llama.cpp then loads libggml-cpu-x64.so — its scalar baseline — instead of the backend built for the host's instruction set, and an inference node is not slightly slower but unusable. Measured on node05, an 8-core VM on a Xeon E5-2680 v4, same weights, same thread count, the only change being --cpu host: Qwen2.5-Coder kvm64 host 1.5B q4_K_M 5.80 39.09 tok/s 3B q4_K_M 2.90 20.51 7B q4_K_M 1.33 9.88 Prompt processing went from 7.6 to 143.8, a factor of nineteen. This cost an afternoon because nothing looks broken. The node registers, the engine starts, the replies are correct; every number is just small, and the conclusion that suggests itself is that CPU inference is hopeless rather than that the VM cannot multiply efficiently. A 14B model at 1.7 tokens a second was nearly written off on that basis. The trade is live migration, which a VM pinned to the host CPU cannot do. For a node whose job is arithmetic that is the right way round, and --cpu overrides it where it is not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-16 20:42:48 +02:00
Ilker ManapandClaude Opus 5	aa4da3264d	Make adding a node one command, and check the engine actually runs ... Building node05 by hand turned up three things worth fixing, in the order they bit. Onboarding died on "Host key verification failed" — an error about ssh, in the middle of a script whose entire job is to talk to a machine created sixty seconds ago. It now uses accept-new, which learns an unknown host key and still refuses a changed one; the risk being guarded against is a substituted machine, not a new one. Then ssh started refusing connections on a node that had answered a moment before: cloud-init regenerates the host keys and restarts sshd partway through a new machine's first minute. The agent was registered and healthy throughout. Onboarding now waits for ssh rather than failing on it, which is also what makes running the two halves back to back safe. Then llama-server: present, executable, and unable to start for want of libgomp1. The node registered ready and would have stayed ready until a replica was placed on it and the shard died with a missing shared library three logs away. The docs already warned about this shape of failure; nothing checked for it. Onboarding now runs a configured engine and says plainly when it will not start. add-node.sh drives both halves. The reason is not keystrokes: the VM name, the hostname, the node id and the certificate's CN must all be the same string, and the address appears in three places. Split across two commands run minutes apart, that agreement is held together by whoever is typing. Here it is stated once, and the VMID defaults to the last octet of the address so qm list and ctl nodes stay in the same order. The two halves remain separately runnable, because needing the hypervisor and needing the CA really are different access. Verified by building a throwaway node06 with it: 45 seconds from nothing to registered, and the engine check caught that node06 had no llama-server. Destroyed afterwards. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-16 19:07:56 +02:00