Commit Graph
3 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 4f280a3546 Give each GPU node a vLLM build it can run, chosen on the cluster page
A node that cannot run the CUDA 13 build installs the same version's
CUDA 12.9 build from vLLM's GitHub release, with PyTorch's cu129 torch;
the install computes on the GPU before it counts. The build can be
chosen per node — auto, cu13, cu129, none — on the cluster page, with
ctl engine-build, or over the admin API; the control plane keeps the
choice and tells the node at once.

Refs #2.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:39:14 +02:00
Ilker ManapandClaude Opus 5.5 e313f32696 Decide engines per node, and let a node join without one it cannot run
engine_overrides give nodes picked by GPU class, id or labels their own
engine versions; node add --disable-engine keeps one off a node; what
vLLM cannot do on a node is a warning rather than a refused node; the
CUDA requirement follows the version; a hand-named binary is no longer
replaced by an installed one.

Refs #2 (items 4 and 7).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-01 12:07:25 +02:00
Ilker ManapandClaude Opus 5.5 62657f4c2a Release v0.10.4: CPU threads, tracing, and the rest of the review
CPU replicas start on half the machine's CPUs, not all of them: two busy
llama.cpp servers each spinning on all 8 vCPUs fell to 0.8 tok/s apiece,
where at 4 threads they keep 5.5. llamacpp.threads sets it per node.

Inference requests can be traced over OTLP/HTTP (tracing.endpoint), with
admission, schedule and upstream spans and traceparent sent to the engine.

Security: onboarding no longer puts the cluster root in the system trust
store and removes it where it was; the mirror's mutual-TLS API can be
limited to named certificates (--api-client); init.log no longer keeps the
administrator's password; a warning when auth is off on a public listen;
the recorded actor is sanitised and sudo-aware; Ray's dashboard is off, and
the remaining engine gaps are documented.

reconciler.go and agent.go are split by topic, the status mapping of each
failure is tested, and the plan's checklist is brought up to date.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 04:00:35 +02:00