Release v0.10.4: CPU threads, tracing, and the rest of the review
Release / build (push) Successful in 4m11s
Release / notes (push) Successful in 3s
CI / test (push) Successful in 4m57s

CPU replicas start on half the machine's CPUs, not all of them: two busy
llama.cpp servers each spinning on all 8 vCPUs fell to 0.8 tok/s apiece,
where at 4 threads they keep 5.5. llamacpp.threads sets it per node.

Inference requests can be traced over OTLP/HTTP (tracing.endpoint), with
admission, schedule and upstream spans and traceparent sent to the engine.

Security: onboarding no longer puts the cluster root in the system trust
store and removes it where it was; the mirror's mutual-TLS API can be
limited to named certificates (--api-client); init.log no longer keeps the
administrator's password; a warning when auth is off on a public listen;
the recorded actor is sanitised and sudo-aware; Ray's dashboard is off, and
the remaining engine gaps are documented.

reconciler.go and agent.go are split by topic, the status mapping of each
failure is tested, and the plan's checklist is brought up to date.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-30 04:00:35 +02:00
1 parent adb1b3cf2a
commit 62657f4c2a
41 files changed
+2485 -1609

No files matched your search

+8
View File
@@ -1124,3 +1124,11 @@ requests. On a slow model, set `timeouts.generation` higher.
has them. Single-node placement on real GPUs is the well-trodden path; if you
are the first to run a TP group across real cards, run `llmgw doctor --json`
on each node first and keep the reports.
- **Two engine setups are weaker than the rest.** An engine's key is passed
in its environment, where only root and its own user can read it, except
SGLang's, which takes it only as a flag: other users on the node can see it
in the process list. Mount `/proc` with `hidepid=2` on nodes where that
matters. vLLM's `launch_mode: ray` starts a Ray head without
authentication, so anyone who can reach its port can run code on the
node. The dashboard is off, but firewall the port to the other nodes in the
replica, or use the default `mp` launch mode, which needs no Ray at all.