Files
llmgw/docs
Ilker ManapandClaude Opus 5.5 45e3f52da3 Launch multi-node vLLM its own way by default, not through Ray
An empty launch_mode meant ray: a Ray head and joins on every node, a
dependency to install, and a path never checked against a real cluster.
It now means mp - every node runs vllm serve with its rank and the
leader's address, the workers headless - which is how vLLM documents a
multi-node launch today and needs nothing but vLLM. ray is still there
for a vLLM too old for mp's flags. The example agent configuration,
doctor and the reference say so.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 09:13:59 +02:00
..
2026-09-24 04:04:03 +03:00