Launch multi-node vLLM its own way by default, not through Ray

An empty launch_mode meant ray: a Ray head and joins on every node, a
dependency to install, and a path never checked against a real cluster.
It now means mp - every node runs vllm serve with its rank and the
leader's address, the workers headless - which is how vLLM documents a
multi-node launch today and needs nothing but vLLM. ray is still there
for a vLLM too old for mp's flags. The example agent configuration,
doctor and the reference say so.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-29 09:13:59 +02:00
1 parent 20479a2d5f
commit 45e3f52da3
5 files changed
+34 -16

No files matched your search

+5 -4
View File
@@ -96,10 +96,11 @@ discovery:
engines:
vllm:
bin: /opt/venvs/vllm/bin/vllm
# Which multi-node mechanism works depends on the vLLM version, and the
# flags have changed more than once. Check the version you deploy.
launch_mode: ray # "ray" or "mp"
ray_bin: /opt/venvs/vllm/bin/ray
# How a replica spanning nodes is launched. mp, the default, is what vLLM
# documents today and needs nothing but vLLM; ray is for a vLLM too old
# for mp's multi-node flags, and needs Ray on every node.
launch_mode: mp # "mp" or "ray"
# ray_bin: /opt/venvs/vllm/bin/ray
sglang:
# SGLang is launched as a module, which is its documented entry point.
python: /opt/venvs/sglang/bin/python