Launch multi-node vLLM its own way by default, not through Ray
An empty launch_mode meant ray: a Ray head and joins on every node, a dependency to install, and a path never checked against a real cluster. It now means mp - every node runs vllm serve with its rank and the leader's address, the workers headless - which is how vLLM documents a multi-node launch today and needs nothing but vLLM. ray is still there for a vLLM too old for mp's flags. The example agent configuration, doctor and the reference say so. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
20479a2d5f
commit
45e3f52da3
5 files changed
+34
-16
No files matched your search
+6
-4
@@ -1351,11 +1351,13 @@ Bin is the vllm executable.
|
||||
|
||||
`string`
|
||||
|
||||
LaunchMode selects the multi-node mechanism: "ray" or "mp".
|
||||
LaunchMode selects the multi-node mechanism: "mp", the default, or
|
||||
"ray".
|
||||
|
||||
Which one works depends on the vLLM version, and the flags have changed
|
||||
more than once (plan §14). Both paths exist so a cluster can pick the one
|
||||
its version supports instead of waiting for a code change.
|
||||
mp is what vLLM documents today — every node runs serve with its rank,
|
||||
the workers headless — and needs nothing but vLLM. ray is the older way,
|
||||
kept for a vLLM too old for mp's multi-node flags; it needs Ray on every
|
||||
node and has not been checked against a real cluster.
|
||||
|
||||
### `ray_bin`
|
||||
|
||||
|
||||
Reference in new issue
Block a user