SGLang read in its own metric names, started serving them, and asked for its own help
The router read SGLang replicas with vLLM's names, and SGLang was started without --enable-metrics, so it served no /metrics: every scrape failed as if unreachable and the replica was routed to blind. It now gets --enable-metrics and a mapping of its own (sglang:num_running_reqs, num_queue_reqs, token_usage, the token counters and num_requests_total, as 0.5.21 publishes them). engine-check asks python -m sglang.launch_server --help instead of Python's own help, and SGLang is given --tp-size rather than the --tp argparse only takes as an abbreviation. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
d41104a452
commit
8cc8d7139b
6 files changed
+251
-3
No files matched your search
@@ -597,6 +597,11 @@ engines:
|
||||
sglang: "0.5.20" # SGLang only when named
|
||||
```
|
||||
|
||||
SGLang is started with `--enable-metrics`, without which it serves no
|
||||
`/metrics`; the router reads `sglang:num_running_reqs`, `sglang:num_queue_reqs`
|
||||
and `sglang:token_usage` there. Check an SGLang against the flags llmgw builds
|
||||
with `llmgw ctl engine-check sglang --bin <its environment>/bin/python`.
|
||||
|
||||
A changed version reaches every node when it next connects. To use an engine
|
||||
you installed yourself on a node, name its path in that node's `agent.yaml`
|
||||
(`engines.llamacpp.bin`, `engines.vllm.bin`, `engines.sglang.python`,
|
||||
|
||||
Reference in new issue
Block a user