SGLang read in its own metric names, started serving them, and asked for its own help

The router read SGLang replicas with vLLM's names, and SGLang was started
without --enable-metrics, so it served no /metrics: every scrape failed as
if unreachable and the replica was routed to blind. It now gets
--enable-metrics and a mapping of its own (sglang:num_running_reqs,
num_queue_reqs, token_usage, the token counters and num_requests_total, as
0.5.21 publishes them). engine-check asks python -m sglang.launch_server
--help instead of Python's own help, and SGLang is given --tp-size rather
than the --tp argparse only takes as an abbreviation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 00:20:25 +02:00
1 parent d41104a452
commit 8cc8d7139b
6 files changed
+251 -3

No files matched your search

+5
View File
@@ -597,6 +597,11 @@ engines:
sglang: "0.5.20" # SGLang only when named
```
SGLang is started with `--enable-metrics`, without which it serves no
`/metrics`; the router reads `sglang:num_running_reqs`, `sglang:num_queue_reqs`
and `sglang:token_usage` there. Check an SGLang against the flags llmgw builds
with `llmgw ctl engine-check sglang --bin <its environment>/bin/python`.
A changed version reaches every node when it next connects. To use an engine
you installed yourself on a node, name its path in that node's `agent.yaml`
(`engines.llamacpp.bin`, `engines.vllm.bin`, `engines.sglang.python`,