Merge engine fixes: SGLang metrics and flags, bench reading guarded metrics
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
commit
cc4de36674
12 files changed
+655
-28
No files matched your search
@@ -597,6 +597,11 @@ engines:
|
||||
sglang: "0.5.20" # SGLang only when named
|
||||
```
|
||||
|
||||
SGLang is started with `--enable-metrics`, without which it serves no
|
||||
`/metrics`; the router reads `sglang:num_running_reqs`, `sglang:num_queue_reqs`
|
||||
and `sglang:token_usage` there. Check an SGLang against the flags llmgw builds
|
||||
with `llmgw ctl engine-check sglang --bin <its environment>/bin/python`.
|
||||
|
||||
A changed version reaches every node when it next connects. To use an engine
|
||||
you installed yourself on a node, name its path in that node's `agent.yaml`
|
||||
(`engines.llamacpp.bin`, `engines.vllm.bin`, `engines.sglang.python`,
|
||||
|
||||
Reference in new issue
Block a user