Commit Graph
1 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 688b205604 What llmgw adds to a request, measured: ctl bench overhead
CI / test (push) Successful in 8m56s
An engine that answers at once and the gateway in front of it, in one
process on loopback, given the same load straight and through the gateway:
bare, with authentication and accounting, and with the guardrails' pattern
checks. It reports p50 and p99, what the gateway added, and a rough CPU
per thousand requests.

On an Apple M5 at 100 to 2,000 requests a second (10 s a step, 72,000
requests, none failed), the gateway added 0.05–0.42 ms at p50 and at most
0.49 ms at p99. That is now in the README, with how it was measured and
why the CPU figure is rough. CI runs a one-second version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-17 00:23:26 +02:00