Ship Caddy and nginx configurations for the client port
install.md said the client port belongs behind a reverse proxy and gave nothing to put there. deploy/proxy/ has both, attached to every release. What they get right is what a first attempt gets wrong: streamed replies are not buffered, the read timeout outlasts a model's silence before its first token (nginx's 60s default is a 504), the body limit matches the router's, and X-Forwarded-Proto and -Host are set by the proxy rather than passed through — llmgw marks session cookies Secure from the first and builds the landing page's base URL from the second. The Caddyfile was validated and run against a streaming upstream: events arrive as sent, and forged forwarded headers are replaced. nginx uses `listen ... http2` because `http2 on;` needs 1.25.1 and Ubuntu 22.04 and Debian 12 ship older. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
4865a96ec2
commit
069d334652
6 files changed
+153
-2
No files matched your search
@@ -0,0 +1,36 @@
|
||||
# A reverse proxy in front of llmgw
|
||||
|
||||
The client-facing port (8080) is plain HTTP. On anything but a trusted network,
|
||||
put it behind a proxy that holds a publicly trusted certificate.
|
||||
|
||||
| File | For |
|
||||
| --- | --- |
|
||||
| [Caddyfile](Caddyfile) | Caddy — obtains and renews its own certificate; the least work |
|
||||
| [nginx.conf](nginx.conf) | nginx, with a certificate from certbot |
|
||||
|
||||
Both are also attached to every release.
|
||||
|
||||
Four things both files get right, and any other proxy has to as well:
|
||||
|
||||
- **Streaming is not buffered.** Replies are server-sent events; a buffering
|
||||
proxy delivers them in one burst at the end.
|
||||
- **The read timeout is longer than a model's silence.** nginx's default of 60
|
||||
seconds turns a slow first token into a 504.
|
||||
- **`X-Forwarded-Proto` and `X-Forwarded-Host` are set by the proxy**, replacing
|
||||
whatever the client sent. llmgw uses the first to mark session cookies
|
||||
`Secure` and the second for the base URL on its landing page.
|
||||
- **The body limit matches llmgw's** (32 MiB, `router.max_body_bytes`), so an
|
||||
oversized request is refused before it is uploaded.
|
||||
|
||||
Only port 8080 is proxied. Agents reach 7443 directly — it is mutual TLS with
|
||||
the cluster's own authority, and a proxy terminating it would take away the
|
||||
node identity it carries. 9090 (admin, pprof) stays on loopback; 9091
|
||||
(metrics) goes to your monitoring host only.
|
||||
|
||||
With a proxy on the same machine, bind llmgw to loopback so nothing reaches it
|
||||
around the proxy:
|
||||
|
||||
```yaml
|
||||
# /etc/llmgw/server.yaml
|
||||
listen: "127.0.0.1:8080"
|
||||
```
|
||||
Reference in new issue
Block a user