Ilker ManapandClaude Opus 5.5
e751c89937
The website was a file built by hand outside the repository. It is now a template in internal/web: every gateway serves it at /about, without a key, linked from the landing page's footer, and the published site is rendered from the same template. The fonts are vendored (Latin subsets, OFL) so the page sends nobody's browser to Google. The proxy configurations it shows are copies of deploy/proxy, kept equal by a test. A development build says so instead of linking to release files that do not exist. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
40 lines
1.6 KiB
Markdown
40 lines
1.6 KiB
Markdown
# A reverse proxy in front of llmgw
|
|
|
|
The client-facing port (8080) is plain HTTP. On anything but a trusted network,
|
|
put it behind a proxy that holds a publicly trusted certificate.
|
|
|
|
| File | For |
|
|
| --- | --- |
|
|
| [Caddyfile](Caddyfile) | Caddy — obtains and renews its own certificate; the least work |
|
|
| [nginx.conf](nginx.conf) | nginx, with a certificate from certbot |
|
|
|
|
Both are also attached to every release.
|
|
|
|
The `/about` page shows both, from copies in `internal/web/proxy`; a test fails
|
|
when those differ from these, so change them here and copy them over.
|
|
|
|
Four things both files get right, and any other proxy has to as well:
|
|
|
|
- **Streaming is not buffered.** Replies are server-sent events; a buffering
|
|
proxy delivers them in one burst at the end.
|
|
- **The read timeout is longer than a model's silence.** nginx's default of 60
|
|
seconds turns a slow first token into a 504.
|
|
- **`X-Forwarded-Proto` and `X-Forwarded-Host` are set by the proxy**, replacing
|
|
whatever the client sent. llmgw uses the first to mark session cookies
|
|
`Secure` and the second for the base URL on its landing page.
|
|
- **The body limit matches llmgw's** (32 MiB, `router.max_body_bytes`), so an
|
|
oversized request is refused before it is uploaded.
|
|
|
|
Only port 8080 is proxied. Agents reach 7443 directly — it is mutual TLS with
|
|
the cluster's own authority, and a proxy terminating it would take away the
|
|
node identity it carries. 9090 (admin, pprof) stays on loopback; 9091
|
|
(metrics) goes to your monitoring host only.
|
|
|
|
With a proxy on the same machine, bind llmgw to loopback so nothing reaches it
|
|
around the proxy:
|
|
|
|
```yaml
|
|
# /etc/llmgw/server.yaml
|
|
listen: "127.0.0.1:8080"
|
|
```
|