Ilker ManapandClaude Opus 5.5
e751c89937
The website was a file built by hand outside the repository. It is now a template in internal/web: every gateway serves it at /about, without a key, linked from the landing page's footer, and the published site is rendered from the same template. The fonts are vendored (Latin subsets, OFL) so the page sends nobody's browser to Google. The proxy configurations it shows are copies of deploy/proxy, kept equal by a test. A development build says so instead of linking to release files that do not exist. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
42 lines
1.5 KiB
Caddyfile
42 lines
1.5 KiB
Caddyfile
# llmgw behind Caddy: the client-facing port (8080) on a public name, with a
|
|
# certificate Caddy obtains and renews by itself.
|
|
#
|
|
# Replace llm.example.com, point its DNS at this machine, open 80 and 443, and
|
|
# run: caddy run --config /etc/caddy/Caddyfile
|
|
#
|
|
# Only 8080 goes through here. 7443 (agents) is mutual TLS with the cluster's
|
|
# own authority and must reach llmgw directly; 9090 and 9091 stay private.
|
|
|
|
llm.example.com {
|
|
# The router reads at most 32 MiB of body (router.max_body_bytes). Refusing
|
|
# more here gives the client a 413 before the upload, not after it.
|
|
request_body {
|
|
max_size 32MB
|
|
}
|
|
|
|
reverse_proxy 127.0.0.1:8080 {
|
|
# Tokens are written the moment they are generated. Without this a
|
|
# streamed reply can arrive in bursts, and a client measuring time to
|
|
# first token measures the proxy instead.
|
|
flush_interval -1
|
|
|
|
# Caddy sets X-Forwarded-Proto and X-Forwarded-Host itself and, with
|
|
# no trusted_proxies, discards whatever the client sent. llmgw relies
|
|
# on that: the proto decides whether session cookies are Secure, and
|
|
# the host is the base URL the landing page tells people to use.
|
|
# Behind a load balancer of your own, name it in trusted_proxies.
|
|
|
|
transport http {
|
|
# A long generation can be silent for a while before the first
|
|
# token. llmgw has its own limits (timeouts.*); this must not be
|
|
# the shorter one.
|
|
response_header_timeout 15m
|
|
}
|
|
}
|
|
|
|
# No `encode` on purpose: compressing a stream buffers it.
|
|
|
|
# Access log to stderr, which under systemd is the journal.
|
|
log
|
|
}
|