Close the gateway: API keys, tenancy and model allowlists
CI / test (push) Successful in 2m34s

Until now anything that could reach the client port could use every model in
the catalog. A key now resolves to an identity, and that identity decides what
it may do.

Authentication wraps the whole mux rather than checking inside each handler,
so adding a route cannot add an unauthenticated one by omission. Health stays
open, because a load balancer has no key and a health check that needs one
reports the gateway down.

Nothing on the request path reads the database. A key is resolved once and
answered from memory until the entry expires; rejections are cached too, so a
client looping with a bad key cannot turn into a database read per request.
The price is that a revocation takes up to cache_ttl to take effect, which is
the trade stated in the configuration next to the knob. A store failure is
not an authentication answer: it is never cached as one and returns 503, not
401 — sending someone hunting for a key problem that does not exist wastes the
one thing an incident is short of. A revoked key is refused with the same
message as an unknown one, because saying otherwise confirms the key was real.

With auth on the band comes from the key and the band header is ignored. The
header was trusted only while there was nothing to consult, and it let any
client promote itself into the top priority band. Tenant fairness within a
band is now per key rather than per session, so one caller cannot take a band
by opening many sessions.

Allowlists are decided per model, not per string: a model's canonical name and
every alias name the same thing, so either form works in an allowlist and no
alias can reach a model that was not allowed. The first version of that test
passed without the resolution being there at all, which is why it now checks
all four combinations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 08:44:33 +02:00
1 parent 29617f8cb1
commit 45c8cd23a3
11 files changed
+1491 -14

No files matched your search

+25 -3
View File
@@ -36,6 +36,28 @@ agent_listen: ":7443"
# cert: /etc/llmgw/server.pem
# key: /etc/llmgw/server-key.pem
# API keys on the client-facing port (plan §7.9). Off by default, because a
# gateway on a trusted network with no keys issued still has to serve — but
# nothing else protects that port, so turning it on is what makes it safe to
# expose.
#
# llmgw keys init --store /var/lib/llmgw/llmgw.db
# llmgw keys create --store /var/lib/llmgw/llmgw.db --project default "laptop"
#
# The key is printed once. Only its hash is stored, so there is nothing to
# recover it from.
#
# auth:
# enabled: true
# store: /var/lib/llmgw/llmgw.db
# # How long a resolved key is trusted before being read again. This is the
# # price of keeping the database off the request path: it bounds how long a
# # revoked key keeps working.
# cache_ttl: 30s
# # How long a rejected key is remembered as rejected, so a client looping
# # with a bad key cannot become a database read per request.
# negative_ttl: 5s
router:
scrape_interval: 2s
scrape_timeout: 1s
@@ -75,9 +97,9 @@ router:
# traffic, because an engine queue is FIFO and knows nothing about tenants.
admission:
enabled: true
# Until authentication lands, the band comes from a header and is trusted.
# Do not expose this port to callers who should not be able to promote
# themselves.
# With auth on, the key decides the band and this header is ignored. With
# auth off the header is trusted, so any client can promote itself into the
# top band — do not expose the port in that configuration.
band_header: x-llmgw-band
default_band: gold
# A band waits once pool saturation passes its ceiling, so the cluster