Give every engine a key only the router holds

Engines listen on every address their node has, with no key. Checked on the
live cluster: from a neighbouring node, a request straight to an engine's
port was answered — past the gateway's authentication, quotas and
accounting, by anyone who could reach the node.

Each replica's engine is now started with a key and refuses requests without
it. The key is derived from the replica id with an HMAC over a secret kept
in the store, rather than stored per replica: the router has to send the
right key to replicas this process never started — adopted after a restart,
or placed by the other control plane — and a function of a shared secret
gives that without a table to keep in step. A key taken from one node opens
that replica only. The secret is made on first use and copied with the rest
of the store when moving to PostgreSQL; a new one would lock the router out
of every running engine at the moment of the move.

The key reaches llama.cpp and vLLM through their environment
(LLAMA_API_KEY, VLLM_API_KEY) rather than their command line, which every
user on the node can read; SGLang only takes it as an argument. The agent's
readiness check, the router's proxy and its metrics scrape send it, the
client's own Authorization still never reaches the engine, and `ctl bench`
fetches the key of the replica it is pointed at from the admin API.

Engines started before this keep running without a key until their replica
is next started. With no store there is nowhere to keep the secret, and
engines start without one, with a warning.

Also formats two files that went in unformatted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-30 00:38:29 +02:00
1 parent d167686cf1
commit 9853936e0b
34 files changed
+356 -29

No files matched your search

+30
View File
@@ -4,6 +4,9 @@ package control
import (
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"fmt"
"log/slog"
"strings"
@@ -304,6 +307,7 @@ func (c *Controller) endpoints() []endpoint.Info {
ID: rep.ID,
URL: strings.TrimRight(rep.Endpoint, "/"),
Models: names,
APIKey: c.engineKey(rep.ID),
Engine: prof.Engine,
Pool: rep.Pool,
Weight: rep.Weight,
@@ -321,3 +325,29 @@ func fingerprint(infos []endpoint.Info) string {
}
return b.String()
}
// EngineKey is the key a replica's engine demands, empty when engines run
// without one.
func (c *Controller) EngineKey(replicaID string) string { return c.engineKey(replicaID) }
func (c *Controller) engineKey(replicaID string) string {
if c.cfg.Reconciler.EngineKey == nil {
return ""
}
return c.cfg.Reconciler.EngineKey(replicaID)
}
// EngineKeys derives each replica's engine key from one secret.
//
// HMAC rather than a stored random key per replica: the router needs the key
// for every replica it sends to, including ones adopted after a restart that
// this process never started, and a second control plane has to agree. A
// function of a shared secret and the replica id gives all three without a
// table to keep in step. A key taken from one node opens that replica only.
func EngineKeys(secret []byte) func(replicaID string) string {
return func(replicaID string) string {
m := hmac.New(sha256.New, secret)
m.Write([]byte("llmgw engine key\x00" + replicaID))
return "llmgw-" + hex.EncodeToString(m.Sum(nil))[:40]
}
}