Give every engine a key only the router holds

Engines listen on every address their node has, with no key. Checked on the
live cluster: from a neighbouring node, a request straight to an engine's
port was answered — past the gateway's authentication, quotas and
accounting, by anyone who could reach the node.

Each replica's engine is now started with a key and refuses requests without
it. The key is derived from the replica id with an HMAC over a secret kept
in the store, rather than stored per replica: the router has to send the
right key to replicas this process never started — adopted after a restart,
or placed by the other control plane — and a function of a shared secret
gives that without a table to keep in step. A key taken from one node opens
that replica only. The secret is made on first use and copied with the rest
of the store when moving to PostgreSQL; a new one would lock the router out
of every running engine at the moment of the move.

The key reaches llama.cpp and vLLM through their environment
(LLAMA_API_KEY, VLLM_API_KEY) rather than their command line, which every
user on the node can read; SGLang only takes it as an argument. The agent's
readiness check, the router's proxy and its metrics scrape send it, the
client's own Authorization still never reaches the engine, and `ctl bench`
fetches the key of the replica it is pointed at from the admin API.

Engines started before this keep running without a key until their replica
is next started. With no store there is nowhere to keep the secret, and
engines start without one, with a warning.

Also formats two files that went in unformatted.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-30 00:38:29 +02:00
1 parent d167686cf1
commit 9853936e0b
34 files changed
+356 -29

No files matched your search

+20
View File
@@ -41,6 +41,11 @@ func newBenchCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
cfg.MetricsURL = metrics
cfg.Engine = engine
cfg.APIKey = key
// A replica's engine demands its own key. Asked for rather than
// typed: the admin API knows which replica answers at the target.
if cfg.APIKey == "" {
cfg.APIKey = replicaKey(cmd, client, target)
}
if len(steps) > 0 {
cfg.Concurrency = steps
}
@@ -227,3 +232,18 @@ func ms(d time.Duration) string {
return d.Round(10 * time.Millisecond).String()
}
}
// replicaKey is the engine key of the replica at target, or "" when target is
// not a replica (the gateway, say) or the admin API cannot be asked.
func replicaKey(cmd *cobra.Command, client func() *ctlClient, target string) string {
var reps []adminapi.ReplicaView
if err := client().get(cmd.Context(), "/api/v1/replicas", &reps); err != nil {
return ""
}
for _, rep := range reps {
if sameEndpoint(rep.Endpoint, target) {
return rep.EngineKey
}
}
return ""
}
+14
View File
@@ -385,6 +385,20 @@ func startControlPlane(ctx context.Context, cfg config.Server, cat *catalog.Live
ccfg.Reconciler.PortRange = [2]int{cfg.Control.PortRange[0], cfg.Control.PortRange[1]}
}
// Each replica's engine demands a key only the router has. The secret it
// is derived from lives in the store, so every control plane against it —
// and this one after a restart — computes the same key for a replica.
if h, ok := sink.(interface{ Store() *storepkg.DB }); ok && h.Store() != nil {
secret, err := h.Store().ClusterSecret(ctx, "engine-key")
if err != nil {
return nil, nil, fmt.Errorf("reading the engine key secret: %w", err)
}
ccfg.Reconciler.EngineKey = control.EngineKeys([]byte(secret))
} else {
log.Warn("engines start without a key: there is no store to keep the secret " +
"they are derived from, so anyone who can reach a node's ports reaches its models")
}
// The service needs the inventory and the status sink, both of which the
// controller owns; the controller needs the service as its dispatcher. The
// cycle is broken by giving the controller a dispatcher that forwards to