Ilker ManapandClaude Opus 5.5
9853936e0b
Engines listen on every address their node has, with no key. Checked on the live cluster: from a neighbouring node, a request straight to an engine's port was answered — past the gateway's authentication, quotas and accounting, by anyone who could reach the node. Each replica's engine is now started with a key and refuses requests without it. The key is derived from the replica id with an HMAC over a secret kept in the store, rather than stored per replica: the router has to send the right key to replicas this process never started — adopted after a restart, or placed by the other control plane — and a function of a shared secret gives that without a table to keep in step. A key taken from one node opens that replica only. The secret is made on first use and copied with the rest of the store when moving to PostgreSQL; a new one would lock the router out of every running engine at the moment of the move. The key reaches llama.cpp and vLLM through their environment (LLAMA_API_KEY, VLLM_API_KEY) rather than their command line, which every user on the node can read; SGLang only takes it as an argument. The agent's readiness check, the router's proxy and its metrics scrape send it, the client's own Authorization still never reaches the engine, and `ctl bench` fetches the key of the replica it is pointed at from the admin API. Engines started before this keep running without a key until their replica is next started. With no store there is nowhere to keep the secret, and engines start without one, with a warning. Also formats two files that went in unformatted. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
32 lines
879 B
Go
32 lines
879 B
Go
package store
|
|
|
|
import (
|
|
"context"
|
|
"crypto/rand"
|
|
"encoding/hex"
|
|
"time"
|
|
)
|
|
|
|
// ClusterSecret returns the named secret, creating a random one the first time
|
|
// it is asked for.
|
|
//
|
|
// Insert-if-absent and then read, so two control planes asking at once end up
|
|
// with the same value: the loser's insert changes nothing and it reads the
|
|
// winner's.
|
|
func (s *DB) ClusterSecret(ctx context.Context, name string) (string, error) {
|
|
b := make([]byte, 32)
|
|
if _, err := rand.Read(b); err != nil {
|
|
return "", err
|
|
}
|
|
if _, err := s.db.ExecContext(ctx,
|
|
`INSERT INTO cluster_secrets (name, value, created_at) VALUES (?, ?, ?)
|
|
ON CONFLICT (name) DO NOTHING`,
|
|
name, hex.EncodeToString(b), time.Now().UnixMilli()); err != nil {
|
|
return "", err
|
|
}
|
|
var v string
|
|
err := s.db.QueryRowContext(ctx,
|
|
`SELECT value FROM cluster_secrets WHERE name = ?`, name).Scan(&v)
|
|
return v, err
|
|
}
|