Add the store: tenancy, keys and usage on SQLite

Phase 5 begins. This is the persistence layer for the things the cluster
cannot reconstruct — who may call, what they are allowed, what they used.
Everything else llmgw knows is still derived from the fleet, which cannot be
wrong about itself the way a database can (§6.5).

SQLite behind an interface. The plan justified that by write volume, which was
the wrong argument: the write path is batched anyway, and a 500-row
transaction every 200ms is nothing for either engine. The real argument is the
failure mode. Postgres is a second service, and a gateway that stops serving
because its accounting database is unreachable is worse than one that buffers
some accounting. Embedded means the store shares the gateway's fate instead of
adding an independent way to be down. That reasoning stops holding at two
instances, where budgets have to be counted somewhere shared — so the
interface is here from the start, the SQL is portable, and migrations are per
dialect with shared version numbers.

Keys are stored as sha256 and nothing else, so a database leak hands over no
working credential. The fast hash is deliberate: the input is 256 random bits
with no dictionary to attack, so argon2 would buy nothing and charge tens of
milliseconds per request. That assumption is written down next to the code
that depends on it.

Two things the schema now says that it did not: budgets are integer
micro-units, because repeated addition of floats drifts; and an empty model
allowlist forbids everything while an absent one permits everything, which the
resolver has to keep distinct.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-16 08:37:40 +02:00
1 parent 41cf894c72
commit 29617f8cb1
10 files changed
+1585 -9

No files matched your search

+10 -1
View File
@@ -14,23 +14,32 @@ require (
github.com/prometheus/client_model v0.6.3
github.com/prometheus/common v0.71.0
github.com/spf13/cobra v1.10.2
golang.org/x/net v0.59.0
golang.org/x/sync v0.23.0
google.golang.org/protobuf v1.36.12
modernc.org/sqlite v1.59.0
)
require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/google/uuid v1.6.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/knadh/koanf/maps v0.1.2 // indirect
github.com/mattn/go-isatty v0.0.24 // indirect
github.com/mitchellh/copystructure v1.2.0 // indirect
github.com/mitchellh/reflectwalk v1.0.2 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/prometheus/procfs v0.21.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/spf13/pflag v1.0.9 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
golang.org/x/net v0.59.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/text v0.42.0 // indirect
modernc.org/libc v1.75.7 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.12.1 // indirect
)
+46
View File
@@ -7,12 +7,20 @@ github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6r
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3 h1:LMLX+LgTNWpfvCBdFebv6EsYotImrt/Ppc5cXIriCSo=
github.com/google/pprof v0.0.0-20260802141513-ef3492d7dac3/go.mod h1:jl5iWTm0/hd5PjEYEOuwAJ57L/CibdZfrqZ5XA5GrCk=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
@@ -29,12 +37,16 @@ github.com/knadh/koanf/v2 v2.3.6 h1:JoQPSJmvS4aP0xNc8xMDr5tcrkSEInL23/Il7pITAKo=
github.com/knadh/koanf/v2 v2.3.6/go.mod h1:gRb40VRAbd4iJMYYD5IxZ6hfuopFcXBpc9bbQpZwo28=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/mattn/go-isatty v0.0.24 h1:tGZZoVgT/KiqK1c8ocVLeDS8BSWMRd47J3Lbz7vsReI=
github.com/mattn/go-isatty v0.0.24/go.mod h1:nMCL3Zebbrt45jsMDgnfIwz6ydEQApk5oEI3HqDio6A=
github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/prometheus/client_golang v1.24.1 h1:JnJkREXzWxUdCuPFpIWZiPispT9xVV59uiuyR2bPlnU=
github.com/prometheus/client_golang v1.24.1/go.mod h1:F+oSRECHg4sse5ucfYpYDeIv/hu68Zo0uoHKetWnzcE=
github.com/prometheus/client_model v0.6.3 h1:O0jaTVAYNxTHYInEPFJt5I3+sN8zqBtVMPTB1qyxiEo=
@@ -43,6 +55,8 @@ github.com/prometheus/common v0.71.0 h1:9KDAKb7Mj3HEVKyFCK6Dc/HIwlBzZIN2l7/lrHl3
github.com/prometheus/common v0.71.0/go.mod h1:CLJ5H8TEsGX8bl31BdMkfhIZ+QmZ9tBPPotUxUbfcmk=
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
@@ -57,6 +71,8 @@ go.yaml.in/yaml/v2 v2.4.4/go.mod h1:gMZqIpDtDqOfM0uNfy0SkpRhvUryYH0Z6wdMYcacYXQ=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
go.yaml.in/yaml/v3 v3.0.5 h1:N6y/pJk8buWs9NY5ERU2HSMfm+IuD/OtfdAnq6kESPw=
go.yaml.in/yaml/v3 v3.0.5/go.mod h1:HVTZu1O7/Vkt2N+BFy8Zza+lnLsABggaTM2ZpNIGuKg=
golang.org/x/mod v0.41.0 h1:qJmnOUb4YB+FsEuM3HcWucdZASCPGhsX6uljO6pog0c=
golang.org/x/mod v0.41.0/go.mod h1:Ek9pY8RKWXwsWvd3rQiHYtMqkjSUV+s1Rj7j4H5Ur6o=
golang.org/x/net v0.59.0 h1:5zfYln+w5XCxwrnMMJPufRgNoXEaGxl0wo5GqPXyues=
golang.org/x/net v0.59.0/go.mod h1:2DA/G1UfVbCpQPeWTmMPGY7Cs2PkBkwu743bVX5PIVg=
golang.org/x/sync v0.23.0 h1:KameEIfc1IkluZyXWLn39Wd4tURc6GbCiISGiZm2bQk=
@@ -65,6 +81,36 @@ golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/text v0.42.0 h1:JbOZXgfeCPU9gacVtYliJqOhD+zhrEqK4LfdpmlUZqI=
golang.org/x/text v0.42.0/go.mod h1:ojzP1Z+2QtioaF8DTtO8K5q7JWVVYwZKenzujK0Zd0E=
golang.org/x/tools v0.49.0 h1:3NI7VXzL9+1WZD52Dx2ttoPwD5DWrFGpl9mFZDlmisI=
golang.org/x/tools v0.49.0/go.mod h1:SJNXV9DBKT0UbdttsQjbfJlAE/q+y36++zo3uL3N0Oo=
google.golang.org/protobuf v1.36.12 h1:pJOKDDOyeXErUroCihFAd5LQuwXBSpVnKGrj5o/fwxc=
google.golang.org/protobuf v1.36.12/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
modernc.org/cc/v4 v4.29.2 h1:h6+9ciCnPKutf4I03CvheAvDLX7+IHlqR6Iy6J+cgd8=
modernc.org/cc/v4 v4.29.2/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.35.0 h1:F+TUsmw09QxLzmi3aeYYGxjAXarmZaKgj3mKQHNaA8w=
modernc.org/ccgo/v4 v4.35.0/go.mod h1:qrVGs9S3Sr2Ztcg9ve+kTAYMp5a3YvWjo+SoN06kJ5I=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.5 h1:21ldfPfRYE31Tb7B3mwAK8gy1AxP4+dKjrOQPfqakoc=
modernc.org/gc/v3 v3.1.5/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.75.7 h1:o3DTP9/0p9pKmY2WCKQaySW6wIiZhNM7wc2lUoyhfew=
modernc.org/libc v1.75.7/go.mod h1:bO5o2ztHxBb2rjz0PgdHN0sSMw57CgxGFLZ3Qd/QpVQ=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.12.1 h1:nFMiWrpStgZczNl6XI9GnIk/rWhYIyHGUaR04pGbp9g=
modernc.org/memory v1.12.1/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.59.0 h1:X1es1GpqBlS/5T+vbM4HLUdaa8OtQx468DF2vrx+38A=
modernc.org/sqlite v1.59.0/go.mod h1:+paeT2A3iPRHkQDwG7oA6Tk0zQd5woMEI8q7orfry8k=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
+124
View File
@@ -0,0 +1,124 @@
package store
import (
"crypto/rand"
"crypto/sha256"
"crypto/subtle"
"errors"
"math/big"
"strings"
)
// KeyPrefix is what every issued key starts with, so a leaked key is
// recognizable in a log or a repository scan.
const KeyPrefix = "sk-llmgw-"
// keyBytes is 256 bits of randomness. Enough that guessing is infeasible at any
// hash rate, which is the assumption the fast hash below rests on.
const keyBytes = 32
const base62 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
// NewKey mints a key and returns both the secret, which the caller must show
// once and then forget, and the record to store.
//
// The two are deliberately returned together and separately: nothing in the
// record can reconstruct the secret, so the caller either shows it now or it is
// gone.
func NewKey(id, name, projectID string) (secret string, k Key, err error) {
raw := make([]byte, keyBytes)
if _, err := rand.Read(raw); err != nil {
return "", Key{}, err
}
secret = KeyPrefix + encodeBase62(raw)
sum := sha256.Sum256([]byte(secret))
return secret, Key{
ID: id,
Hash: sum[:],
Prefix: DisplayPrefix(secret),
Name: name,
ProjectID: projectID,
}, nil
}
// HashKey is what authentication computes from what a client presented.
//
// SHA-256 rather than bcrypt or argon2, on purpose. Slow password hashing
// exists because humans pick guessable passwords, and its cost buys time
// against an offline dictionary attack. A 256-bit random value has no
// dictionary: guessing it is infeasible at any hash rate. The slow hash would
// buy nothing here and charge tens of milliseconds on every request.
//
// This is the assumption to re-examine if the key format ever changes to
// something with less entropy.
func HashKey(secret string) []byte {
sum := sha256.Sum256([]byte(secret))
return sum[:]
}
// EqualHash compares two hashes without leaking where they differ.
//
// The lookup is by hash so timing tells an attacker little, but a key
// comparison is exactly the place not to economize.
func EqualHash(a, b []byte) bool { return subtle.ConstantTimeCompare(a, b) == 1 }
// DisplayPrefix is the part of a key safe to show: enough to tell two keys
// apart in a listing, far too little to use.
func DisplayPrefix(secret string) string {
body := strings.TrimPrefix(secret, KeyPrefix)
if len(body) > 4 {
body = body[:4]
}
return KeyPrefix + body + "…"
}
// ErrMalformedKey says the presented credential is not one of ours.
var ErrMalformedKey = errors.New("not an llmgw key")
// ParseKey checks the shape of a presented credential before it is hashed.
//
// Cheap rejection of everything that cannot possibly be a key — a bearer token
// from another system, a truncated copy/paste — so those never reach the store
// at all.
func ParseKey(presented string) error {
if !strings.HasPrefix(presented, KeyPrefix) {
return ErrMalformedKey
}
body := presented[len(KeyPrefix):]
if len(body) < 40 {
return ErrMalformedKey
}
for _, r := range body {
if !strings.ContainsRune(base62, r) {
return ErrMalformedKey
}
}
return nil
}
// encodeBase62 renders bytes as a base62 string.
//
// Base62 rather than base64 so a key survives being pasted into a URL, a shell
// argument or a YAML file without quoting or escaping.
func encodeBase62(b []byte) string {
n := new(big.Int).SetBytes(b)
base := big.NewInt(62)
mod := new(big.Int)
var out []byte
for n.Sign() > 0 {
n.DivMod(n, base, mod)
out = append(out, base62[mod.Int64()])
}
// Leading zero bytes carry no digits, so pad to a fixed width; a key of
// varying length would leak a little about its value and look inconsistent.
for len(out) < 43 {
out = append(out, base62[0])
}
// Built least-significant first.
for i, j := 0, len(out)-1; i < j; i, j = i+1, j-1 {
out[i], out[j] = out[j], out[i]
}
return string(out)
}
+78
View File
@@ -0,0 +1,78 @@
package store
import (
"crypto/sha256"
"strings"
"testing"
)
func TestMintedKeysAreUniqueAndWellFormed(t *testing.T) {
seen := map[string]struct{}{}
for range 200 {
secret, k, err := NewKey("id", "name", "proj")
if err != nil {
t.Fatal(err)
}
if _, dup := seen[secret]; dup {
t.Fatal("two keys came out identical")
}
seen[secret] = struct{}{}
if err := ParseKey(secret); err != nil {
t.Fatalf("a minted key does not parse: %v (%q)", err, secret)
}
// Fixed width, so a key's length says nothing about its value and
// listings line up.
if len(secret) != len(KeyPrefix)+43 {
t.Fatalf("length %d: %q", len(secret), secret)
}
sum := sha256.Sum256([]byte(secret))
if string(k.Hash) != string(sum[:]) {
t.Fatal("the stored hash is not the hash of the key")
}
}
}
func TestParseRejectsWhatCannotBeAKey(t *testing.T) {
// Cheap rejection, so a bearer token from another system never reaches the
// database at all.
for _, bad := range []string{
"", "hello", "Bearer abc",
"sk-llmgw-",
"sk-llmgw-short",
"sk-proj-0123456789012345678901234567890123456789012",
"sk-llmgw-0123456789012345678901234567890123456789+/=",
} {
if err := ParseKey(bad); err == nil {
t.Errorf("%q was accepted as a key", bad)
}
}
}
func TestDisplayPrefixRevealsAlmostNothing(t *testing.T) {
secret, k, _ := NewKey("id", "name", "proj")
if !strings.HasPrefix(secret, k.Prefix[:len(KeyPrefix)+4]) {
t.Errorf("prefix %q does not belong to %q", k.Prefix, secret)
}
// Four characters of base62 is enough to tell keys apart in a listing and
// about 15 million times too few to guess the rest.
body := strings.TrimSuffix(strings.TrimPrefix(k.Prefix, KeyPrefix), "…")
if len(body) != 4 {
t.Errorf("the display prefix shows %d characters", len(body))
}
}
func TestEqualHashIsUsable(t *testing.T) {
a := HashKey("sk-llmgw-aaa")
b := HashKey("sk-llmgw-aaa")
c := HashKey("sk-llmgw-bbb")
if !EqualHash(a, b) {
t.Error("equal hashes compared unequal")
}
if EqualHash(a, c) {
t.Error("different hashes compared equal")
}
if EqualHash(a, c[:16]) {
t.Error("a truncated hash compared equal")
}
}
+106
View File
@@ -0,0 +1,106 @@
package store
import (
"context"
"database/sql"
"embed"
"fmt"
"io/fs"
"sort"
"strings"
)
//go:embed migrations/sqlite/*.sql
var migrationFS embed.FS
// migrate applies every unapplied migration in order, each in its own
// transaction, and records it.
//
// Deliberately small. A migration tool would be a dependency and a second way
// to be wrong about which migrations have run; the whole mechanism is a sorted
// directory, a version table, and a transaction each.
//
// The directory is per dialect. SQLite and Postgres diverge enough in DDL —
// SQLite's ALTER TABLE barely exists — that a single shared set would be a
// promise this could not keep. Same version numbers, same meaning, different
// files.
func migrate(ctx context.Context, db *sql.DB, dialect string) error {
if _, err := db.ExecContext(ctx,
`CREATE TABLE IF NOT EXISTS schema_migrations (version TEXT PRIMARY KEY, applied_at INTEGER NOT NULL)`,
); err != nil {
return fmt.Errorf("creating the migration table: %w", err)
}
applied := map[string]struct{}{}
rows, err := db.QueryContext(ctx, `SELECT version FROM schema_migrations`)
if err != nil {
return err
}
for rows.Next() {
var v string
if err := rows.Scan(&v); err != nil {
rows.Close()
return err
}
applied[v] = struct{}{}
}
if err := rows.Err(); err != nil {
rows.Close()
return err
}
rows.Close()
names, err := migrationNames(dialect)
if err != nil {
return err
}
for _, name := range names {
if _, done := applied[name]; done {
continue
}
body, err := migrationFS.ReadFile("migrations/" + dialect + "/" + name)
if err != nil {
return err
}
if err := applyOne(ctx, db, name, string(body)); err != nil {
return fmt.Errorf("migration %s: %w", name, err)
}
}
return nil
}
func migrationNames(dialect string) ([]string, error) {
entries, err := fs.ReadDir(migrationFS, "migrations/"+dialect)
if err != nil {
return nil, fmt.Errorf("no migrations for dialect %q: %w", dialect, err)
}
var names []string
for _, e := range entries {
if !e.IsDir() && strings.HasSuffix(e.Name(), ".sql") {
names = append(names, e.Name())
}
}
// Lexical order is version order, which is why the files are zero padded.
sort.Strings(names)
return names, nil
}
func applyOne(ctx context.Context, db *sql.DB, name, body string) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
if _, err := tx.ExecContext(ctx, body); err != nil {
return err
}
if _, err := tx.ExecContext(ctx,
`INSERT INTO schema_migrations (version, applied_at) VALUES (?, ?)`,
name, nowMillis(),
); err != nil {
return err
}
return tx.Commit()
}
@@ -0,0 +1,72 @@
-- The tenancy tree. Three levels, each owning the next.
CREATE TABLE organizations (
id TEXT PRIMARY KEY,
name TEXT NOT NULL
);
CREATE TABLE teams (
id TEXT PRIMARY KEY,
org_id TEXT NOT NULL REFERENCES organizations(id),
name TEXT NOT NULL
);
CREATE INDEX teams_by_org ON teams(org_id);
CREATE TABLE projects (
id TEXT PRIMARY KEY,
team_id TEXT NOT NULL REFERENCES teams(id),
name TEXT NOT NULL
);
CREATE INDEX projects_by_team ON projects(team_id);
-- Only sha256(key) is stored, so a database leak hands over no working
-- credential. SHA-256 rather than a slow password hash on purpose: the input is
-- 256 random bits with no dictionary to attack, so the cost of argon2 would buy
-- nothing and charge tens of milliseconds on every request (plan §7.9.1).
CREATE TABLE api_keys (
id TEXT PRIMARY KEY,
key_hash BLOB NOT NULL UNIQUE,
key_prefix TEXT NOT NULL,
name TEXT NOT NULL,
project_id TEXT NOT NULL REFERENCES projects(id),
band TEXT,
routing_profile TEXT,
created_at INTEGER NOT NULL,
expires_at INTEGER,
-- Revocation is a soft delete. Usage rows reference the key, and an audit
-- trail that loses its subject is not an audit trail.
revoked_at INTEGER,
last_used_at INTEGER
);
CREATE INDEX api_keys_by_project ON api_keys(project_id);
-- Limits hang off any scope rather than being duplicated onto each one. A NULL
-- column means "no limit at this level", which is not the same as 0.
CREATE TABLE limits (
scope_type TEXT NOT NULL,
scope_id TEXT NOT NULL,
rpm INTEGER,
tpm INTEGER,
-- Micro-units, not a float: a budget is accumulated by repeated addition, and
-- repeated addition of floats drifts.
budget_micros INTEGER,
budget_period TEXT,
allowed_models TEXT,
PRIMARY KEY (scope_type, scope_id)
);
-- One row per request, written in batches off the request path.
CREATE TABLE usage (
ts INTEGER NOT NULL,
key_id TEXT NOT NULL,
project_id TEXT NOT NULL,
model TEXT NOT NULL,
served_model TEXT NOT NULL,
prompt_tokens INTEGER NOT NULL,
completion_tokens INTEGER NOT NULL,
cost_micros INTEGER NOT NULL,
status INTEGER NOT NULL,
latency_ms INTEGER NOT NULL
);
-- Budget restore reads by key within a period, which is this index exactly.
CREATE INDEX usage_by_key_ts ON usage(key_id, ts);
CREATE INDEX usage_by_ts ON usage(ts);
+506
View File
@@ -0,0 +1,506 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"slices"
"strings"
"time"
_ "modernc.org/sqlite" // the cgo-free driver, so the binary stays static
)
// SQLite is the single-instance store.
type SQLite struct {
db *sql.DB
}
// OpenSQLite opens (creating if needed) a database and brings it up to date.
//
// The pragmas are the ones that matter. WAL gives concurrent readers alongside
// the single writer, which is the access pattern exactly: many reads for
// authentication, one batched writer for usage. foreign_keys is off by default
// in SQLite, which would silently let the tenancy tree grow orphans. busy_timeout
// turns "database is locked" from an error into a wait, which is what a caller
// wants when the batch writer holds the write lock for a few milliseconds.
func OpenSQLite(ctx context.Context, path string) (*SQLite, error) {
dsn := path + "?_pragma=journal_mode(WAL)&_pragma=foreign_keys(1)&_pragma=busy_timeout(5000)&_pragma=synchronous(NORMAL)"
db, err := sql.Open("sqlite", dsn)
if err != nil {
return nil, err
}
// One writer is what SQLite supports; more connections would only produce
// lock contention that looks like a deadlock under load.
db.SetMaxOpenConns(1)
if err := db.PingContext(ctx); err != nil {
_ = db.Close()
return nil, fmt.Errorf("opening %s: %w", path, err)
}
if err := migrate(ctx, db, "sqlite"); err != nil {
_ = db.Close()
return nil, err
}
return &SQLite{db: db}, nil
}
// Close releases the database.
func (s *SQLite) Close() error { return s.db.Close() }
// --- tenancy ---
// PutOrg upserts an organization.
func (s *SQLite) PutOrg(ctx context.Context, o Org) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO organizations (id, name) VALUES (?, ?)
ON CONFLICT (id) DO UPDATE SET name = excluded.name`, o.ID, o.Name)
return err
}
// PutTeam upserts a team.
func (s *SQLite) PutTeam(ctx context.Context, t Team) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO teams (id, org_id, name) VALUES (?, ?, ?)
ON CONFLICT (id) DO UPDATE SET org_id = excluded.org_id, name = excluded.name`,
t.ID, t.OrgID, t.Name)
return err
}
// PutProject upserts a project.
func (s *SQLite) PutProject(ctx context.Context, p Project) error {
_, err := s.db.ExecContext(ctx,
`INSERT INTO projects (id, team_id, name) VALUES (?, ?, ?)
ON CONFLICT (id) DO UPDATE SET team_id = excluded.team_id, name = excluded.name`,
p.ID, p.TeamID, p.Name)
return err
}
// Orgs lists every organization.
func (s *SQLite) Orgs(ctx context.Context) ([]Org, error) {
return queryAll(ctx, s.db, `SELECT id, name FROM organizations ORDER BY name`,
nil, func(rows *sql.Rows) (Org, error) {
var o Org
return o, rows.Scan(&o.ID, &o.Name)
})
}
// Teams lists an organization's teams.
func (s *SQLite) Teams(ctx context.Context, orgID string) ([]Team, error) {
return queryAll(ctx, s.db, `SELECT id, org_id, name FROM teams WHERE org_id = ? ORDER BY name`,
[]any{orgID}, func(rows *sql.Rows) (Team, error) {
var t Team
return t, rows.Scan(&t.ID, &t.OrgID, &t.Name)
})
}
// Projects lists a team's projects.
func (s *SQLite) Projects(ctx context.Context, teamID string) ([]Project, error) {
return queryAll(ctx, s.db, `SELECT id, team_id, name FROM projects WHERE team_id = ? ORDER BY name`,
[]any{teamID}, func(rows *sql.Rows) (Project, error) {
var p Project
return p, rows.Scan(&p.ID, &p.TeamID, &p.Name)
})
}
// --- keys ---
const keyColumns = `id, key_hash, key_prefix, name, project_id, band, routing_profile,
created_at, expires_at, revoked_at, last_used_at`
// CreateKey stores a key. Only its hash is written; the key itself is the
// caller's to show once and then forget.
func (s *SQLite) CreateKey(ctx context.Context, k Key) error {
if len(k.Hash) == 0 {
return errors.New("store: a key needs a hash")
}
_, err := s.db.ExecContext(ctx,
`INSERT INTO api_keys (`+keyColumns+`) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
k.ID, k.Hash, k.Prefix, k.Name, k.ProjectID,
nullString(k.Band), nullString(k.RoutingProfile),
millis(k.CreatedAt), nullMillis(k.ExpiresAt), nullMillis(k.RevokedAt), nullMillis(k.LastUsedAt))
return err
}
// KeyByHash finds a key by the hash of the presented secret.
func (s *SQLite) KeyByHash(ctx context.Context, hash []byte) (Key, error) {
row := s.db.QueryRowContext(ctx,
`SELECT `+keyColumns+` FROM api_keys WHERE key_hash = ?`, hash)
k, err := scanKey(row)
if errors.Is(err, sql.ErrNoRows) {
return Key{}, ErrNotFound
}
return k, err
}
// Keys lists a project's keys, revoked ones included.
func (s *SQLite) Keys(ctx context.Context, projectID string) ([]Key, error) {
return queryAll(ctx, s.db,
`SELECT `+keyColumns+` FROM api_keys WHERE project_id = ? ORDER BY created_at`,
[]any{projectID}, func(rows *sql.Rows) (Key, error) { return scanKey(rows) })
}
// RevokeKey marks a key unusable without deleting it.
func (s *SQLite) RevokeKey(ctx context.Context, id string, at time.Time) error {
res, err := s.db.ExecContext(ctx,
`UPDATE api_keys SET revoked_at = ? WHERE id = ? AND revoked_at IS NULL`,
millis(at), id)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
// Either it does not exist or it was already revoked. Both mean the key
// cannot authenticate, so only the first is worth an error.
var exists int
if err := s.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM api_keys WHERE id = ?`, id).
Scan(&exists); err != nil {
return err
}
if exists == 0 {
return ErrNotFound
}
}
return nil
}
// TouchKeys records when keys were last used.
//
// Batched, and at most once a minute per key, because nothing reads this field
// in real time and writing it per request would put a database write back on
// the request path.
func (s *SQLite) TouchKeys(ctx context.Context, seen map[string]time.Time) error {
if len(seen) == 0 {
return nil
}
return s.inTx(ctx, func(tx *sql.Tx) error {
stmt, err := tx.PrepareContext(ctx,
`UPDATE api_keys SET last_used_at = ? WHERE id = ? AND (last_used_at IS NULL OR last_used_at < ?)`)
if err != nil {
return err
}
defer stmt.Close()
for id, at := range seen {
if _, err := stmt.ExecContext(ctx, millis(at), id, millis(at)); err != nil {
return err
}
}
return nil
})
}
// --- limits ---
// PutLimits upserts the limits at one scope.
func (s *SQLite) PutLimits(ctx context.Context, l Limits) error {
models, err := encodeModels(l.AllowedModels)
if err != nil {
return err
}
_, err = s.db.ExecContext(ctx,
`INSERT INTO limits (scope_type, scope_id, rpm, tpm, budget_micros, budget_period, allowed_models)
VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (scope_type, scope_id) DO UPDATE SET
rpm = excluded.rpm, tpm = excluded.tpm,
budget_micros = excluded.budget_micros, budget_period = excluded.budget_period,
allowed_models = excluded.allowed_models`,
string(l.Scope), l.ScopeID, nullInt(l.RPM), nullInt(l.TPM),
nullInt64(l.Budget), nullString(string(l.BudgetPeriod)), models)
return err
}
// LimitsFor reads the limits set at one scope, without inheritance.
func (s *SQLite) LimitsFor(ctx context.Context, scope Scope, id string) (Limits, error) {
row := s.db.QueryRowContext(ctx,
`SELECT rpm, tpm, budget_micros, budget_period, allowed_models
FROM limits WHERE scope_type = ? AND scope_id = ?`, string(scope), id)
l := Limits{Scope: scope, ScopeID: id}
var rpm, tpm sql.NullInt64
var budget sql.NullInt64
var period, models sql.NullString
if err := row.Scan(&rpm, &tpm, &budget, &period, &models); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return Limits{}, ErrNotFound
}
return Limits{}, err
}
if rpm.Valid {
v := int(rpm.Int64)
l.RPM = &v
}
if tpm.Valid {
v := int(tpm.Int64)
l.TPM = &v
}
if budget.Valid {
v := budget.Int64
l.Budget = &v
}
l.BudgetPeriod = Period(period.String)
var err error
if l.AllowedModels, err = decodeModels(models); err != nil {
return Limits{}, err
}
return l, nil
}
// Resolve walks a key's chain and returns the most restrictive of each limit.
//
// One query rather than four lookups: the chain is a fixed three-level join, and
// this is called on a cache miss during authentication.
func (s *SQLite) Resolve(ctx context.Context, keyID string) (Effective, error) {
var e Effective
row := s.db.QueryRowContext(ctx,
`SELECT k.id, k.band, k.routing_profile, p.id, t.id, o.id
FROM api_keys k
JOIN projects p ON p.id = k.project_id
JOIN teams t ON t.id = p.team_id
JOIN organizations o ON o.id = t.org_id
WHERE k.id = ?`, keyID)
var band, profile sql.NullString
if err := row.Scan(&e.KeyID, &band, &profile, &e.ProjectID, &e.TeamID, &e.OrgID); err != nil {
if errors.Is(err, sql.ErrNoRows) {
return Effective{}, ErrNotFound
}
return Effective{}, err
}
e.Band, e.RoutingProfile = band.String, profile.String
// Widest first, so a narrower scope can only tighten what it inherits.
chain := []struct {
scope Scope
id string
}{
{ScopeOrg, e.OrgID},
{ScopeTeam, e.TeamID},
{ScopeProject, e.ProjectID},
{ScopeKey, e.KeyID},
}
for _, step := range chain {
l, err := s.LimitsFor(ctx, step.scope, step.id)
if errors.Is(err, ErrNotFound) {
continue
}
if err != nil {
return Effective{}, err
}
e.tighten(l)
}
return e, nil
}
// tighten applies one scope's limits, keeping whichever is stricter.
func (e *Effective) tighten(l Limits) {
e.RPM = minPtr(e.RPM, l.RPM)
e.TPM = minPtr(e.TPM, l.TPM)
if l.Budget != nil && (e.Budget == nil || *l.Budget < *e.Budget) {
e.Budget = l.Budget
e.BudgetPeriod = l.BudgetPeriod
}
if l.AllowedModels != nil {
if e.AllowedModels == nil {
e.AllowedModels = slices.Clone(l.AllowedModels)
} else {
// Intersection: a narrower scope cannot add a model its parent
// withheld, or an allowlist would be advisory.
var keep []string
for _, m := range e.AllowedModels {
if slices.Contains(l.AllowedModels, m) {
keep = append(keep, m)
}
}
// An empty intersection is still an allowlist, and an empty one.
if keep == nil {
keep = []string{}
}
e.AllowedModels = keep
}
}
}
// --- accounting ---
// WriteUsage appends a batch of records in one transaction.
func (s *SQLite) WriteUsage(ctx context.Context, records []Usage) error {
if len(records) == 0 {
return nil
}
return s.inTx(ctx, func(tx *sql.Tx) error {
stmt, err := tx.PrepareContext(ctx,
`INSERT INTO usage (ts, key_id, project_id, model, served_model,
prompt_tokens, completion_tokens, cost_micros, status, latency_ms)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`)
if err != nil {
return err
}
defer stmt.Close()
for _, r := range records {
if _, err := stmt.ExecContext(ctx, millis(r.At), r.KeyID, r.ProjectID,
r.Model, r.ServedModel, r.PromptTokens, r.CompletionTokens,
r.CostMicros, r.Status, r.LatencyMs); err != nil {
return err
}
}
return nil
})
}
// SpendSince totals cost per key since a moment.
func (s *SQLite) SpendSince(ctx context.Context, since time.Time) (map[string]int64, error) {
rows, err := s.db.QueryContext(ctx,
`SELECT key_id, SUM(cost_micros) FROM usage WHERE ts >= ? GROUP BY key_id`, millis(since))
if err != nil {
return nil, err
}
defer rows.Close()
out := map[string]int64{}
for rows.Next() {
var id string
var total sql.NullInt64
if err := rows.Scan(&id, &total); err != nil {
return nil, err
}
out[id] = total.Int64
}
return out, rows.Err()
}
// --- helpers ---
func (s *SQLite) inTx(ctx context.Context, fn func(*sql.Tx) error) error {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer func() { _ = tx.Rollback() }()
if err := fn(tx); err != nil {
return err
}
return tx.Commit()
}
// scanner is what both *sql.Row and *sql.Rows satisfy.
type scanner interface{ Scan(...any) error }
func scanKey(sc scanner) (Key, error) {
var k Key
var band, profile sql.NullString
var expires, revoked, lastUsed sql.NullInt64
var created int64
if err := sc.Scan(&k.ID, &k.Hash, &k.Prefix, &k.Name, &k.ProjectID,
&band, &profile, &created, &expires, &revoked, &lastUsed); err != nil {
return Key{}, err
}
k.Band, k.RoutingProfile = band.String, profile.String
k.CreatedAt = fromMillis(created)
k.ExpiresAt = fromNullMillis(expires)
k.RevokedAt = fromNullMillis(revoked)
k.LastUsedAt = fromNullMillis(lastUsed)
return k, nil
}
func queryAll[T any](ctx context.Context, db *sql.DB, query string, args []any,
scan func(*sql.Rows) (T, error),
) ([]T, error) {
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, err
}
defer rows.Close()
var out []T
for rows.Next() {
v, err := scan(rows)
if err != nil {
return nil, err
}
out = append(out, v)
}
return out, rows.Err()
}
func encodeModels(models []string) (any, error) {
if models == nil {
return nil, nil
}
b, err := json.Marshal(models)
if err != nil {
return nil, err
}
return string(b), nil
}
func decodeModels(v sql.NullString) ([]string, error) {
if !v.Valid || strings.TrimSpace(v.String) == "" {
return nil, nil
}
var out []string
if err := json.Unmarshal([]byte(v.String), &out); err != nil {
return nil, fmt.Errorf("allowed_models is not a JSON array: %w", err)
}
if out == nil {
out = []string{}
}
return out, nil
}
func minPtr(a, b *int) *int {
switch {
case b == nil:
return a
case a == nil:
return b
case *b < *a:
return b
default:
return a
}
}
func nullString(s string) any {
if s == "" {
return nil
}
return s
}
func nullInt(v *int) any {
if v == nil {
return nil
}
return *v
}
func nullInt64(v *int64) any {
if v == nil {
return nil
}
return *v
}
func millis(t time.Time) int64 { return t.UnixMilli() }
func nullMillis(t time.Time) any {
if t.IsZero() {
return nil
}
return t.UnixMilli()
}
func fromMillis(v int64) time.Time { return time.UnixMilli(v).UTC() }
func fromNullMillis(v sql.NullInt64) time.Time {
if !v.Valid {
return time.Time{}
}
return time.UnixMilli(v.Int64).UTC()
}
func nowMillis() int64 { return time.Now().UnixMilli() }
// SQLite satisfies Store.
var _ Store = (*SQLite)(nil)
+198
View File
@@ -0,0 +1,198 @@
// Package store persists what the cluster cannot reconstruct: who may call the
// gateway, what they are allowed, and what they have used (plan §7.9).
//
// Everything else llmgw knows is derived from the fleet and is deliberately not
// stored — a database that describes the cluster can be wrong, and the cluster
// cannot (§6.5). This package holds only the facts that have no other source.
//
// The interface exists so the engine can change. SQLite is right for a single
// instance: it is embedded, so the store shares the gateway's fate instead of
// adding a second thing that can be down, and an inference gateway that stops
// serving because its accounting database is unreachable is worse than one that
// buffers some accounting. That argument stops holding the moment there are two
// gateway instances, because budgets and rate limits then have to be counted
// somewhere shared; Postgres arrives with that, not before (plan §7.9.6).
package store
import (
"context"
"errors"
"time"
)
// ErrNotFound is returned when a lookup finds nothing.
var ErrNotFound = errors.New("not found")
// Scope is a level of the tenancy tree. Limits hang off any of them.
type Scope string
// The tenancy levels, from widest to narrowest.
const (
ScopeOrg Scope = "org"
ScopeTeam Scope = "team"
ScopeProject Scope = "project"
ScopeKey Scope = "key"
)
// Org, Team and Project are the tenancy tree.
type Org struct {
ID, Name string
}
// Team belongs to an organization.
type Team struct {
ID, OrgID, Name string
}
// Project belongs to a team and owns keys.
type Project struct {
ID, TeamID, Name string
}
// Key is an API key as stored: never the key itself.
type Key struct {
ID string
// Hash is sha256 of the key, and the only copy of anything derived from it.
Hash []byte
// Prefix is shown in listings so a key can be identified without revealing
// it. It is never used for authentication.
Prefix string
Name string
ProjectID string
// Band and RoutingProfile are empty when the key inherits them.
Band string
RoutingProfile string
CreatedAt time.Time
// ExpiresAt and RevokedAt are zero when unset. Revocation is a soft delete:
// usage rows reference the key, and an audit trail that loses its subject
// is not an audit trail.
ExpiresAt time.Time
RevokedAt time.Time
LastUsedAt time.Time
}
// Usable reports whether a key may authenticate at this instant.
func (k Key) Usable(now time.Time) bool {
if !k.RevokedAt.IsZero() {
return false
}
return k.ExpiresAt.IsZero() || now.Before(k.ExpiresAt)
}
// Limits are the caps at one scope. A nil field means "no limit here", which is
// different from zero: zero forbids everything.
type Limits struct {
Scope Scope
ScopeID string
RPM *int
TPM *int
// Budget is spend per period, in micro-units — integers rather than floats
// because a budget is accumulated by repeated addition, and repeated
// addition of floats drifts. What a unit means is the operator's choice;
// self-hosted models have no per-token price (plan §7.9.4).
Budget *int64
BudgetPeriod Period
// AllowedModels nil means every model.
AllowedModels []string
}
// Period is how often a budget resets.
type Period string
// The supported budget periods.
const (
PeriodNone Period = ""
PeriodDay Period = "day"
PeriodMonth Period = "month"
)
// Effective is a key's resolved limits, one value per field.
//
// Resolution takes the most restrictive value along key → project → team → org.
// A team cannot raise a limit its organization set, which is the only
// composition rule under which a budget means anything.
type Effective struct {
KeyID string
ProjectID string
TeamID string
OrgID string
Band string
RoutingProfile string
RPM *int
TPM *int
Budget *int64
BudgetPeriod Period
// AllowedModels is the intersection along the chain. nil means every model.
AllowedModels []string
}
// Allows reports whether a model may be served to this key.
func (e Effective) Allows(model string) bool {
if e.AllowedModels == nil {
return true
}
for _, m := range e.AllowedModels {
if m == model {
return true
}
}
return false
}
// Usage is one request's accounting record.
type Usage struct {
At time.Time
KeyID string
// ProjectID is denormalized so reporting does not join on every row, and so
// a row stays meaningful if the key is later revoked.
ProjectID string
Model string
// ServedModel differs from Model when a fallback served the request.
ServedModel string
PromptTokens int64
CompletionTokens int64
// CostMicros is what this request spent against the budget.
CostMicros int64
Status int
LatencyMs int64
}
// Store is the persistent half of tenancy and accounting.
//
// Nothing here is called on the request path. Authentication reads a cache that
// LookupKey fills on a miss, and usage is written in batches (plan §7.9.3).
type Store interface {
// Tenancy. Put is an upsert so applying a configuration twice is a no-op.
PutOrg(ctx context.Context, o Org) error
PutTeam(ctx context.Context, t Team) error
PutProject(ctx context.Context, p Project) error
Orgs(ctx context.Context) ([]Org, error)
Teams(ctx context.Context, orgID string) ([]Team, error)
Projects(ctx context.Context, teamID string) ([]Project, error)
// Keys.
CreateKey(ctx context.Context, k Key) error
KeyByHash(ctx context.Context, hash []byte) (Key, error)
Keys(ctx context.Context, projectID string) ([]Key, error)
RevokeKey(ctx context.Context, id string, at time.Time) error
TouchKeys(ctx context.Context, seen map[string]time.Time) error
// Limits.
PutLimits(ctx context.Context, l Limits) error
LimitsFor(ctx context.Context, scope Scope, id string) (Limits, error)
// Resolve walks a key's chain and returns the most restrictive of each.
Resolve(ctx context.Context, keyID string) (Effective, error)
// Accounting.
WriteUsage(ctx context.Context, records []Usage) error
// SpendSince totals cost per key since a moment, so an in-memory budget
// counter can be rebuilt after a restart. A restart must not reset a
// budget, which it would if the counter lived only in memory.
SpendSince(ctx context.Context, since time.Time) (map[string]int64, error)
Close() error
}
+405
View File
@@ -0,0 +1,405 @@
package store
import (
"context"
"errors"
"path/filepath"
"slices"
"strings"
"testing"
"time"
)
func open(t *testing.T) *SQLite {
t.Helper()
s, err := OpenSQLite(context.Background(), filepath.Join(t.TempDir(), "llmgw.db"))
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = s.Close() })
return s
}
// tree builds org → team → project and returns the project id.
func tree(t *testing.T, s *SQLite) string {
t.Helper()
ctx := context.Background()
if err := s.PutOrg(ctx, Org{ID: "org1", Name: "Acme"}); err != nil {
t.Fatal(err)
}
if err := s.PutTeam(ctx, Team{ID: "team1", OrgID: "org1", Name: "Research"}); err != nil {
t.Fatal(err)
}
if err := s.PutProject(ctx, Project{ID: "proj1", TeamID: "team1", Name: "Chat"}); err != nil {
t.Fatal(err)
}
return "proj1"
}
func mustKey(t *testing.T, s *SQLite, id, project string) (string, Key) {
t.Helper()
secret, k, err := NewKey(id, id, project)
if err != nil {
t.Fatal(err)
}
k.CreatedAt = time.Now()
if err := s.CreateKey(context.Background(), k); err != nil {
t.Fatal(err)
}
return secret, k
}
func intp(v int) *int { return &v }
func i64p(v int64) *int64 { return &v }
func TestMigrationsAreIdempotent(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "llmgw.db")
s, err := OpenSQLite(context.Background(), path)
if err != nil {
t.Fatal(err)
}
tree(t, s)
_ = s.Close()
// Reopening must not try to create what is already there, and must not lose
// what was written.
s2, err := OpenSQLite(context.Background(), path)
if err != nil {
t.Fatalf("reopening ran the migrations again: %v", err)
}
defer s2.Close()
orgs, err := s2.Orgs(context.Background())
if err != nil || len(orgs) != 1 {
t.Fatalf("orgs after reopen: %v %v", orgs, err)
}
}
func TestTheKeyItselfIsNeverStored(t *testing.T) {
// The whole point of hashing: a database leak hands over no working
// credential.
s := open(t)
project := tree(t, s)
secret, _ := mustKey(t, s, "key1", project)
rows, err := s.db.Query(`SELECT id, key_hash, key_prefix, name FROM api_keys`)
if err != nil {
t.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var id, prefix, name string
var hash []byte
if err := rows.Scan(&id, &hash, &prefix, &name); err != nil {
t.Fatal(err)
}
for _, field := range []string{id, prefix, name, string(hash)} {
if strings.Contains(field, strings.TrimPrefix(secret, KeyPrefix)) {
t.Fatal("the key itself reached the database")
}
}
if len(hash) != 32 {
t.Errorf("hash is %d bytes", len(hash))
}
}
}
func TestAuthenticationFindsAKeyByWhatWasPresented(t *testing.T) {
s := open(t)
project := tree(t, s)
secret, created := mustKey(t, s, "key1", project)
got, err := s.KeyByHash(context.Background(), HashKey(secret))
if err != nil {
t.Fatal(err)
}
if got.ID != created.ID || got.ProjectID != project {
t.Errorf("found %+v", got)
}
if !got.Usable(time.Now()) {
t.Error("a fresh key is not usable")
}
if _, err := s.KeyByHash(context.Background(), HashKey(secret+"x")); !errors.Is(err, ErrNotFound) {
t.Errorf("a wrong key was accepted: %v", err)
}
}
func TestRevocationKeepsTheAuditTrail(t *testing.T) {
// A revoked key must stop working and must still exist: usage rows
// reference it, and an audit trail that loses its subject is not one.
s := open(t)
ctx := context.Background()
project := tree(t, s)
secret, k := mustKey(t, s, "key1", project)
if err := s.RevokeKey(ctx, k.ID, time.Now()); err != nil {
t.Fatal(err)
}
got, err := s.KeyByHash(ctx, HashKey(secret))
if err != nil {
t.Fatalf("a revoked key vanished from the database: %v", err)
}
if got.Usable(time.Now()) {
t.Error("a revoked key still authenticates")
}
if err := s.RevokeKey(ctx, "nope", time.Now()); !errors.Is(err, ErrNotFound) {
t.Errorf("revoking a key that does not exist: %v", err)
}
}
func TestExpiryIsCheckedAgainstTheClockNotTheDatabase(t *testing.T) {
s := open(t)
project := tree(t, s)
secret, k := mustKey(t, s, "key1", project)
k.ExpiresAt = time.Now().Add(-time.Hour)
if err := s.CreateKey(context.Background(), Key{
ID: "key2", Hash: HashKey("other"), Prefix: "x", Name: "n",
ProjectID: project, CreatedAt: time.Now(), ExpiresAt: k.ExpiresAt,
}); err != nil {
t.Fatal(err)
}
expired, err := s.KeyByHash(context.Background(), HashKey("other"))
if err != nil {
t.Fatal(err)
}
if expired.Usable(time.Now()) {
t.Error("an expired key is usable")
}
fresh, _ := s.KeyByHash(context.Background(), HashKey(secret))
if !fresh.Usable(time.Now()) {
t.Error("an unexpired key is not usable")
}
}
func TestOrphansAreRefused(t *testing.T) {
// SQLite leaves foreign keys off by default, which would let the tenancy
// tree grow keys belonging to no project.
s := open(t)
_, k, _ := NewKey("key1", "n", "no-such-project")
k.CreatedAt = time.Now()
if err := s.CreateKey(context.Background(), k); err == nil {
t.Fatal("a key was created under a project that does not exist")
}
}
func TestLimitsResolveToTheMostRestrictive(t *testing.T) {
s := open(t)
ctx := context.Background()
project := tree(t, s)
_, k := mustKey(t, s, "key1", project)
// The organization is generous, the team less so, the key least. Whichever
// is tightest has to win at every field independently.
must(t, s.PutLimits(ctx, Limits{Scope: ScopeOrg, ScopeID: "org1", RPM: intp(1000), TPM: intp(100000)}))
must(t, s.PutLimits(ctx, Limits{Scope: ScopeTeam, ScopeID: "team1", RPM: intp(100)}))
must(t, s.PutLimits(ctx, Limits{Scope: ScopeKey, ScopeID: k.ID, TPM: intp(500)}))
e, err := s.Resolve(ctx, k.ID)
if err != nil {
t.Fatal(err)
}
if e.RPM == nil || *e.RPM != 100 {
t.Errorf("rpm=%v, the team's limit should win", e.RPM)
}
if e.TPM == nil || *e.TPM != 500 {
t.Errorf("tpm=%v, the key's limit should win", e.TPM)
}
if e.OrgID != "org1" || e.TeamID != "team1" || e.ProjectID != project {
t.Errorf("chain resolved to %+v", e)
}
}
func TestANarrowerScopeCannotRaiseALimit(t *testing.T) {
// The only composition rule under which a budget means anything.
s := open(t)
ctx := context.Background()
project := tree(t, s)
_, k := mustKey(t, s, "key1", project)
must(t, s.PutLimits(ctx, Limits{Scope: ScopeOrg, ScopeID: "org1", RPM: intp(10),
Budget: i64p(1_000_000), BudgetPeriod: PeriodMonth}))
must(t, s.PutLimits(ctx, Limits{Scope: ScopeKey, ScopeID: k.ID, RPM: intp(9999),
Budget: i64p(999_000_000), BudgetPeriod: PeriodDay}))
e, err := s.Resolve(ctx, k.ID)
if err != nil {
t.Fatal(err)
}
if *e.RPM != 10 {
t.Errorf("a key raised its organization's rate limit to %d", *e.RPM)
}
if *e.Budget != 1_000_000 || e.BudgetPeriod != PeriodMonth {
t.Errorf("a key raised its organization's budget to %d/%s", *e.Budget, e.BudgetPeriod)
}
}
func TestAllowlistsIntersectRatherThanReplace(t *testing.T) {
s := open(t)
ctx := context.Background()
project := tree(t, s)
_, k := mustKey(t, s, "key1", project)
must(t, s.PutLimits(ctx, Limits{Scope: ScopeTeam, ScopeID: "team1",
AllowedModels: []string{"qwen", "llama"}}))
must(t, s.PutLimits(ctx, Limits{Scope: ScopeKey, ScopeID: k.ID,
AllowedModels: []string{"llama", "gpt-4"}}))
e, err := s.Resolve(ctx, k.ID)
if err != nil {
t.Fatal(err)
}
if !slices.Equal(e.AllowedModels, []string{"llama"}) {
t.Fatalf("allowed=%v; a key must not add a model its team withheld", e.AllowedModels)
}
if e.Allows("gpt-4") {
t.Error("a model the team never allowed is served")
}
if !e.Allows("llama") {
t.Error("the intersection is not served")
}
}
func TestNoAllowlistMeansEveryModel(t *testing.T) {
s := open(t)
project := tree(t, s)
_, k := mustKey(t, s, "key1", project)
e, err := s.Resolve(context.Background(), k.ID)
if err != nil {
t.Fatal(err)
}
if e.AllowedModels != nil {
t.Errorf("allowed=%v", e.AllowedModels)
}
if !e.Allows("anything") {
t.Error("an unrestricted key was refused a model")
}
}
func TestAnEmptyAllowlistForbidsEverything(t *testing.T) {
// Empty is not the same as absent, and the store must not conflate them:
// one is "no restriction", the other is "nothing".
s := open(t)
ctx := context.Background()
project := tree(t, s)
_, k := mustKey(t, s, "key1", project)
must(t, s.PutLimits(ctx, Limits{Scope: ScopeKey, ScopeID: k.ID, AllowedModels: []string{}}))
e, err := s.Resolve(ctx, k.ID)
if err != nil {
t.Fatal(err)
}
if e.AllowedModels == nil {
t.Fatal("an empty allowlist was read back as no allowlist")
}
if e.Allows("qwen") {
t.Error("an empty allowlist served a model")
}
}
func TestBudgetSurvivesARestart(t *testing.T) {
// A counter that lives only in memory resets on restart, which would make a
// budget a suggestion.
dir := t.TempDir()
path := filepath.Join(dir, "llmgw.db")
ctx := context.Background()
s, err := OpenSQLite(ctx, path)
if err != nil {
t.Fatal(err)
}
project := tree(t, s)
_, k := mustKey(t, s, "key1", project)
now := time.Now()
must(t, s.WriteUsage(ctx, []Usage{
{At: now, KeyID: k.ID, ProjectID: project, Model: "qwen", ServedModel: "qwen",
PromptTokens: 100, CompletionTokens: 50, CostMicros: 1500, Status: 200},
{At: now, KeyID: k.ID, ProjectID: project, Model: "qwen", ServedModel: "qwen",
PromptTokens: 200, CompletionTokens: 80, CostMicros: 2800, Status: 200},
// Last period: must not count towards this one.
{At: now.Add(-48 * time.Hour), KeyID: k.ID, ProjectID: project, Model: "qwen",
ServedModel: "qwen", CostMicros: 999999, Status: 200},
}))
_ = s.Close()
s2, err := OpenSQLite(ctx, path)
if err != nil {
t.Fatal(err)
}
defer s2.Close()
spend, err := s2.SpendSince(ctx, now.Add(-time.Hour))
if err != nil {
t.Fatal(err)
}
if spend[k.ID] != 4300 {
t.Errorf("spend after restart = %d, want 4300", spend[k.ID])
}
}
func TestLastUsedIsOnlyEverMovedForward(t *testing.T) {
// The batch writer sees keys out of order; an older timestamp must not
// rewind the newer one.
s := open(t)
ctx := context.Background()
project := tree(t, s)
_, k := mustKey(t, s, "key1", project)
newer := time.Now().Truncate(time.Millisecond)
older := newer.Add(-time.Hour)
must(t, s.TouchKeys(ctx, map[string]time.Time{k.ID: newer}))
must(t, s.TouchKeys(ctx, map[string]time.Time{k.ID: older}))
got, err := s.KeyByHash(ctx, k.Hash)
if err != nil {
t.Fatal(err)
}
if !got.LastUsedAt.Equal(newer.UTC()) {
t.Errorf("last used = %v, want %v", got.LastUsedAt, newer.UTC())
}
}
func TestUpsertsAreIdempotent(t *testing.T) {
// Applying a configuration twice must not double the tree or fail.
s := open(t)
ctx := context.Background()
tree(t, s)
tree(t, s)
orgs, _ := s.Orgs(ctx)
teams, _ := s.Teams(ctx, "org1")
projects, _ := s.Projects(ctx, "team1")
if len(orgs) != 1 || len(teams) != 1 || len(projects) != 1 {
t.Errorf("tree duplicated: %d orgs, %d teams, %d projects", len(orgs), len(teams), len(projects))
}
must(t, s.PutOrg(ctx, Org{ID: "org1", Name: "Renamed"}))
orgs, _ = s.Orgs(ctx)
if orgs[0].Name != "Renamed" {
t.Errorf("upsert did not update: %q", orgs[0].Name)
}
}
func TestResolveRejectsAnUnknownKey(t *testing.T) {
s := open(t)
tree(t, s)
if _, err := s.Resolve(context.Background(), "nope"); !errors.Is(err, ErrNotFound) {
t.Errorf("resolving an unknown key: %v", err)
}
}
func must(t *testing.T, err error) {
t.Helper()
if err != nil {
t.Fatal(err)
}
}
+40 -8
View File
@@ -549,13 +549,39 @@ mechanism.
#### Store choice
SQLite in WAL mode, via `modernc.org/sqlite` so the server stays cgo-free.
Embedded means no second service to operate, and WAL gives concurrent readers
alongside one writer, which matches the access pattern: many reads for auth,
one batched writer for usage.
It stops being enough at the point where several router instances must write at
once. The store therefore sits behind an interface from the start, and the swap
to Postgres comes with multi-instance deployment rather than before it.
The deciding argument is not throughput. One row per request at 1000 req/s is 86
million rows a day, which sounds like a reason to reach for Postgres, but the
write path is batched anyway (§7.9.3) and a transaction of 500 rows every 200ms
is nothing for either engine. Batching is required for a different reason, and
once it exists the volume argument favours neither.
What decides it is the failure mode. Postgres is a second service, and the
question it raises is whether the gateway still serves traffic when that service
is unreachable. An inference gateway that stops serving because its accounting
database is down is worse than one that buffers some accounting. SQLite is
embedded, so the store shares the gateway's fate rather than adding an
independent way to be down — and the single-binary goal falls out of the same
choice.
It stops being enough at the point where there is more than one gateway
instance, because budgets and rate-limit counters then have to be counted
somewhere shared; per-instance counters mean N instances allow N times the
configured limit (§7.9.6). Postgres arrives with that, not before.
Three rules keep the swap cheap, and they are worth following from the first
commit:
1. **Nothing on the request path touches the database.** Keys are cached in
memory and read through on a miss, usage goes to a batch writer, budgets are
in-memory counters reconciled against the store. This matters more than the
engine choice and is what makes either engine survivable.
2. **Portable SQL.** No `AUTOINCREMENT` and no rowid tricks, `ON CONFLICT … DO
UPDATE` rather than `INSERT OR REPLACE`, timestamps as integer unix millis,
money and tokens as integers.
3. **Migrations per dialect, sharing version numbers.** A single migration set
for both engines would be a promise that cannot be kept — SQLite's
`ALTER TABLE` barely exists. Same numbering, same meaning, different files.
---
@@ -959,7 +985,7 @@ CREATE TABLE limits (
scope_id TEXT NOT NULL,
rpm INTEGER, -- NULL means no limit at this level
tpm INTEGER,
budget_units REAL, -- per period; NULL means no budget
budget_micros INTEGER, -- per period, in micro-units; NULL means no budget
budget_period TEXT, -- 'day' | 'month'
allowed_models TEXT, -- JSON array; NULL means all
PRIMARY KEY (scope_type, scope_id)
@@ -968,7 +994,13 @@ CREATE TABLE limits (
A request's effective limit is the **most restrictive** value along
key → project → team → org. A team cannot raise a limit its organization set,
which is the only composition rule that makes a budget mean anything.
which is the only composition rule that makes a budget mean anything. Model
allowlists compose the same way, by intersection: a key cannot add a model its
team withheld. An **empty** allowlist is not an absent one — absent means every
model, empty means none.
Budgets are counted in integer micro-units rather than a float. A budget is
accumulated by repeated addition, and repeated addition of floats drifts.
Resolving four levels per request would be four lookups on the hot path, so the
resolved view is computed once and cached per key hash, invalidated whenever