Run the store on PostgreSQL, chosen by what the address looks like

A path is still SQLite and a postgres:// URL is PostgreSQL, so every existing
configuration keeps working untouched and moving is one line changed rather
than a new setting to find. An address of the form @/path/to/file reads the
real one out of that file, which is how a DSN carrying a password stays out of
a configuration that gets copied, diffed and pasted into issues.

The driver is pgx, which is pure Go. That is not a preference: the whole binary
is built with CGO_ENABLED=0 so it runs on a node whose libc is older than the
builder's, and a cgo driver would take that away from every deployment,
including the ones still on SQLite.

Tested against a real PostgreSQL, which found two things nothing else could.
The schema_migrations table is created in Go rather than in a migration file,
so it escaped the port and still said INTEGER — 64 bits in SQLite, 32 in
PostgreSQL, and a millisecond timestamp does not fit, so the very first
migration failed to record that it had run. And the memory migration uses an
FTS5 virtual table with three triggers keeping it in step, which has no
counterpart: PostgreSQL derives a tsvector from the row, so there is nothing
to keep in step and no trigger to forget.

That leaves exactly one query that is not the same sentence in both dialects —
the note search, MATCH against @@ — and it is written out twice rather than
hidden behind a shared spelling that would mean something subtly different in
each. Everything else is one set of queries.

The text search uses the 'simple' configuration rather than English. These are
an agent's own notes, in whatever language it chose, and stemming English
rules over Turkish would quietly stop finding things.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5 committed 2026-09-25 16:18:10 +02:00
1 parent ccc9aa8b76
commit a76ebdb79a
18 files changed
+750 -12

No files matched your search

@@ -0,0 +1,81 @@
-- Ported from the SQLite migration of the same number. Same version, same
-- meaning, different spelling: SQLite's storage classes and PostgreSQL's types
-- do not line up, and one shared file would be a promise neither could keep.
--
-- Times are milliseconds since the epoch in BIGINT, exactly as in SQLite,
-- rather than TIMESTAMPTZ. The Go code reads and writes int64 milliseconds
-- everywhere; converting here would mean two time representations in one
-- codebase, and the conversion would be the thing that is wrong at 2am.
-- The tenancy tree. Three levels, each owning the next.
CREATE TABLE organizations (
id TEXT PRIMARY KEY,
name TEXT NOT NULL
);
CREATE TABLE teams (
id TEXT PRIMARY KEY,
org_id TEXT NOT NULL REFERENCES organizations(id),
name TEXT NOT NULL
);
CREATE INDEX teams_by_org ON teams(org_id);
CREATE TABLE projects (
id TEXT PRIMARY KEY,
team_id TEXT NOT NULL REFERENCES teams(id),
name TEXT NOT NULL
);
CREATE INDEX projects_by_team ON projects(team_id);
-- Only sha256(key) is stored, so a database leak hands over no working
-- credential. SHA-256 rather than a slow password hash on purpose: the input is
-- 256 random bits with no dictionary to attack, so the cost of argon2 would buy
-- nothing and charge tens of milliseconds on every request (plan §7.9.1).
CREATE TABLE api_keys (
id TEXT PRIMARY KEY,
key_hash BYTEA NOT NULL UNIQUE,
key_prefix TEXT NOT NULL,
name TEXT NOT NULL,
project_id TEXT NOT NULL REFERENCES projects(id),
band TEXT,
routing_profile TEXT,
created_at BIGINT NOT NULL,
expires_at BIGINT,
-- Revocation is a soft delete. Usage rows reference the key, and an audit
-- trail that loses its subject is not an audit trail.
revoked_at BIGINT,
last_used_at BIGINT
);
CREATE INDEX api_keys_by_project ON api_keys(project_id);
-- Limits hang off any scope rather than being duplicated onto each one. A NULL
-- column means "no limit at this level", which is not the same as 0.
CREATE TABLE limits (
scope_type TEXT NOT NULL,
scope_id TEXT NOT NULL,
rpm BIGINT,
tpm BIGINT,
-- Micro-units, not a float: a budget is accumulated by repeated addition, and
-- repeated addition of floats drifts.
budget_micros BIGINT,
budget_period TEXT,
allowed_models TEXT,
PRIMARY KEY (scope_type, scope_id)
);
-- One row per request, written in batches off the request path.
CREATE TABLE usage (
ts BIGINT NOT NULL,
key_id TEXT NOT NULL,
project_id TEXT NOT NULL,
model TEXT NOT NULL,
served_model TEXT NOT NULL,
prompt_tokens BIGINT NOT NULL,
completion_tokens BIGINT NOT NULL,
cost_micros BIGINT NOT NULL,
status BIGINT NOT NULL,
latency_ms BIGINT NOT NULL
);
-- Budget restore reads by key within a period, which is this index exactly.
CREATE INDEX usage_by_key_ts ON usage(key_id, ts);
CREATE INDEX usage_by_ts ON usage(ts);
@@ -0,0 +1,29 @@
-- Ported from the SQLite migration of the same number. Same version, same
-- meaning, different spelling: SQLite's storage classes and PostgreSQL's types
-- do not line up, and one shared file would be a promise neither could keep.
--
-- Times are milliseconds since the epoch in BIGINT, exactly as in SQLite,
-- rather than TIMESTAMPTZ. The Go code reads and writes int64 milliseconds
-- everywhere; converting here would mean two time representations in one
-- codebase, and the conversion would be the thing that is wrong at 2am.
-- A key has to reach the person it was issued for, and a key pasted into a chat
-- window stays in that chat window forever. A claim is a one-time link instead:
-- the operator sends the link, the person opens it once, and the link dies.
--
-- The key is not stored here in the clear. It is sealed with a key derived from
-- the claim token, and the token is never written down — only its hash is. So a
-- database leak yields two useless hashes and a blob nothing in the database can
-- open, which keeps the property the api_keys table has.
CREATE TABLE key_claims (
token_hash BYTEA PRIMARY KEY,
key_id TEXT NOT NULL REFERENCES api_keys(id),
-- The sealed key. Emptied the moment it is claimed, so a claimed link leaves
-- nothing behind even before the row is purged.
sealed BYTEA NOT NULL,
created_at BIGINT NOT NULL,
expires_at BIGINT NOT NULL,
claimed_at BIGINT
);
CREATE INDEX key_claims_by_key ON key_claims(key_id);
CREATE INDEX key_claims_by_expiry ON key_claims(expires_at);
@@ -0,0 +1,56 @@
-- Ported from the SQLite migration of the same number. Same version, same
-- meaning, different spelling: SQLite's storage classes and PostgreSQL's types
-- do not line up, and one shared file would be a promise neither could keep.
--
-- Times are milliseconds since the epoch in BIGINT, exactly as in SQLite,
-- rather than TIMESTAMPTZ. The Go code reads and writes int64 milliseconds
-- everywhere; converting here would mean two time representations in one
-- codebase, and the conversion would be the thing that is wrong at 2am.
-- Accounts, so a person can sign up, be approved, and then issue their own keys
-- without an operator running a command for each one.
CREATE TABLE users (
id TEXT PRIMARY KEY,
-- Lowercased at the boundary. Two accounts differing only in case would be
-- two accounts to the database and one to every human who looks at them.
email TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
-- argon2id, encoded with its own parameters so the cost can be raised later
-- without invalidating existing hashes. Slow hashing is right here and wrong
-- for API keys: this input is chosen by a human and has a dictionary.
password_hash TEXT NOT NULL,
-- 'pending' until an operator approves, then 'active'. 'suspended' keeps the
-- account and stops it working.
state TEXT NOT NULL,
-- Assigned at approval: which project this person's keys belong to, and
-- therefore which limits they inherit.
project_id TEXT REFERENCES projects(id),
created_at BIGINT NOT NULL,
approved_at BIGINT,
approved_by TEXT,
last_login_at BIGINT
);
CREATE INDEX users_by_state ON users(state);
-- Server-side sessions rather than signed cookies, so signing out and
-- suspending an account take effect immediately instead of whenever a token
-- happens to expire.
CREATE TABLE sessions (
token_hash BYTEA PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users(id),
-- Per session, submitted with every form. A cookie alone authenticates the
-- browser, not the page that asked, which is what makes a cross-site POST
-- work.
csrf_hash BYTEA NOT NULL,
created_at BIGINT NOT NULL,
expires_at BIGINT NOT NULL,
last_seen BIGINT NOT NULL
);
CREATE INDEX sessions_by_user ON sessions(user_id);
CREATE INDEX sessions_by_expiry ON sessions(expires_at);
-- A key issued through the profile page belongs to the person who made it, so
-- they can see and revoke their own and nobody else's. NULL means a key an
-- operator issued directly.
ALTER TABLE api_keys ADD COLUMN user_id TEXT REFERENCES users(id);
CREATE INDEX api_keys_by_user ON api_keys(user_id);
@@ -0,0 +1,20 @@
-- Ported from the SQLite migration of the same number. Same version, same
-- meaning, different spelling: SQLite's storage classes and PostgreSQL's types
-- do not line up, and one shared file would be a promise neither could keep.
--
-- Times are milliseconds since the epoch in BIGINT, exactly as in SQLite,
-- rather than TIMESTAMPTZ. The Go code reads and writes int64 milliseconds
-- everywhere; converting here would mean two time representations in one
-- codebase, and the conversion would be the thing that is wrong at 2am.
-- Who may see the cluster.
--
-- Until now the answer was "whoever has a shell on the control plane", because
-- the dashboard lived on a loopback-bound port. Serving it over the web needs
-- an actual answer, and this is the smallest one that is not a lie: a flag on
-- an account that already has a password, a session and a CSRF token.
--
-- Deliberately not a role system. One bit is enough for "can see the cluster"
-- and impossible to get subtly wrong; roles arrive when there is a second
-- distinction worth drawing.
ALTER TABLE users ADD COLUMN is_admin BIGINT NOT NULL DEFAULT 0;
@@ -0,0 +1,34 @@
-- Ported from the SQLite migration of the same number. Same version, same
-- meaning, different spelling: SQLite's storage classes and PostgreSQL's types
-- do not line up, and one shared file would be a promise neither could keep.
--
-- Times are milliseconds since the epoch in BIGINT, exactly as in SQLite,
-- rather than TIMESTAMPTZ. The Go code reads and writes int64 milliseconds
-- everywhere; converting here would mean two time representations in one
-- codebase, and the conversion would be the thing that is wrong at 2am.
-- Models added while the cluster is running.
--
-- The catalog came from a YAML file read once at startup, so adding a model
-- meant editing a file on the control plane and restarting it: a settling
-- window, adoption churn, and on one occasion two minutes of outage. That is a
-- reasonable way to manage a fixed production set and a bad one for a platform
-- whose promise is that people can try things.
--
-- These live alongside the ones in configuration rather than replacing them.
-- Config-as-code stays declarative — a model in YAML is in the catalog on every
-- start, whatever the database says — and this table holds the ones added
-- afterwards. A model added here may not take a name or alias that
-- configuration already uses, which the catalog's own constructor enforces for
-- free, because it refuses a duplicate whatever the source.
--
-- The definition is stored as JSON rather than as columns. A profile is a
-- nested, evolving shape, and every field added to it would otherwise be a
-- migration; the catalog already knows how to validate one, and a row that
-- fails validation is reported and skipped rather than blocking startup.
CREATE TABLE models (
name TEXT PRIMARY KEY,
definition TEXT NOT NULL,
created_at TIMESTAMP NOT NULL,
created_by TEXT NOT NULL DEFAULT ''
);
@@ -0,0 +1,57 @@
-- Ported from the SQLite migration of the same number. Same version, same
-- meaning, different spelling: SQLite's storage classes and PostgreSQL's types
-- do not line up, and one shared file would be a promise neither could keep.
--
-- Times are milliseconds since the epoch in BIGINT, exactly as in SQLite,
-- rather than TIMESTAMPTZ. The Go code reads and writes int64 milliseconds
-- everywhere; converting here would mean two time representations in one
-- codebase, and the conversion would be the thing that is wrong at 2am.
-- Server-side conversations, so a thread can continue on a different model
-- than it started on.
--
-- Off unless a project turns it on. Until this migration ran, llmgw stored no
-- message content anywhere — usage rows carry token counts and costs, never
-- text — and that stays true for every project that does not ask. An on-prem
-- gateway keeping nothing anyone typed is much easier to put in front of a
-- team than one keeping everything, so storing is a choice a project makes
-- rather than a default it discovers.
-- retention_hours is the switch and the limit at once. NULL means this project
-- stores nothing, which is every project that existed before this and every
-- one created since without asking.
--
-- One column rather than a boolean and a duration: a thread store with no
-- expiry is a transcript of everything a team ever typed, kept forever by
-- omission. Turning it on therefore requires saying for how long.
ALTER TABLE projects ADD COLUMN thread_retention_hours BIGINT;
CREATE TABLE threads (
id TEXT PRIMARY KEY,
-- Scoped to a project, and checked on every read. Thread ids are opaque, but
-- an unguessable id is a secret and secrets leak into logs, shell history
-- and bug reports; the scope is what makes a leaked id useless to someone
-- outside the project.
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
created_at BIGINT NOT NULL,
-- Touched on every append, so retention measures idleness rather than age:
-- expiring a conversation someone is still having would be a strange way to
-- keep a promise about storage.
updated_at BIGINT NOT NULL
);
CREATE INDEX threads_by_project ON threads(project_id);
CREATE INDEX threads_by_updated ON threads(updated_at);
CREATE TABLE thread_messages (
thread_id TEXT NOT NULL REFERENCES threads(id) ON DELETE CASCADE,
-- Position rather than a timestamp, because order is the point and two
-- messages can share a millisecond.
seq BIGINT NOT NULL,
role TEXT NOT NULL,
content TEXT NOT NULL,
-- Which model produced an assistant turn. A thread that moved between models
-- otherwise reads as one voice, and when the answers disagree that is the
-- first thing anyone wants to know.
model TEXT,
PRIMARY KEY (thread_id, seq)
);
@@ -0,0 +1,41 @@
-- Ported from the SQLite migration of the same number, and the only one that
-- is not a mechanical translation.
--
-- SQLite gets full-text search from an FTS5 virtual table with an external
-- content table and three triggers keeping it in step. PostgreSQL derives a
-- tsvector from the row itself, so there is nothing to keep in step and no
-- trigger to forget: the column cannot disagree with the row it is computed
-- from.
--
-- The search query differs too, and that difference lives in the Go code
-- rather than being hidden here — a query that means "match these words" is
-- spelled MATCH in one and @@ in the other, and pretending otherwise would
-- cost more than admitting it.
ALTER TABLE projects ADD COLUMN memory_retention_hours BIGINT;
CREATE TABLE memory_notes (
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
-- Scoped to a project and checked in SQL on every read, like threads.
project_id TEXT NOT NULL REFERENCES projects(id) ON DELETE CASCADE,
-- A namespace inside the project, so two agents sharing a key do not read
-- each other's notes.
agent_id TEXT NOT NULL,
key TEXT NOT NULL,
text TEXT NOT NULL,
-- Normalised to ",a,b," so a tag matches whole, never as part of another.
tags TEXT NOT NULL DEFAULT '',
created_at BIGINT NOT NULL,
-- Touched on every write, so retention measures how long a note has gone
-- unrefreshed rather than how old it is.
updated_at BIGINT NOT NULL,
-- 'simple' rather than a language configuration: these are an agent's own
-- notes, in whatever language it chose, and stemming English rules over
-- Turkish text would quietly stop finding things.
search tsvector GENERATED ALWAYS AS (
to_tsvector('simple', coalesce(key, '') || ' ' || coalesce(text, '') || ' ' || coalesce(tags, ''))
) STORED,
UNIQUE (project_id, agent_id, key)
);
CREATE INDEX memory_notes_by_updated ON memory_notes(updated_at);
CREATE INDEX memory_notes_search ON memory_notes USING GIN (search);