bf2d171e8941bbff7cbbfba078a5ceb18c388f9e
34
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Ilker ManapandClaude Opus 5.5
|
16a1e8b4d8 |
Show a person their own use and what is left of their budgets
The account page (was "Your keys") shows the last thirty days of the person's own use - requests, tokens, spend and tokens per day - and, for each place they can spend, what they used there and a bar for every budget that applies: their own, a per-member one, or a project's, team's or organization's shared with others, named as such. Bars turn amber past four-fifths and red when spent. It sits under the keys now, above the password form. Spend is read from the usage table for the pool each limit counts against, from the start of its period. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5.5
|
9b663eb23c |
Name people and projects in usage reports, and total a usage legend
The cluster page's usage table showed session:usr-... and proj-... ids; it now shows the person's email, whether it came through a key or the chat page, and the project's path or "personal". Times read "3 hours ago" rather than "3h7m25s ago", everywhere. The usage page's legend gave each line's latest slice, which was 0 beside a table of hundreds; it gives the window's total, and the table's numbers are right-aligned like everywhere else. The PostgreSQL placeholder rewriting skips line comments. An apostrophe in one opened a string that swallowed every placeholder after it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5.5
|
e493b2f53d |
Give an organization's managers a page of their own
A member of an organization can be its manager (users join --org X --role manager, or the star on the people page). /org shows a manager their organization: usage over time by team, project or person, its teams and projects with limits and 30-day spend, and its members, each once with every place they belong. Read-only; administrators see every organization. Only what was spent in the organization's projects is shown — UsageBucketsIn filters by where a request was spent. Found on the way: - Usage by org, team and project, and the quotas page's spend, joined through api_keys and so dropped every chat-page request, which has no key row. They now read usage.project_id; keys no longer move between projects, so the two agree. - /org was not exempt from API-key auth and answered browsers with a JSON error. A router test now drives the session pages through the middleware; it fails without the fix. Membership queries, org-scoped usage and the reworked joins pass the PostgreSQL store tests on 16. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5.5
|
098886e0e4 |
Let a person belong to many places, or to none
A person was in exactly one project, chosen at approval, and could not use the gateway before that. People belong to several teams and to none, so: - memberships: a person is a member of any number of organizations, teams and projects, or of none; a member of a team or organization may use every project in it. users join/leave, and the people page adds and takes out from either side. Approval no longer needs a project. - personal keys: a key may have an owner and no project. /profile asks what a new key spends against; keys create --for does the same. auth.personal sets default limits for personal use or turns it off. - limits for people: --scope user holds a person across every key, project and the chat page; --per-member on an org, team or project gives each member their own share there. Resolution folds the tree, each per-member limit for the key's owner, and the owner's own, each in its own pool; the quotas page shows and sets both. - the chat page asks what to spend against and checks it per message. - usage records who spent it, so a person's budget survives a restart, personal use and chat included; restore no longer drops rows with no project. Leaving a project revokes the keys made there unless another membership still reaches it. A personal key keeps no threads or notes, which are per project, and is refused rather than dropped into the anonymous pool. Migration 0010 turns each person's project into a membership, adds usage.user_id and backfills it, and makes api_keys.project_id nullable. SQLite cannot drop NOT NULL, so the table is rebuilt; that needs foreign keys off, which only works outside a transaction, so migrations marked for it run on one connection with foreign_key_check before commit. Tested on a populated SQLite store with claims, and on a copy of the cluster's PostgreSQL store: 5 people became members, 17 keys and 3 claims intact, 145 of 577 usage rows and all 69 chat rows attributed. The PostgreSQL store tests, including new membership cases, pass on 16. users move is gone; join and leave replace it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5.5
|
3104f4b706 |
Manage people and the tenancy tree, from the web and the CLI
Nothing in the tree could be renamed, moved or removed, and an approved person stayed in the project they were first put in — from the web or the CLI. Now: - /admin/users is the tree, each project opening onto its people and keys, with approve, move, suspend, reinstate and remove for people, revoke for keys, and add, rename, move and remove at every level, plus a search over every account. Forms post and get the page back, so it works with no script; htmx 2.0.11 (0BSD, embedded, checked against npm's integrity hash) swaps only the changed part, keeping open projects open. - llmgw org tree/add/rename/move/remove, users move/delete, and keys list --all. - Projects, teams and organizations are named by id, path (Acme/platform/gateway) or unique name; users by id or email. With a single project --project can be left out, instead of defaulting to "default", which a tree built from the web does not have. Moving a person moves the keys they made; operator keys stay with the project. Only empty things are removed, and never a project that had a key, since usage history refers to it. Also: keys init no longer reparents an existing team or project; reinstating an account with no project is refused (a rejected signup used to come back active and fail at its first key); rejecting a signup removes it; keys limits --budget takes whole units like the quotas page. Checked in headless Chrome: a move from an open project updates the page in place, keeps the project open, and the search still filters. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5.5
|
cba65f5be7 |
Move a control plane to PostgreSQL with server init --postgres
Installs PostgreSQL with the machine's own package manager when it has none (initdb with password auth on the RHEL family, whose default is ident), creates the llmgw role and database with a generated password kept in /etc/llmgw/secrets/store-dsn, and checks the address connects. On a machine already on SQLite it stops the gateway, copies the store so accounts and keys survive, points auth.store at the new address and starts it again. A move that fails starts the gateway on SQLite again. server.yaml is edited as text so its comments survive, and kept beside it. A PostgreSQL init did not install is not resized: it may serve something else. It refuses to move one PostgreSQL store into another. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
07809490e3 |
Show usage over time, per organization, team, project and person
CI / test (push) Has been cancelled
Usage could be read as a total — per key, per user, per model, per group — and the cluster's own charts covered the fleet's hardware. What nobody could see was when a group used it. "Research doubled its traffic three weeks ago" is not a question a column of totals can be asked, and it is the question somebody asks before a quota conversation. /admin/usage draws requests, tokens and spend over a chosen window, grouped by any level of the tenancy tree or by person. Person is not a level of that tree — one person holds keys in several projects — so the grouping is its own type rather than store.Scope with a fourth value nothing else could use. Three things learned from the charts already here. Buckets are gap-filled per group, because a line drawn straight across an absence claims traffic that did not happen, and because the chart sums its tail by timestamp and can only do that if every series shares a clock. Series are ranked by total rather than by peak — a counter's question is "who used the most", and ranking by peak would colour the team that made one burst over the team that ran steadily all week. And the spend chart is replaced by a sentence when no model has a price, rather than drawn flat along the bottom. The bucket query groups by ordinal and was planned against the cluster's PostgreSQL before being believed, with a test alongside the others that run there: three reports have shipped working on SQLite and failing on it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
f999ee2fb2 |
Make an organization's budget bound the sum of its teams
CI / test (push) Successful in 1m40s
Resolution took the most restrictive value along the chain and enforced only that one. For a single caller's ceiling that is right: a team cannot raise what its organization set. For a pool it is not, because two levels are two pools. An organization given 1000 with four teams given 500 each spent 2000. Each team filled its own counter, the organization's value lost to the tighter one and was never looked at again. The number was written, displayed, reported against — and enforced by nothing. Every level that sets a limit is now checked against its own counter, and settled onto all of them, so an organization's pool fills from its teams' spending rather than only from requests that somehow named it. A team still stops before its organization when its own pool runs out first, which is what a sub-allocation is for. Effective carries the chain as well as the collapsed values. The collapsed ones are still what a header or a profile page should show — they are the ceiling a single caller faces — they are just not enough to enforce with. Rate limits charge as they check, deliberately, so that two concurrent requests cannot both see room. With several levels that means a refusal at one has already taken from the levels before it, so those are handed back. Without the refund a team sitting at its organization's ceiling would quietly eat its own rate limit with requests that were never served. Budgets are checked before any of that, and only read, so a refusal on a budget charges nothing at all. Status reports the fullest pool rather than the smallest number. A team inside an organization with the same budget is bounded by whichever is nearer its ceiling, and that is the organization as soon as a second team spends anything; a header showing the team's would disagree with the refusal after it. A cluster default joins the chain as a key-level entry. Left out, it would be reported as the ceiling and then never checked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
55262418c9 |
Put quotas somewhere they can be seen and changed
CI / test (push) Successful in 1m35s
A quota could be enforced and reported, and set nowhere but a shell. Nothing in the browser showed one at all — not what a team may spend, not what it has spent, not that it had no limit. In practice that means quotas nobody sets. /admin/quotas lists the whole tree, whether or not a group has a limit: the one without a quota is the one most likely to need one, so it cannot be the row the page leaves out. Spend over the last thirty days sits beside the budget, with the fraction used. Setting a quota here is a change of position. It was kept to the shell because raising a budget is a financial act and a stolen admin session should not perform one. That argument holds for exactly one row on this page: resolution takes the most restrictive value along the chain, so raising a team's or a project's limit cannot exceed what the organization allows. Only an organization's own limit is bounded by nothing. Every change is logged with who made it and what it was before. Per-key limits stay on the CLI. A key belongs to a person and is already bounded by its project; letting an admin session reach individual keys would put every caller's throughput one click away, and the page refuses a key scope rather than rendering it and failing later. A limit an ancestor already caps lower is marked as written-but-unreachable. Resolution means a project given more than its team is not a bigger budget, it is a number nobody will reach — and without saying so the page would show it as though it applied, with the first sign being a refusal that disagrees with the screen. The group is chosen, not typed. Ids became random in the previous commit so two organizations could both have a "platform" team, which also made them impossible to remember and a typo impossible to notice: it would write a quota onto nothing and report success. A blank field is no limit of that kind, not zero — zero is a limit of nothing, which refuses every request. Clearing removes the row rather than writing nulls across it: both resolve the same, only one tells a reader nobody has set a limit here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
fc1b47976d |
Let an admin build the tenancy tree, instead of sending them to a shell
CI / test (push) Successful in 1m35s
An account is approved into a project, a project belongs to a team, a team to an organization — and until now the only way to make any of them was `llmgw keys init` on the control plane. So the first thing to do after installing was leave the interface you were already in, and an approval form with an empty dropdown gave no hint why. The accounts page now creates all three. Creating a container is not granting access: nobody can use one until an account is approved into it, which is the step already on this page. Quotas stay on the CLI. Making somewhere to put people is structural; raising what they may spend is financial, and a stolen admin session should be able to do the first and not the second. Ids are random now instead of slugs of the name, which fixes a bug that only appears when you have a second organization — which is exactly what this feature invites. Team ids were slugs in one flat namespace and PutTeam upserts on id, so creating "platform" under a second organization moved the first one's team into it. Nothing failed. The team changed owner. Create* is separate from Put* rather than replacing it. The upserts exist so `keys init` can be re-run while provisioning a machine; a person filling in a form means something else by a name that already exists, and gets told. A duplicate is refused among siblings only. Two companies both having a "platform" team is ordinary, and refusing the second would be the flat namespace again wearing a better error message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
a83493bddb |
Show what a group spent, and fix the reports that never ran on PostgreSQL
Quotas can be written on an organization, a team or a project, and nothing could show what any of them had used: usage came back per key, per user and per model only. A quota whose consumption cannot be seen is one nobody reviews until the day it refuses somebody — and since the previous commit made a group quota a shared pool, it is now a number several people are spending against without being able to look at it. `llmgw ctl usage --by team` sums every key beneath each group, with the budget that level sets beside it. Beside it, and only when that level sets one. A team bounded by its organization's budget has none of its own, and printing the organization's against each of its teams would read as though each had that much — the exact misreading this work exists to remove. Joined through api_keys rather than read from usage.project_id: that column records where the key was when the request was served, while the quota is written against the tree. They agree until a key is moved, and a report that disagrees with the limit shown beside it is the useless one. Three existing reports were fixed on the way, all failing on PostgreSQL and all passing in tests, because the tests run on SQLite: column "k.name" must appear in the GROUP BY clause (UsageSince) column "us.email" must appear in the GROUP BY clause (UsageByUser) column "usage.ts" must appear in the GROUP BY clause (UsageBuckets) SQLite picks an arbitrary row for an ungrouped column and carries on; PostgreSQL refuses. The first two were selecting names they had not grouped. The third grouped "ts / ?" while selecting "(ts / ?) * ?", which PostgreSQL matches as text and so saw a bare ts — it is grouped by ordinal now, the same expression by construction. The dashboard's usage charts have been empty on the cluster this whole time, with the reason only in the log. All four queries were run against the cluster's own PostgreSQL before this was committed, because passing on SQLite is what they were already doing. The grouping level picks from a fixed table of levels rather than being interpolated, so an unknown one is an error and not a fragment of SQL. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
ebce41e22d |
Stop calling the PostgreSQL store SQLite, and stop guessing where it is
Two things that were only ever confusing, and one of them stopped a command working on the cluster. The store type was named SQLite and kept the name when PostgreSQL was added, so OpenPostgres returned a *SQLite and all seventy-odd methods on the production store were spelled after the database it was not running on. Nothing broke; people did. Earlier today I added three methods to *SQLite for a table that on this cluster only ever lives in PostgreSQL, and the name did not once look wrong. It is a field, which is all it ever was. `llmgw users` and `llmgw keys` defaulted --store to a SQLite path. On a deployment whose store is PostgreSQL every one of them failed with "unable to open database file" while the right answer sat in auth.store in the config file beside them; the conclusion an operator draws is that the tool is broken, and they are not far off. They now ask the server's own configuration before falling back, and the error names which of the three sources chose the address — the failure was never the path, it was which one picked it. The single-machine default still applies last, so a deployment with no config file keeps working with no flags. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
3febb6ef63 |
Count a group's quota against the group, not against each of its keys
The tenancy tree handed limits down and never pooled what was spent against them. A limit written on a team was inherited by every key beneath it, and then enforced with the key as the counter — so a project given a budget of 1000 and ten keys permitted ten thousand, and a team given 60 RPM got 60 per member. That is "each of you may spend this much", which is a useful thing to be able to say. It is not what somebody writing a budget on a team means, and nothing in the system said which one you were getting. Resolution now records which scope set each value that won, and the counter is keyed by that scope. The value and the counter come from the same place, so a project tightening its team's limit is enforced against the project's own pool rather than eating into a quota another project never agreed to. Each limit is owned separately, because they resolve separately: a team budget with a per-key rate limit — the group's money, the individual's throughput — is the ordinary arrangement and has to stay expressible. Cluster defaults are owned by the key. Read as a shared pool, one default would become a single cluster-wide rate limit that every key in every organization competes for: not a default, an outage. Restoring after a restart had to change with it. Per-key totals rebuilt a counter named after the key while the check read one named after the team, so the control plane came back believing the team had spent nothing — a budget that reset on every restart, for exactly the scopes where it matters most. SpendSinceByScope returns each key's place in the tree and the accountant folds it into every scope that might own a budget. Counters are namespaced by level throughout. A bare key id in one path and a "key:"-prefixed one in another are two counters for the same key, which is how the restore was silently doing nothing. Still missing, and worth naming: no usage report rolls up by project, team or org, so a shared quota can now be enforced but not yet shown. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
3aa250428a |
Keep what a benchmark found out, instead of leaving it in a terminal
`ctl bench` has always been able to work out what one replica can serve — it ramps concurrency until the SLO breaks and reports the last level that held. It printed that and nothing kept it, so the one number saying what a replica is worth lived in whoever ran it's scrollback. Everything downstream then had to guess: admission applies a single global default for healthy queue depth, and sizing a model against real demand had nothing to size against. `ctl bench --save` records it; `ctl capacity` reads it back. Keyed by model and profile. Capacity belongs to the hardware and the engine arguments as much as to the weights, so the same model on CPU and on an H100 are two numbers, and one key would overwrite the other on every run. The profile is looked up from the replica rather than asked for, because the control plane already knows which profile answers on which endpoint and a measurement filed under the wrong one is worse than none. A target that is not a replica — the gateway, usually — is refused with that said: benchmarking the whole path is worth doing and says nothing about the hardware. The conclusion is stored, not the ramp. Every level and its latencies are worth reading once while judging whether the run was sound; keeping them would grow without bound as a model is re-benchmarked. Re-measuring replaces. A second run is a better answer to the same question, not another data point, and keeping both would leave a reader choosing between numbers with no basis for the choice. The SLO and the date are stored with it. Capacity means nothing without the target it was measured against, and hardware and engine builds change — a measurement is an observation with a date on it, not a fact about the model. A failed save is reported and not fatal: the run has already cost the GPU time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
f2f375d824 |
Stop the text-search comment naming one language
CI / test (push) Successful in 1m41s
The choice is unchanged and now better argued: a deployment does not announce which language its notes are in, so any stemmer picked here is the wrong one somewhere — and applying the wrong one does not fail loudly, it quietly stops finding things. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
ade689fc8e |
Refuse to run the destructive store tests against a real database
CI / test (push) Successful in 1m47s
These TRUNCATE every table before each test. Pointed at a deployment that is a silent and complete data loss — every key, every user, every conversation. Not hypothetical. I did it: an SSH tunnel to the production database had been opened minutes earlier for a perfectly good reason, and the same DSN was then handed to the test suite. Twenty-one users, nine API keys, eighteen sessions, four hundred and sixty-one usage records and three runtime-added models, gone between one command and the next. Recovered only because the pre-migration SQLite file was still there. The guard is the database name. A test database has to say so, because a check that could be satisfied by accident would not have stopped this one — the DSN was entirely correct for what it had been opened for, and catastrophic for what it was then used for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
5b5e61cb97 |
Stop a second control plane from placing work behind the first's back
CI / test (push) Successful in 1m35s
The first thing HA needs that a shared store does not give. Two control planes against one store both reconcile: both see a model wanting two replicas and none running, and both place. The model ends up with twice what it asked for, each replica holding GPUs the other's inventory believes are free — and nothing crashes to say so. A lease in a table rather than a lock held by a connection. PostgreSQL has advisory locks and SQLite has nothing like them, and this store keeps one set of queries for both; a row with an expiry works everywhere, and it survives a leader being powered off rather than asked to stop, which is the case that matters. Taking and renewing are the same statement, so the two cannot disagree about what a held lease looks like, and it is one statement, so two processes racing cannot both believe they won. The owner id is random rather than the hostname. A hostname would let a restarted process match its own stale lease and believe it had been leading all along — including through the gap that made it restart, during which somebody else may have taken over and started placing. A store that cannot be reached is not evidence that somebody else is leading, so an unreachable store keeps the claim. Two leaders is worse than one, but no leader means nothing is placed and nothing is evicted, and a brief database blip should not stop a cluster healing itself. Unset means leader. Every deployment that existed before this, every single-control-plane install and every test acts unconditionally, with nothing to configure. The same seven assertions run against both backends, which is how the test harness was caught inheriting a lease between subtests — it passed on SQLite, where each subtest gets a fresh file, and failed only on PostgreSQL. What this does not yet do is move agents. A standby holding an agent's stream is one the leader cannot command, so failover is degraded rather than clean until agents know more than one address. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
b6de469351 |
Move a store to another one without reissuing every key
CI / test (push) Has been cancelled
Keys are stored hashed, so a migration that drops them cannot recreate them: every person and every script needs a new one issued, and the old ones cannot even be looked up to say which they were. `llmgw keys move-store` copies instead. Found by running it against the real cluster, twice, after the gateway had already been stopped. The first order came from the order the migrations create tables in, and users is created in a later migration than api_keys even though api_keys references it — so the copy died three tables in. The order is now checked by a test that derives the foreign keys from the migrations rather than restating them, so a new reference fails there instead of during somebody's migration. Checked by putting the wrong order back and watching it fail. Then key_claims. Its sealed column is emptied the moment a claim is used, so every claimed row holds an empty blob in a NOT NULL column. An empty blob scans as a typed nil slice and a genuine NULL as an untyped one — a distinction this side can see and pgx cannot, since it encodes any nil slice as NULL. Restored before the insert, which is the difference between a copy that completes and one that stops four tables in. One transaction per table rather than one for the whole copy: a single transaction holding every row of a large usage table is how a migration runs out of memory on the machine least able to spare it. A failure therefore leaves a partial copy, which is why the destination must have been empty — starting again is dropping and recreating it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
a76ebdb79a |
Run the store on PostgreSQL, chosen by what the address looks like
CI / test (push) Successful in 3m11s
A path is still SQLite and a postgres:// URL is PostgreSQL, so every existing configuration keeps working untouched and moving is one line changed rather than a new setting to find. An address of the form @/path/to/file reads the real one out of that file, which is how a DSN carrying a password stays out of a configuration that gets copied, diffed and pasted into issues. The driver is pgx, which is pure Go. That is not a preference: the whole binary is built with CGO_ENABLED=0 so it runs on a node whose libc is older than the builder's, and a cgo driver would take that away from every deployment, including the ones still on SQLite. Tested against a real PostgreSQL, which found two things nothing else could. The schema_migrations table is created in Go rather than in a migration file, so it escaped the port and still said INTEGER — 64 bits in SQLite, 32 in PostgreSQL, and a millisecond timestamp does not fit, so the very first migration failed to record that it had run. And the memory migration uses an FTS5 virtual table with three triggers keeping it in step, which has no counterpart: PostgreSQL derives a tsvector from the row, so there is nothing to keep in step and no trigger to forget. That leaves exactly one query that is not the same sentence in both dialects — the note search, MATCH against @@ — and it is written out twice rather than hidden behind a shared spelling that would mean something subtly different in each. Everything else is one set of queries. The text search uses the 'simple' configuration rather than English. These are an agent's own notes, in whatever language it chose, and stemming English rules over Turkish would quietly stop finding things. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
ccc9aa8b76 |
Route every query through a dialect, before there is a second one
CI / test (push) Successful in 2m30s
Groundwork for PostgreSQL, with no PostgreSQL in it yet: the store still opens SQLite and behaves exactly as it did. One set of queries rather than one per database. The SQL here is almost entirely portable — ON CONFLICT upserts work in both, and the only construct that genuinely differs is the placeholder — so two copies of every query would buy nothing and drift, one query at a time, discovered in production. The rewrite is a wrapper around the handle rather than a change at each call site. There are sixty of those and sixty chances to forget one; this way the only route to the driver goes through the rewrite. Making it a wrapper found six places that reached past it, including all three manual transactions and two prepared statements — every one of which would have worked on SQLite and failed on PostgreSQL, on a path nobody exercises before deploying. The rewriter skips quoted sections. No query here contains a question mark in a literal today; this is so that adding one is a test failure rather than a statement that silently binds the wrong argument. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
 deniz
|
2eb79f3400 |
agenta memory eklendi
CI / test (pull_request) Successful in 2m31s
|
||
|
Ilker ManapandClaude Opus 5
|
b0592670a4 |
Give projects somewhere to keep a conversation, off unless they ask
CI / test (push) Successful in 1m39s
The storage half of continuing a thread on a different model than it started on. Nothing uses it yet. Until now llmgw stored no message content anywhere: usage rows carry token counts, costs and latencies, never text, and even its own chat page keeps the conversation in a JavaScript variable that a refresh discards. That property is worth more than the feature, so it is the default and stays the default. A project stores nothing until someone turns it on for that project. Retention is a number of hours rather than a boolean, because a thread store with no expiry is a transcript of everything a team ever typed, kept forever because nobody chose a number. Turning it on means saying for how long. The window measures idleness rather than age — expiring a conversation someone is still having would be a strange way to keep that promise. Threads are scoped to a project and the scope is part of every lookup, not a check afterwards. Reading a thread from another project and reading one that does not exist return the same error: two errors would let a caller learn which ids are real in projects they cannot read. Writing across projects is refused for the same reason in the other direction — append, then read your own reply back. Deleting a thread takes its messages with it, which the test checks by counting rows rather than by trusting the cascade. Someone has to be able to say "forget that", and a retention window is no answer to a message sent by mistake. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
eb9e9b348c |
Show who is using the cluster and what they are running
CI / test (push) Successful in 1m28s
The dashboard could say what the cluster was made of and not who was spending it. Usage was per key, which is the right unit for revoking something and the wrong one for the question actually being asked: one person holds a laptop key, a CI key and a notebook key, and adding three rows up by eye is a chore that gets skipped. Per person now, joined through the keys they own, and per model — counted by the model that answered rather than the one requested, since those differ whenever a fallback serves and what actually ran is the question. Headline figures are stat tiles with a sparkline: a single current value is a number, and plotting one is how a chart ends up with a single bar in it. The sparkline says only rising, falling or flat, which is the one thing the number cannot. Ranked horizontal bars, because the labels are email addresses and model names and the label is the point — the question is which person, not what shape the distribution has. Rotated or truncated axis labels say the chart matters more than reading it. Trimmed where drawn and complete on hover, so the plot keeps its geometry and a reader can still find out whose row it is. The palette is validated rather than chosen: every adjacent pair clears the colour-vision separation threshold, and the dark values are the same hues re-stepped for the dark surface instead of an automatic flip. Run through a validator, because that is the only way anyone can honestly claim a palette is colourblind-safe. Which turned up a bug in the existing chart code. cssVar is an allowlist of four names that silently returns --accent for anything else, and callers wrap it in var() themselves — so every new series would have rendered in one colour with accent-coloured text. Caught by a test written from the rule that chart text wears text tokens, never the series colour. A key with no owner is labelled "operator-issued keys" rather than unknown: it predates accounts and is doing its job, and "unknown" would imply something is wrong. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
277add5ccb |
Keep runtime models, so an added model outlives a restart
CI / test (push) Successful in 1m25s
The catalog can change while the cluster runs; this is where a change goes so it is still there tomorrow. Models declared in configuration and models added at runtime are two sources on purpose: config-as-code stays declarative and under review, and the database holds what someone added to try it. Configuration wins every conflict, and the direction matters. A request for a reviewed production model must never be answered by something added from a laptop, so a stored model colliding with a declared name is refused when it is added. A declared model cannot be removed at runtime either — it would return on the next restart, leaving the cluster disagreeing with its own config file until then, which is a worse answer than refusing and saying where to edit. The whole catalog is built before anything is written. A model naming a missing fallback, or colliding, or malformed in any of the ways catalog.New refuses, is rejected rather than persisted and then skipped forever after — a model that looks added and never appears is the worst outcome available. Removal is checked the same way, because another model may fall back to the one being removed, and a fallback pointing at nothing is discovered during the outage it was meant to cover. At startup each stored model is validated on its own against what is already accepted, so one bad row is skipped and named rather than taking the cluster with it. The declared models are precisely the ones that must survive that. Catalog types now carry json tags mirroring their koanf ones, so a stored definition is the same shape as the YAML someone writes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
c115681259 |
Meter a project nobody set limits on
CI / test (push) Has been cancelled
Unlimited was the default. Creating a project without limits is the easy path and the common one — the "default" project on this cluster is one, and the person using it is metered by nothing at all. The first sign of that is the bill or the outage. auth.default_rpm and default_tpm now apply to a key no scope gives a limit to. Zero keeps the old behaviour, which is still a reasonable choice; the point is that it becomes a choice rather than an omission. Applied after the chain rather than folded in as its widest scope, because a default is a fallback and not a ceiling: a project deliberately given 600 rpm must keep it, and tightening would silently cap it at the default. An explicit limit wins in both directions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
de21649ad6 |
Add an in-browser chat, without putting a key in the browser
CI / test (push) Successful in 1m13s
So someone can try a model without setting up a client, and — more often — tell whether a problem is the gateway or their code. The page carries a session, and the server turns that into the identity an API key would have produced, resolved from the person's project. The request then goes down the ordinary path by being handed to the same handler /v1 uses: admission, model allowlist, rate limits, budget, accounting. There is exactly one request path, so there is no second place for its rules to drift — and without that, a playground is a hole in every limit the rest of the gateway enforces. The allowlist test posts a forbidden model and expects 403 for that reason. The body is rebuilt rather than forwarded, so the page cannot set fields it has no business setting: a stream:false that would hang the reader, a max_tokens large enough to occupy a replica for an hour. One turn is capped at 512 tokens. The picker offers what is actually served and allowed, not what is catalogued, because offering a model with no replica produces a 503 the reader cannot interpret. Inline vanilla JavaScript, for the same reason the charts are hand-drawn SVG. It streams, because on CPU inference ten seconds is normal and a chat that hangs and then dumps a paragraph feels broken. Replies are written with textContent throughout: model output and user input have no business being parsed as markup. Its POSTs carry the CSRF token in a header, which a cross-site form cannot set at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
859f08da9b |
Approve accounts from the browser, and draw the traffic
CI / test (push) Successful in 1m19s
Two things the dashboard was missing: it could not act, and it was a column of numbers. Account administration moves to /admin/users — approving into a project, suspending, reinstating. That is the work an operator does constantly and it should not need a terminal. Three things stay on the CLI, and the line is not arbitrary: promoting someone to admin, resetting a password, and issuing a key on another person's behalf each turn a stolen admin session into more than it was — permanent access, or somebody else's account. Approving and suspending do not. Suspending signs the account out everywhere, or suspension is advisory until the session happens to expire, and an admin cannot suspend themselves, which is not a thing an interface should let you do by misclicking. The charts are inline SVG generated in Go. A charting library would mean a JavaScript toolchain in CI and a megabyte of bundle for what is a list of rectangles, and the single-binary goal rules that out. Hourly over a day: fine enough to show a working day, coarse enough that one slow minute is not an incident. Empty hours are filled in, because a chart with hours missing where nothing happened reads as a chart with hours missing from the data. The SVG is marked as trusted HTML, so everything that reaches it is either a number this package computed or escaped explicitly, and a bar's colour can only be one of four names — a colour that could be anything is a way to write CSS into every page the chart appears on. There are tests for both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
e519dc587c |
Serve the cluster dashboard over the web, without serving pprof with it
CI / test (push) Successful in 1m21s
Reaching it meant an SSH tunnel, because the page lived on the admin listener and that listener is bound to loopback. Opening the port was never the way to fix that: it also serves pprof, and pprof can dump the process heap, where every API key sits in plaintext. Exposing the port would have been exposing the key store. So the page moves rather than the port. The same dashboard is now mounted at /admin on the client-facing port, behind the session and CSRF machinery that already guards /profile, gated on a flag on the account. The admin listener keeps pprof and metrics and stays on loopback. One bit, not a role system: it grants looking. Everything that changes the cluster or issues a credential stays on the CLI, where it needs a shell rather than a session — which keeps the worst case of a stolen admin cookie to "saw the cluster" rather than "drained your fleet". Someone without the flag gets 404, not 403. A 403 confirms the page is there and that admin accounts exist here, which is free reconnaissance; 404 is what every other unknown path returns. The profile page does not show a link to a door the visitor cannot open. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
79e459d9ff |
Let people change their own password, and operators reset a lost one
CI / test (push) Failing after 1m11s
The account system shipped without either, which made the first forgotten password an unanswerable support request. Changing it requires the current one. A session is what an attacker gets from a borrowed laptop or a stolen cookie; the password is what they do not have, and without the check the first thing they would do is take the account. Every session is closed, including the browser doing the changing — leaving the others open would mean a password change does nothing about the reason someone is changing it. A new session is then opened for the browser in front of us, so the person who just proved their password is not thrown back to the sign-in page. That response also carries a new CSRF cookie, so the forms it renders take the new token rather than the one the request arrived with; there is a test that posts a form from that exact page. Guessing the current password is bounded the same way sign-in is, because someone holding a stolen session should not get unlimited attempts at the password that would let them keep it. There is no self-service reset: this gateway sends no email, and a reset link with nowhere to send it is not a reset. `llmgw users reset-password` is the operator's answer, and it closes their sessions too — if someone lost their password, the reason might be that somebody else has it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
216fc9bb03 |
Let people sign up, be approved, and issue their own keys
CI / test (push) Successful in 1m9s
Every key needed an operator with a shell. This adds the login system that made that necessary, which is the thing I argued against building until it was asked for — so it is built as if it will be attacked, because a public login form is. Signing up produces an account that can do nothing. Approval is what makes it work and stays on the CLI: having a shell on the control plane is what makes someone an administrator, and that is a smaller thing to get right than roles and an admin UI that mints credentials. The details that matter: Passwords are argon2id, where a slow hash is correct — the input is chosen by a human and has a dictionary, which is the opposite of an API key. Both kinds of authentication now live in this codebase and they are in separate packages so neither set of assumptions leaks into the other. Length is the only rule: composition rules push people towards Password1! and cost more entropy than they buy. Signup never says whether an address already has an account, and sign-in answers the same way for a wrong password and a missing account — verifying against a dummy hash when there is no user, so a stopwatch cannot enumerate accounts. Account state is checked after the password, not before, so guessing an address reveals nothing about it. Guessing is bounded per address rather than per IP: an attacker has many addresses to come from and one to guess against, and limiting by IP locks out an office behind one NAT. Sessions are server-side, so signing out and suspending take effect immediately rather than whenever a token expires; suspension is checked in the same query that resolves the session, because a second round trip is a check somebody eventually forgets. Every state-changing form carries a CSRF token verified against a hash in the session — a cookie authenticates the browser, not the page that asked. Key ownership is enforced in the UPDATE rather than by reading first, so a guessed key id from another account changes nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
3ed211b2f4 |
Onboard users with one-time links instead of pasted keys
CI / test (push) Successful in 2m43s
Issuing a key worked; getting it to the person it was issued for did not. The operator had a secret on their terminal and no way to send it that did not leave a bearer credential in a chat log forever. `keys create --claim` prints a link instead. GET shows a button and consumes nothing; POST reveals the key and burns the link. That split is not politeness — chat clients, mail scanners and link previewers fetch URLs the moment they are posted, so a GET that consumed the claim would mean the first thing to unfurl the link destroys the key before anyone reads it. A pending claim has to hold the key somewhere, which would otherwise put a working credential in the database and undo what hashing bought. So the key is sealed with a key derived from the link's own token, and the token is never stored — only its hash. A database leak yields two useless hashes and a blob nothing in the database can open. HKDF rather than the raw token, so the value used for lookup can never become the value that decrypts. Consuming is one UPDATE moving claimed_at off NULL, so two people opening the same link produce one key rather than two copies. A used link, an expired one and a token that was never real all render identically, because saying which confirms that a token was once valid. The page is no-store, no-referrer and noindex: the URL is itself the credential. `GET /me` reports a key's own limits and spend, so nobody has to ask an operator to read a dashboard they cannot reach. Outside /v1, because it is not part of the OpenAI API. docs/onboarding.md is written for whoever operates this, including the parts that are deliberately absent — no self-service, no SSO, and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
cd3233ffe8 |
Do not leave the key store world-readable
CI / test (push) Successful in 1m1s
SQLite creates its file with the process umask, which on a stock system means every account on the machine can read it. What is in there is not a working credential — keys are stored hashed — but it is the whole tenancy tree and every request anyone made, and that should not depend on an operator remembering to chmod it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
1839832b8b |
Add a landing page and a read-only cluster dashboard
CI / test (push) Successful in 1m2s
Pasting the gateway's address into a browser returned 404, which looks like a broken endpoint rather than one that tells you how to use it. Two pages, and the split between them is a security boundary rather than a layout choice. The landing page is served without a key — it has to be, because the person reading it is the one who does not have one yet — so it contains no data at all: it says how to connect, not what is here. Model names stay behind /v1/models. It reconstructs the address from the forwarded headers so the copy-pasteable example names the address the client actually used rather than the internal one; those headers are reflected into displayed text and never used for a decision. The dashboard shows nodes, models, replicas, placement failures and usage, and lives on the admin listener only. That listener is loopback-bound and also serves pprof, which can dump the process heap — where API keys sit in plaintext — so reaching the page from elsewhere is an SSH tunnel's job. It is read-only: anything that mints a credential stays in `llmgw keys`, where it needs a shell rather than a browser session this project has no way to authenticate. Mounting the landing page uncovered a hole in the exemption list. An entry ending in "/" was a prefix rule, and "/" ends in "/", so exempting the landing page would have exempted every path there is — authentication still installed, still logging, and letting the whole API through. "/" is now exact, with a test enumerating what must and must not be exempt. Server-rendered templates and one stylesheet, embedded. The single-binary goal rules out a JavaScript toolchain and nothing on either page needs one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Ilker ManapandClaude Opus 5
|
29617f8cb1 |
Add the store: tenancy, keys and usage on SQLite
Phase 5 begins. This is the persistence layer for the things the cluster cannot reconstruct — who may call, what they are allowed, what they used. Everything else llmgw knows is still derived from the fleet, which cannot be wrong about itself the way a database can (§6.5). SQLite behind an interface. The plan justified that by write volume, which was the wrong argument: the write path is batched anyway, and a 500-row transaction every 200ms is nothing for either engine. The real argument is the failure mode. Postgres is a second service, and a gateway that stops serving because its accounting database is unreachable is worse than one that buffers some accounting. Embedded means the store shares the gateway's fate instead of adding an independent way to be down. That reasoning stops holding at two instances, where budgets have to be counted somewhere shared — so the interface is here from the start, the SQL is portable, and migrations are per dialect with shared version numbers. Keys are stored as sha256 and nothing else, so a database leak hands over no working credential. The fast hash is deliberate: the input is 256 random bits with no dictionary to attack, so argon2 would buy nothing and charge tens of milliseconds per request. That assumption is written down next to the code that depends on it. Two things the schema now says that it did not: budgets are integer micro-units, because repeated addition of floats drifts; and an empty model allowlist forbids everything while an absent one permits everything, which the resolver has to keep distinct. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |