Let a person belong to many places, or to none
A person was in exactly one project, chosen at approval, and could not use the gateway before that. People belong to several teams and to none, so: - memberships: a person is a member of any number of organizations, teams and projects, or of none; a member of a team or organization may use every project in it. users join/leave, and the people page adds and takes out from either side. Approval no longer needs a project. - personal keys: a key may have an owner and no project. /profile asks what a new key spends against; keys create --for does the same. auth.personal sets default limits for personal use or turns it off. - limits for people: --scope user holds a person across every key, project and the chat page; --per-member on an org, team or project gives each member their own share there. Resolution folds the tree, each per-member limit for the key's owner, and the owner's own, each in its own pool; the quotas page shows and sets both. - the chat page asks what to spend against and checks it per message. - usage records who spent it, so a person's budget survives a restart, personal use and chat included; restore no longer drops rows with no project. Leaving a project revokes the keys made there unless another membership still reaches it. A personal key keeps no threads or notes, which are per project, and is refused rather than dropped into the anonymous pool. Migration 0010 turns each person's project into a membership, adds usage.user_id and backfills it, and makes api_keys.project_id nullable. SQLite cannot drop NOT NULL, so the table is rebuilt; that needs foreign keys off, which only works outside a transaction, so migrations marked for it run on one connection with foreign_key_check before commit. Tested on a populated SQLite store with claims, and on a copy of the cluster's PostgreSQL store: 5 people became members, 17 keys and 3 claims intact, 145 of 577 usage rows and all 69 chat rows attributed. The PostgreSQL store tests, including new membership cases, pass on 16. users move is gone; join and leave replace it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
429c05a319
commit
098886e0e4
46 files changed
+2372
-468
No files matched your search
+67
-12
@@ -39,10 +39,11 @@ The landing page then links to it, and the rest is:
|
||||
|
||||
```
|
||||
llmgw users list --pending --store …
|
||||
ID EMAIL NAME STATE PROJECT SIGNED UP
|
||||
usr-k4mq… ayse@example.com Ayşe pending - 4m12s ago
|
||||
ID EMAIL STATE ADMIN MEMBER OF SIGNED UP
|
||||
usr-k4mq… ayse@example.com pending - on their own 4m12s ago
|
||||
|
||||
llmgw users approve usr-k4mq… --project default --by ilker --store …
|
||||
llmgw users approve ayse@example.com --by ilker --store …
|
||||
llmgw users approve ayse@example.com --project Acme/ml/training --store … # or straight into a project
|
||||
```
|
||||
|
||||
That is the whole operator loop. From then on they sign in at `/login`, and
|
||||
@@ -105,15 +106,53 @@ The tree is not fixed once made:
|
||||
llmgw org rename project Acme/platform/gateway api-gateway
|
||||
llmgw org move project Acme/platform/gateway --to Scania/ml
|
||||
llmgw org remove team Acme/old # only when nothing is in it
|
||||
llmgw users move ayse@example.com --project Acme/ml/research
|
||||
llmgw users delete ayse@example.com --yes # signs out, revokes their keys
|
||||
llmgw keys list --all # every key, its project and owner
|
||||
llmgw keys list --all # every key, where it spends, and whose
|
||||
```
|
||||
|
||||
Moving a person takes the keys they made with them, so they stop spending the
|
||||
old project's budget; keys an operator issued in the project stay with it. A
|
||||
project that ever had a key cannot be removed, because usage history refers to
|
||||
it — rename it instead.
|
||||
A project that ever had a key cannot be removed, because usage history refers
|
||||
to it — rename it instead.
|
||||
|
||||
### Where people belong
|
||||
|
||||
A person can be a member of any number of organizations, teams and projects, or
|
||||
of none:
|
||||
|
||||
```
|
||||
llmgw users join ayse@example.com --team Acme/ml # every project in ml
|
||||
llmgw users join ayse@example.com --project Scania/research/vision
|
||||
llmgw users leave ayse@example.com --project Acme/ml/training
|
||||
```
|
||||
|
||||
A member of a team or organization may use every project in it. Leaving revokes
|
||||
the keys the person made in projects they can no longer reach; a project they
|
||||
still reach through another membership keeps them.
|
||||
|
||||
Someone who is a member of nothing still uses the gateway, on their own:
|
||||
personal keys, and the chat page, within their personal limits. When they make
|
||||
a key on `/profile` they choose what it spends against — one of their projects,
|
||||
or "Personal". An operator can do the same:
|
||||
|
||||
```
|
||||
llmgw keys create laptop --for ayse@example.com # personal
|
||||
llmgw keys create ci --for ayse@example.com --project Acme/ml/training
|
||||
```
|
||||
|
||||
Personal use can be limited by default, or turned off so people use the gateway
|
||||
only through their projects:
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
personal:
|
||||
rpm: 20
|
||||
tpm: 50000
|
||||
budget: 10 # whole units
|
||||
period: month
|
||||
# disabled: true
|
||||
```
|
||||
|
||||
A personal key has no project, so it keeps no conversations or agent notes;
|
||||
those are kept per project.
|
||||
|
||||
`llmgw keys init` still works for provisioning scripts and is idempotent, but it
|
||||
will no longer move a team or project that already exists under another parent;
|
||||
@@ -195,6 +234,20 @@ llmgw keys limits --store … --scope project --id Acme/platform/gateway \
|
||||
--budget 50 --period month # 50 units a month
|
||||
```
|
||||
|
||||
Two more kinds of limit follow people rather than the tree:
|
||||
|
||||
```
|
||||
# A person, everywhere: every key they hold, every project, the chat page
|
||||
llmgw keys limits --scope user --id ayse@example.com --budget 25
|
||||
|
||||
# Each member of a project separately — a fair share, not one shared pool
|
||||
llmgw keys limits --scope project --id Acme/ml/training --per-member --rpm 20
|
||||
```
|
||||
|
||||
A request is held to all of them at once: the key's own, its project's, team's
|
||||
and organization's, each per-member limit on those for its owner, and the
|
||||
owner's own. The strictest wins, and each is counted in its own pool.
|
||||
|
||||
## The landing page
|
||||
|
||||
Someone who reaches the gateway without a key sees the landing page. Tell it who
|
||||
@@ -314,9 +367,11 @@ page exists.
|
||||
## Managing accounts from the browser
|
||||
|
||||
`/admin/users` is the tree: organizations, teams and projects, each project
|
||||
opening onto its people and its keys. From there an admin approves, moves,
|
||||
suspends and removes people, revokes keys, and adds, renames, moves and removes
|
||||
organizations, teams and projects. A searchable list of every account is at the
|
||||
opening onto its members and its keys. From there an admin approves people
|
||||
(into a project or on their own), adds them to and takes them out of
|
||||
organizations, teams and projects, suspends and removes them, revokes keys, and
|
||||
adds, renames, moves and removes organizations, teams and projects. `/admin/quotas`
|
||||
sets limits on the tree, per member of any part of it, and per person. A searchable list of every account is at the
|
||||
bottom. The page works as plain forms; with scripts on, a change updates it in
|
||||
place and the projects you had open stay open.
|
||||
|
||||
|
||||
@@ -29,6 +29,7 @@ rather than repeated.
|
||||
| `default_rpm` | `int` | [Auth](#auth) |
|
||||
| `default_tpm` | `int` | [Auth](#auth) |
|
||||
| `signup_open` | `bool` | [Auth](#auth) |
|
||||
| `personal` | `Personal` | [Auth](#auth) |
|
||||
| `pricing` | `map[string]Price` | [Auth](#auth) |
|
||||
| `default_price` | `Price` | [Auth](#auth) |
|
||||
| `flush_interval` | `time.Duration` | [Auth](#auth) |
|
||||
@@ -102,6 +103,11 @@ rather than repeated.
|
||||
| `ib_root` | `string` | [Options](#options) |
|
||||
| `prefer_nvml` | `bool` | [Options](#options) |
|
||||
| `skip_gpus` | `bool` | [Options](#options) |
|
||||
| `disabled` | `bool` | [Personal](#personal) |
|
||||
| `rpm` | `int` | [Personal](#personal) |
|
||||
| `tpm` | `int` | [Personal](#personal) |
|
||||
| `budget` | `float64` | [Personal](#personal) |
|
||||
| `period` | `string` | [Personal](#personal) |
|
||||
| `enabled` | `bool` | [Prefix](#prefix) |
|
||||
| `max_nodes` | `int` | [Prefix](#prefix) |
|
||||
| `min_match` | `int` | [Prefix](#prefix) |
|
||||
@@ -267,6 +273,15 @@ SignupOpen serves the signup form. Accounts it creates are pending until
|
||||
an operator approves them, so this is not a way in — but turning it off
|
||||
means accounts are made with `llmgw users create` instead.
|
||||
|
||||
### `personal`
|
||||
|
||||
`Personal`
|
||||
|
||||
Personal is what a person may do on their own, outside any project: make
|
||||
personal keys and use the chat page personally, within these limits.
|
||||
|
||||
Takes a [Personal](#personal) block.
|
||||
|
||||
### `pricing`
|
||||
|
||||
`map[string]Price`
|
||||
@@ -788,6 +803,33 @@ source of NVLink topology.
|
||||
SkipGPUs omits GPU discovery. A node without nvidia-smi is already
|
||||
found to have none, so this is for one whose driver is broken.
|
||||
|
||||
## Personal
|
||||
|
||||
### `disabled`
|
||||
|
||||
`bool`
|
||||
|
||||
Disabled refuses personal keys and personal chat; people then use the
|
||||
gateway only through projects they are members of.
|
||||
|
||||
### `rpm`
|
||||
|
||||
`int`
|
||||
|
||||
### `tpm`
|
||||
|
||||
`int`
|
||||
|
||||
### `budget`
|
||||
|
||||
`float64`
|
||||
|
||||
Budget is in whole units per Period ("day" or "month").
|
||||
|
||||
### `period`
|
||||
|
||||
`string`
|
||||
|
||||
## Prefix
|
||||
|
||||
### `enabled`
|
||||
|
||||
Reference in new issue
Block a user