Ilker ManapandClaude Opus 5
3ed211b2f4
CI / test (push) Successful in 2m43s
Issuing a key worked; getting it to the person it was issued for did not. The operator had a secret on their terminal and no way to send it that did not leave a bearer credential in a chat log forever. `keys create --claim` prints a link instead. GET shows a button and consumes nothing; POST reveals the key and burns the link. That split is not politeness — chat clients, mail scanners and link previewers fetch URLs the moment they are posted, so a GET that consumed the claim would mean the first thing to unfurl the link destroys the key before anyone reads it. A pending claim has to hold the key somewhere, which would otherwise put a working credential in the database and undo what hashing bought. So the key is sealed with a key derived from the link's own token, and the token is never stored — only its hash. A database leak yields two useless hashes and a blob nothing in the database can open. HKDF rather than the raw token, so the value used for lookup can never become the value that decrypts. Consuming is one UPDATE moving claimed_at off NULL, so two people opening the same link produce one key rather than two copies. A used link, an expired one and a token that was never real all render identically, because saying which confirms that a token was once valid. The page is no-store, no-referrer and noindex: the URL is itself the credential. `GET /me` reports a key's own limits and spend, so nobody has to ask an operator to read a dashboard they cannot reach. Outside /v1, because it is not part of the OpenAI API. docs/onboarding.md is written for whoever operates this, including the parts that are deliberately absent — no self-service, no SSO, and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
151 lines
5.3 KiB
Markdown
151 lines
5.3 KiB
Markdown
# Onboarding a user
|
|
|
|
Written for whoever operates an llmgw deployment: the person with a shell on the
|
|
control plane machine.
|
|
|
|
Everything here runs as the user the server runs as, so the files it creates stay
|
|
readable by it:
|
|
|
|
```
|
|
sudo -u llmgw llmgw keys <command> --store /var/lib/llmgw/llmgw.db
|
|
```
|
|
|
|
The `--store` flag is repeated in every example below. Set it once in a shell
|
|
alias if you do this often; it deliberately has no server-side default beyond
|
|
`/var/lib/llmgw/llmgw.db`.
|
|
|
|
## 1. Decide where the user belongs
|
|
|
|
Keys hang off a project, projects off a team, teams off an organization. Limits
|
|
can be set at any of those levels and compose downward by taking the **most
|
|
restrictive** value — a key cannot raise what its team set, and a team cannot
|
|
raise what its organization set. That is the only rule under which a budget
|
|
means anything.
|
|
|
|
If you have no structure yet, one of each is fine:
|
|
|
|
```
|
|
llmgw keys init --store … --org mynodes --team default --project default
|
|
```
|
|
|
|
It is idempotent: running it again renames rather than duplicating, so it can
|
|
live in whatever provisions the machine.
|
|
|
|
Split it when you want separate limits or separate reporting, not before.
|
|
|
|
## 2. Issue the key as a one-time link
|
|
|
|
```
|
|
llmgw keys create --store … \
|
|
--project default \
|
|
--band gold \
|
|
--base-url https://llmgw.example \
|
|
--claim \
|
|
"ayse-laptop"
|
|
|
|
https://llmgw.example/claim/3mKq…
|
|
```
|
|
|
|
Send the link, not the key.
|
|
|
|
A key pasted into a chat window stays in that chat window forever, searchable by
|
|
everyone who joins the channel later. A link works once and then expires, so the
|
|
worst case is someone opening it before the intended person — which you find out
|
|
about, because they will tell you the link is dead.
|
|
|
|
The name is for you, not the user: it appears in `keys list` and is how you will
|
|
know which key to revoke.
|
|
|
|
**Why the link is safe to store.** The key is sealed with the link's own token,
|
|
and the token is never written down — only its hash is. A database leak yields
|
|
two useless hashes and a blob nothing in the database can open.
|
|
|
|
Without `--claim` the key is printed to stdout instead, for when you are
|
|
provisioning something that is not a person:
|
|
|
|
```
|
|
llmgw keys create --store … --project ci "github-runner" > /run/secrets/llmgw-key
|
|
```
|
|
|
|
## 3. Set limits
|
|
|
|
Limits are optional. A key with none is unlimited, which is the right default on
|
|
a private cluster and the wrong one the moment the gateway faces anything else.
|
|
|
|
```
|
|
# Everyone in this project
|
|
llmgw keys limits --store … --scope project --id default \
|
|
--rpm 60 --tpm 200000
|
|
|
|
# One key, tighter
|
|
llmgw keys limits --store … --scope key --id key-mpmip77twpt7qaxc \
|
|
--rpm 10
|
|
|
|
# Restrict which models
|
|
llmgw keys limits --store … --scope project --id default \
|
|
--models qwen-0.5b,llama-3.3-70b
|
|
```
|
|
|
|
An **absent** model list permits everything. An **empty** one permits nothing —
|
|
`--models ""` is a way to suspend a project without revoking its keys.
|
|
|
|
Allowlists compose by intersection: a key cannot add a model its team withheld.
|
|
A model's canonical name and any of its aliases name the same thing, so either
|
|
form works.
|
|
|
|
Budgets are counted in micro-units, and what a unit means is yours to decide.
|
|
Self-hosted models have no per-token price, so it can be GPU-seconds, a notional
|
|
rate, or 1 000 000 everywhere — which turns a budget into a plain token quota.
|
|
Models are priced in the server's `auth.pricing` section; a model priced at zero
|
|
is exempt from budget checks, which is what lets a key with nothing left still be
|
|
served a cheap fallback.
|
|
|
|
```
|
|
llmgw keys limits --store … --scope project --id default \
|
|
--budget 50000000 --period month # 50 units a month
|
|
```
|
|
|
|
## 4. What the user does
|
|
|
|
They open the link, press the button, and get the key once. The page shows them
|
|
the base URL and a working `curl`, and points at two endpoints:
|
|
|
|
- `GET /v1/models` — what this key may use
|
|
- `GET /me` — its limits and what it has spent
|
|
|
|
`/me` is there so nobody has to ask you "why was I rate limited" or "how much
|
|
have I used". It is outside `/v1` because it is not part of the OpenAI API.
|
|
|
|
## 5. Afterwards
|
|
|
|
```
|
|
llmgw keys list --store … --project default
|
|
```
|
|
|
|
`LAST USED` is written in batches rather than per request, so it lags by up to a
|
|
minute. It is the column that tells you whether a key is still in use, which is
|
|
what makes rotation safe:
|
|
|
|
**To rotate**, issue the new key, let both work while the user switches, watch
|
|
`LAST USED` on the old one stop moving, then revoke it.
|
|
|
|
```
|
|
llmgw keys revoke --store … key-mpmip77twpt7qaxc
|
|
```
|
|
|
|
Revocation is a soft delete: the key stops working, the row stays, and its usage
|
|
history keeps its subject. A running server keeps accepting a revoked key until
|
|
its cache expires — `auth.cache_ttl`, 30 seconds by default. That is the price of
|
|
keeping the database off the request path, and it is the number to lower if you
|
|
need revocation to bite faster.
|
|
|
|
## What this does not do
|
|
|
|
- **No self-service.** Every key needs someone with a shell here. That is
|
|
deliberate: an endpoint that mints credentials needs a login system, and this
|
|
project does not have one. If you outgrow it, that is the thing to build, not a
|
|
password field on the dashboard.
|
|
- **No SSO.** Same reason.
|
|
- **Rate limits are per gateway instance.** With two instances each counts its own,
|
|
so the effective limit doubles. v1 is single-instance by design.
|