Files
llmgw/docs/onboarding.md
T
Ilker ManapandClaude Opus 5 216fc9bb03
CI / test (push) Successful in 1m9s
Let people sign up, be approved, and issue their own keys
Every key needed an operator with a shell. This adds the login system that
made that necessary, which is the thing I argued against building until it was
asked for — so it is built as if it will be attacked, because a public login
form is.

Signing up produces an account that can do nothing. Approval is what makes it
work and stays on the CLI: having a shell on the control plane is what makes
someone an administrator, and that is a smaller thing to get right than roles
and an admin UI that mints credentials.

The details that matter:

Passwords are argon2id, where a slow hash is correct — the input is chosen by
a human and has a dictionary, which is the opposite of an API key. Both kinds
of authentication now live in this codebase and they are in separate packages
so neither set of assumptions leaks into the other. Length is the only rule:
composition rules push people towards Password1! and cost more entropy than
they buy.

Signup never says whether an address already has an account, and sign-in
answers the same way for a wrong password and a missing account — verifying
against a dummy hash when there is no user, so a stopwatch cannot enumerate
accounts. Account state is checked after the password, not before, so guessing
an address reveals nothing about it. Guessing is bounded per address rather
than per IP: an attacker has many addresses to come from and one to guess
against, and limiting by IP locks out an office behind one NAT.

Sessions are server-side, so signing out and suspending take effect
immediately rather than whenever a token expires; suspension is checked in the
same query that resolves the session, because a second round trip is a check
somebody eventually forgets. Every state-changing form carries a CSRF token
verified against a hash in the session — a cookie authenticates the browser,
not the page that asked.

Key ownership is enforced in the UPDATE rather than by reading first, so a
guessed key id from another account changes nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 09:32:35 +02:00

232 lines
8.0 KiB
Markdown

# Onboarding a user
Written for whoever operates an llmgw deployment: the person with a shell on the
control plane machine.
There are two ways a person gets a key. Pick one and put it in the config.
| | Self-service | Operator-issued |
|---|---|---|
| Config | `auth.signup_open: true` | `auth.signup_open: false` |
| They do | sign up, wait, then make their own keys | wait for a link |
| You do | approve once | issue every key |
| Good for | more than a handful of people | a few, or machines |
Self-service does not mean open. A signup produces an account that can do
nothing at all; **approval is what makes it work**, and approval needs a shell
here rather than a session cookie, because it is the step that decides who gets
to spend the cluster.
Everything below runs as the user the server runs as, so the files it creates
stay readable by it:
```
sudo -u llmgw llmgw <command> --store /var/lib/llmgw/llmgw.db
```
## Self-service
Turn on the form:
```yaml
auth:
enabled: true
store: /var/lib/llmgw/llmgw.db
signup_open: true
```
The landing page then links to it, and the rest is:
```
llmgw users list --pending --store …
ID EMAIL NAME STATE PROJECT SIGNED UP
usr-k4mq… ayse@example.com Ayşe pending - 4m12s ago
llmgw users approve usr-k4mq… --project default --by ilker --store …
```
That is the whole operator loop. From then on they sign in at `/login`, and
`/profile` is where they create keys, see when each was last used, and revoke the
ones they are done with. A key is shown once on the page that creates it and
never again.
**This gateway sends no email.** There is no "your account was approved" message
and no password reset, so tell people what to expect: they sign up, they tell
you, you approve, they sign in. If that loop is too manual for your size, it is
the thing to automate — not the approval step.
To close an account:
```
llmgw users suspend usr-k4mq… --store …
```
which closes their sessions immediately. Their **API keys keep working** — those
are separate credentials, and revoking them is a separate step:
```
llmgw keys list --project default --store …
llmgw keys revoke key-… --store …
```
That separation is deliberate: a suspended person's running jobs should not all
fail at once without someone deciding they should.
## Operator-issued
With `signup_open: false` there is no form, and you issue keys yourself. This is
also the path for things that are not people — CI, a notebook server, a script.
### Decide where the key belongs
Keys hang off a project, projects off a team, teams off an organization. Limits
can be set at any of those levels and compose downward by taking the **most
restrictive** value — a key cannot raise what its team set, and a team cannot
raise what its organization set. That is the only rule under which a budget
means anything.
If you have no structure yet, one of each is fine:
```
llmgw keys init --store … --org mynodes --team default --project default
```
It is idempotent: running it again renames rather than duplicating, so it can
live in whatever provisions the machine.
Split it when you want separate limits or separate reporting, not before.
### Issue the key as a one-time link
```
llmgw keys create --store … \
--project default \
--band gold \
--base-url https://llmgw.example \
--claim \
"ayse-laptop"
https://llmgw.example/claim/3mKq…
```
Send the link, not the key.
A key pasted into a chat window stays in that chat window forever, searchable by
everyone who joins the channel later. A link works once and then expires, so the
worst case is someone opening it before the intended person — which you find out
about, because they will tell you the link is dead.
The name is for you, not the user: it appears in `keys list` and is how you will
know which key to revoke.
**Why the link is safe to store.** The key is sealed with the link's own token,
and the token is never written down — only its hash is. A database leak yields
two useless hashes and a blob nothing in the database can open.
Without `--claim` the key is printed to stdout instead, for when you are
provisioning something that is not a person:
```
llmgw keys create --store … --project ci "github-runner" > /run/secrets/llmgw-key
```
## Limits
Limits are optional. A key with none is unlimited, which is the right default on
a private cluster and the wrong one the moment the gateway faces anything else.
```
# Everyone in this project
llmgw keys limits --store … --scope project --id default \
--rpm 60 --tpm 200000
# One key, tighter
llmgw keys limits --store … --scope key --id key-mpmip77twpt7qaxc \
--rpm 10
# Restrict which models
llmgw keys limits --store … --scope project --id default \
--models qwen-0.5b,llama-3.3-70b
```
An **absent** model list permits everything. An **empty** one permits nothing —
`--models ""` is a way to suspend a project without revoking its keys.
Allowlists compose by intersection: a key cannot add a model its team withheld.
A model's canonical name and any of its aliases name the same thing, so either
form works.
Budgets are counted in micro-units, and what a unit means is yours to decide.
Self-hosted models have no per-token price, so it can be GPU-seconds, a notional
rate, or 1 000 000 everywhere — which turns a budget into a plain token quota.
Models are priced in the server's `auth.pricing` section; a model priced at zero
is exempt from budget checks, which is what lets a key with nothing left still be
served a cheap fallback.
```
llmgw keys limits --store … --scope project --id default \
--budget 50000000 --period month # 50 units a month
```
## The landing page
Someone who reaches the gateway without a key sees the landing page. Tell it who
to send them to, or it can only shrug:
```yaml
landing:
contact: "platform@example.com" # or an https:// URL, or "#platform on Slack"
notice: "Internal use only."
```
An address becomes a mailto link, an `http(s)` URL stays a link, anything else is
shown as text. Other schemes are never linked — configuration is not always
written by whoever runs the gateway, and a `javascript:` href on a page served to
every visitor is a short path to a stored XSS.
## What the user sees
They open the link, press the button, and get the key once. The page shows them
the base URL and a working `curl`, and points at two endpoints:
- `GET /v1/models` — what this key may use
- `GET /me` — its limits and what it has spent
`/me` is there so nobody has to ask you "why was I rate limited" or "how much
have I used". It is outside `/v1` because it is not part of the OpenAI API.
## Rotation and revocation
```
llmgw keys list --store … --project default
```
`LAST USED` is written in batches rather than per request, so it lags by up to a
minute. It is the column that tells you whether a key is still in use, which is
what makes rotation safe:
**To rotate**, issue the new key, let both work while the user switches, watch
`LAST USED` on the old one stop moving, then revoke it.
```
llmgw keys revoke --store … key-mpmip77twpt7qaxc
```
Revocation is a soft delete: the key stops working, the row stays, and its usage
history keeps its subject. A running server keeps accepting a revoked key until
its cache expires — `auth.cache_ttl`, 30 seconds by default. That is the price of
keeping the database off the request path, and it is the number to lower if you
need revocation to bite faster.
## What this does not do
- **No email.** No approval notice, no password reset, no address verification.
Someone who forgets their password needs you, and there is no command for that
yet — `llmgw users create` is the workaround, which changes the address.
- **No SSO.** Accounts are local, with local passwords.
- **No roles.** Every approved account can do the same things: make its own keys
and nothing else. Administration is the CLI, and having a shell here is what
makes someone an administrator.
- **Rate limits are per gateway instance.** With two instances each counts its own,
so the effective limit doubles. v1 is single-instance by design.