The page explained the API, said keys come from an operator, and left the reader there. Someone who reaches the address without a key had no route to getting one, which is where onboarding stopped. `landing.contact` is that route, because the gateway cannot guess who to ask. An address becomes a mailto link, an http(s) URL stays a link, and anything else — a channel, a queue, a person — is shown as text. Other schemes are never linked: this string comes from configuration, configuration is not always written by whoever runs the gateway, and a javascript: href on a page served to every visitor is a short path to a stored XSS. Escaping the text does not make a scheme safe. Unconfigured, it now says something rather than nothing, and the page explains that a key arrives as a one-time link so the reader knows what to expect before it lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
167 lines
5.8 KiB
Markdown
167 lines
5.8 KiB
Markdown
# Onboarding a user
|
|
|
|
Written for whoever operates an llmgw deployment: the person with a shell on the
|
|
control plane machine.
|
|
|
|
Everything here runs as the user the server runs as, so the files it creates stay
|
|
readable by it:
|
|
|
|
```
|
|
sudo -u llmgw llmgw keys <command> --store /var/lib/llmgw/llmgw.db
|
|
```
|
|
|
|
The `--store` flag is repeated in every example below. Set it once in a shell
|
|
alias if you do this often; it deliberately has no server-side default beyond
|
|
`/var/lib/llmgw/llmgw.db`.
|
|
|
|
## 1. Decide where the user belongs
|
|
|
|
Keys hang off a project, projects off a team, teams off an organization. Limits
|
|
can be set at any of those levels and compose downward by taking the **most
|
|
restrictive** value — a key cannot raise what its team set, and a team cannot
|
|
raise what its organization set. That is the only rule under which a budget
|
|
means anything.
|
|
|
|
If you have no structure yet, one of each is fine:
|
|
|
|
```
|
|
llmgw keys init --store … --org mynodes --team default --project default
|
|
```
|
|
|
|
It is idempotent: running it again renames rather than duplicating, so it can
|
|
live in whatever provisions the machine.
|
|
|
|
Split it when you want separate limits or separate reporting, not before.
|
|
|
|
## 2. Issue the key as a one-time link
|
|
|
|
```
|
|
llmgw keys create --store … \
|
|
--project default \
|
|
--band gold \
|
|
--base-url https://llmgw.example \
|
|
--claim \
|
|
"ayse-laptop"
|
|
|
|
https://llmgw.example/claim/3mKq…
|
|
```
|
|
|
|
Send the link, not the key.
|
|
|
|
A key pasted into a chat window stays in that chat window forever, searchable by
|
|
everyone who joins the channel later. A link works once and then expires, so the
|
|
worst case is someone opening it before the intended person — which you find out
|
|
about, because they will tell you the link is dead.
|
|
|
|
The name is for you, not the user: it appears in `keys list` and is how you will
|
|
know which key to revoke.
|
|
|
|
**Why the link is safe to store.** The key is sealed with the link's own token,
|
|
and the token is never written down — only its hash is. A database leak yields
|
|
two useless hashes and a blob nothing in the database can open.
|
|
|
|
Without `--claim` the key is printed to stdout instead, for when you are
|
|
provisioning something that is not a person:
|
|
|
|
```
|
|
llmgw keys create --store … --project ci "github-runner" > /run/secrets/llmgw-key
|
|
```
|
|
|
|
## 3. Set limits
|
|
|
|
Limits are optional. A key with none is unlimited, which is the right default on
|
|
a private cluster and the wrong one the moment the gateway faces anything else.
|
|
|
|
```
|
|
# Everyone in this project
|
|
llmgw keys limits --store … --scope project --id default \
|
|
--rpm 60 --tpm 200000
|
|
|
|
# One key, tighter
|
|
llmgw keys limits --store … --scope key --id key-mpmip77twpt7qaxc \
|
|
--rpm 10
|
|
|
|
# Restrict which models
|
|
llmgw keys limits --store … --scope project --id default \
|
|
--models qwen-0.5b,llama-3.3-70b
|
|
```
|
|
|
|
An **absent** model list permits everything. An **empty** one permits nothing —
|
|
`--models ""` is a way to suspend a project without revoking its keys.
|
|
|
|
Allowlists compose by intersection: a key cannot add a model its team withheld.
|
|
A model's canonical name and any of its aliases name the same thing, so either
|
|
form works.
|
|
|
|
Budgets are counted in micro-units, and what a unit means is yours to decide.
|
|
Self-hosted models have no per-token price, so it can be GPU-seconds, a notional
|
|
rate, or 1 000 000 everywhere — which turns a budget into a plain token quota.
|
|
Models are priced in the server's `auth.pricing` section; a model priced at zero
|
|
is exempt from budget checks, which is what lets a key with nothing left still be
|
|
served a cheap fallback.
|
|
|
|
```
|
|
llmgw keys limits --store … --scope project --id default \
|
|
--budget 50000000 --period month # 50 units a month
|
|
```
|
|
|
|
## 4. Point people at the right place
|
|
|
|
Someone who reaches the gateway without a key sees the landing page. Tell it who
|
|
to send them to, or it can only shrug:
|
|
|
|
```yaml
|
|
landing:
|
|
contact: "platform@example.com" # or an https:// URL, or "#platform on Slack"
|
|
notice: "Internal use only."
|
|
```
|
|
|
|
An address becomes a mailto link, an `http(s)` URL stays a link, anything else is
|
|
shown as text. Other schemes are never linked — configuration is not always
|
|
written by whoever runs the gateway, and a `javascript:` href on a page served to
|
|
every visitor is a short path to a stored XSS.
|
|
|
|
## 5. What the user does
|
|
|
|
They open the link, press the button, and get the key once. The page shows them
|
|
the base URL and a working `curl`, and points at two endpoints:
|
|
|
|
- `GET /v1/models` — what this key may use
|
|
- `GET /me` — its limits and what it has spent
|
|
|
|
`/me` is there so nobody has to ask you "why was I rate limited" or "how much
|
|
have I used". It is outside `/v1` because it is not part of the OpenAI API.
|
|
|
|
## 6. Afterwards
|
|
|
|
```
|
|
llmgw keys list --store … --project default
|
|
```
|
|
|
|
`LAST USED` is written in batches rather than per request, so it lags by up to a
|
|
minute. It is the column that tells you whether a key is still in use, which is
|
|
what makes rotation safe:
|
|
|
|
**To rotate**, issue the new key, let both work while the user switches, watch
|
|
`LAST USED` on the old one stop moving, then revoke it.
|
|
|
|
```
|
|
llmgw keys revoke --store … key-mpmip77twpt7qaxc
|
|
```
|
|
|
|
Revocation is a soft delete: the key stops working, the row stays, and its usage
|
|
history keeps its subject. A running server keeps accepting a revoked key until
|
|
its cache expires — `auth.cache_ttl`, 30 seconds by default. That is the price of
|
|
keeping the database off the request path, and it is the number to lower if you
|
|
need revocation to bite faster.
|
|
|
|
## What this does not do
|
|
|
|
- **No self-service.** Every key needs someone with a shell here. That is
|
|
deliberate: an endpoint that mints credentials needs a login system, and this
|
|
project does not have one. If you outgrow it, that is the thing to build, not a
|
|
password field on the dashboard.
|
|
- **No SSO.** Same reason.
|
|
- **Rate limits are per gateway instance.** With two instances each counts its own,
|
|
so the effective limit doubles. v1 is single-instance by design.
|