The page explained the API, said keys come from an operator, and left the reader there. Someone who reaches the address without a key had no route to getting one, which is where onboarding stopped. `landing.contact` is that route, because the gateway cannot guess who to ask. An address becomes a mailto link, an http(s) URL stays a link, and anything else — a channel, a queue, a person — is shown as text. Other schemes are never linked: this string comes from configuration, configuration is not always written by whoever runs the gateway, and a javascript: href on a page served to every visitor is a short path to a stored XSS. Escaping the text does not make a scheme safe. Unconfigured, it now says something rather than nothing, and the page explains that a key arrives as a one-time link so the reader knows what to expect before it lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5.8 KiB
Onboarding a user
Written for whoever operates an llmgw deployment: the person with a shell on the control plane machine.
Everything here runs as the user the server runs as, so the files it creates stay readable by it:
sudo -u llmgw llmgw keys <command> --store /var/lib/llmgw/llmgw.db
The --store flag is repeated in every example below. Set it once in a shell
alias if you do this often; it deliberately has no server-side default beyond
/var/lib/llmgw/llmgw.db.
1. Decide where the user belongs
Keys hang off a project, projects off a team, teams off an organization. Limits can be set at any of those levels and compose downward by taking the most restrictive value — a key cannot raise what its team set, and a team cannot raise what its organization set. That is the only rule under which a budget means anything.
If you have no structure yet, one of each is fine:
llmgw keys init --store … --org mynodes --team default --project default
It is idempotent: running it again renames rather than duplicating, so it can live in whatever provisions the machine.
Split it when you want separate limits or separate reporting, not before.
2. Issue the key as a one-time link
llmgw keys create --store … \
--project default \
--band gold \
--base-url https://llmgw.example \
--claim \
"ayse-laptop"
https://llmgw.example/claim/3mKq…
Send the link, not the key.
A key pasted into a chat window stays in that chat window forever, searchable by everyone who joins the channel later. A link works once and then expires, so the worst case is someone opening it before the intended person — which you find out about, because they will tell you the link is dead.
The name is for you, not the user: it appears in keys list and is how you will
know which key to revoke.
Why the link is safe to store. The key is sealed with the link's own token, and the token is never written down — only its hash is. A database leak yields two useless hashes and a blob nothing in the database can open.
Without --claim the key is printed to stdout instead, for when you are
provisioning something that is not a person:
llmgw keys create --store … --project ci "github-runner" > /run/secrets/llmgw-key
3. Set limits
Limits are optional. A key with none is unlimited, which is the right default on a private cluster and the wrong one the moment the gateway faces anything else.
# Everyone in this project
llmgw keys limits --store … --scope project --id default \
--rpm 60 --tpm 200000
# One key, tighter
llmgw keys limits --store … --scope key --id key-mpmip77twpt7qaxc \
--rpm 10
# Restrict which models
llmgw keys limits --store … --scope project --id default \
--models qwen-0.5b,llama-3.3-70b
An absent model list permits everything. An empty one permits nothing —
--models "" is a way to suspend a project without revoking its keys.
Allowlists compose by intersection: a key cannot add a model its team withheld. A model's canonical name and any of its aliases name the same thing, so either form works.
Budgets are counted in micro-units, and what a unit means is yours to decide.
Self-hosted models have no per-token price, so it can be GPU-seconds, a notional
rate, or 1 000 000 everywhere — which turns a budget into a plain token quota.
Models are priced in the server's auth.pricing section; a model priced at zero
is exempt from budget checks, which is what lets a key with nothing left still be
served a cheap fallback.
llmgw keys limits --store … --scope project --id default \
--budget 50000000 --period month # 50 units a month
4. Point people at the right place
Someone who reaches the gateway without a key sees the landing page. Tell it who to send them to, or it can only shrug:
landing:
contact: "platform@example.com" # or an https:// URL, or "#platform on Slack"
notice: "Internal use only."
An address becomes a mailto link, an http(s) URL stays a link, anything else is
shown as text. Other schemes are never linked — configuration is not always
written by whoever runs the gateway, and a javascript: href on a page served to
every visitor is a short path to a stored XSS.
5. What the user does
They open the link, press the button, and get the key once. The page shows them
the base URL and a working curl, and points at two endpoints:
GET /v1/models— what this key may useGET /me— its limits and what it has spent
/me is there so nobody has to ask you "why was I rate limited" or "how much
have I used". It is outside /v1 because it is not part of the OpenAI API.
6. Afterwards
llmgw keys list --store … --project default
LAST USED is written in batches rather than per request, so it lags by up to a
minute. It is the column that tells you whether a key is still in use, which is
what makes rotation safe:
To rotate, issue the new key, let both work while the user switches, watch
LAST USED on the old one stop moving, then revoke it.
llmgw keys revoke --store … key-mpmip77twpt7qaxc
Revocation is a soft delete: the key stops working, the row stays, and its usage
history keeps its subject. A running server keeps accepting a revoked key until
its cache expires — auth.cache_ttl, 30 seconds by default. That is the price of
keeping the database off the request path, and it is the number to lower if you
need revocation to bite faster.
What this does not do
- No self-service. Every key needs someone with a shell here. That is deliberate: an endpoint that mints credentials needs a login system, and this project does not have one. If you outgrow it, that is the thing to build, not a password field on the dashboard.
- No SSO. Same reason.
- Rate limits are per gateway instance. With two instances each counts its own, so the effective limit doubles. v1 is single-instance by design.