Ilker ManapandClaude Opus 5
3ed211b2f4
Issuing a key worked; getting it to the person it was issued for did not. The operator had a secret on their terminal and no way to send it that did not leave a bearer credential in a chat log forever. `keys create --claim` prints a link instead. GET shows a button and consumes nothing; POST reveals the key and burns the link. That split is not politeness — chat clients, mail scanners and link previewers fetch URLs the moment they are posted, so a GET that consumed the claim would mean the first thing to unfurl the link destroys the key before anyone reads it. A pending claim has to hold the key somewhere, which would otherwise put a working credential in the database and undo what hashing bought. So the key is sealed with a key derived from the link's own token, and the token is never stored — only its hash. A database leak yields two useless hashes and a blob nothing in the database can open. HKDF rather than the raw token, so the value used for lookup can never become the value that decrypts. Consuming is one UPDATE moving claimed_at off NULL, so two people opening the same link produce one key rather than two copies. A used link, an expired one and a token that was never real all render identically, because saying which confirms that a token was once valid. The page is no-store, no-referrer and noindex: the URL is itself the credential. `GET /me` reports a key's own limits and spend, so nobody has to ask an operator to read a dashboard they cannot reach. Outside /v1, because it is not part of the OpenAI API. docs/onboarding.md is written for whoever operates this, including the parts that are deliberately absent — no self-service, no SSO, and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5.3 KiB
Onboarding a user
Written for whoever operates an llmgw deployment: the person with a shell on the control plane machine.
Everything here runs as the user the server runs as, so the files it creates stay readable by it:
sudo -u llmgw llmgw keys <command> --store /var/lib/llmgw/llmgw.db
The --store flag is repeated in every example below. Set it once in a shell
alias if you do this often; it deliberately has no server-side default beyond
/var/lib/llmgw/llmgw.db.
1. Decide where the user belongs
Keys hang off a project, projects off a team, teams off an organization. Limits can be set at any of those levels and compose downward by taking the most restrictive value — a key cannot raise what its team set, and a team cannot raise what its organization set. That is the only rule under which a budget means anything.
If you have no structure yet, one of each is fine:
llmgw keys init --store … --org mynodes --team default --project default
It is idempotent: running it again renames rather than duplicating, so it can live in whatever provisions the machine.
Split it when you want separate limits or separate reporting, not before.
2. Issue the key as a one-time link
llmgw keys create --store … \
--project default \
--band gold \
--base-url https://llmgw.example \
--claim \
"ayse-laptop"
https://llmgw.example/claim/3mKq…
Send the link, not the key.
A key pasted into a chat window stays in that chat window forever, searchable by everyone who joins the channel later. A link works once and then expires, so the worst case is someone opening it before the intended person — which you find out about, because they will tell you the link is dead.
The name is for you, not the user: it appears in keys list and is how you will
know which key to revoke.
Why the link is safe to store. The key is sealed with the link's own token, and the token is never written down — only its hash is. A database leak yields two useless hashes and a blob nothing in the database can open.
Without --claim the key is printed to stdout instead, for when you are
provisioning something that is not a person:
llmgw keys create --store … --project ci "github-runner" > /run/secrets/llmgw-key
3. Set limits
Limits are optional. A key with none is unlimited, which is the right default on a private cluster and the wrong one the moment the gateway faces anything else.
# Everyone in this project
llmgw keys limits --store … --scope project --id default \
--rpm 60 --tpm 200000
# One key, tighter
llmgw keys limits --store … --scope key --id key-mpmip77twpt7qaxc \
--rpm 10
# Restrict which models
llmgw keys limits --store … --scope project --id default \
--models qwen-0.5b,llama-3.3-70b
An absent model list permits everything. An empty one permits nothing —
--models "" is a way to suspend a project without revoking its keys.
Allowlists compose by intersection: a key cannot add a model its team withheld. A model's canonical name and any of its aliases name the same thing, so either form works.
Budgets are counted in micro-units, and what a unit means is yours to decide.
Self-hosted models have no per-token price, so it can be GPU-seconds, a notional
rate, or 1 000 000 everywhere — which turns a budget into a plain token quota.
Models are priced in the server's auth.pricing section; a model priced at zero
is exempt from budget checks, which is what lets a key with nothing left still be
served a cheap fallback.
llmgw keys limits --store … --scope project --id default \
--budget 50000000 --period month # 50 units a month
4. What the user does
They open the link, press the button, and get the key once. The page shows them
the base URL and a working curl, and points at two endpoints:
GET /v1/models— what this key may useGET /me— its limits and what it has spent
/me is there so nobody has to ask you "why was I rate limited" or "how much
have I used". It is outside /v1 because it is not part of the OpenAI API.
5. Afterwards
llmgw keys list --store … --project default
LAST USED is written in batches rather than per request, so it lags by up to a
minute. It is the column that tells you whether a key is still in use, which is
what makes rotation safe:
To rotate, issue the new key, let both work while the user switches, watch
LAST USED on the old one stop moving, then revoke it.
llmgw keys revoke --store … key-mpmip77twpt7qaxc
Revocation is a soft delete: the key stops working, the row stays, and its usage
history keeps its subject. A running server keeps accepting a revoked key until
its cache expires — auth.cache_ttl, 30 seconds by default. That is the price of
keeping the database off the request path, and it is the number to lower if you
need revocation to bite faster.
What this does not do
- No self-service. Every key needs someone with a shell here. That is deliberate: an endpoint that mints credentials needs a login system, and this project does not have one. If you outgrow it, that is the thing to build, not a password field on the dashboard.
- No SSO. Same reason.
- Rate limits are per gateway instance. With two instances each counts its own, so the effective limit doubles. v1 is single-instance by design.