Files
llmgw/docs/onboarding.md
T
Ilker ManapandClaude Opus 5 8d35faf6f8
CI / test (push) Successful in 56s
Give the landing page somewhere to send people
The page explained the API, said keys come from an operator, and left the
reader there. Someone who reaches the address without a key had no route to
getting one, which is where onboarding stopped.

`landing.contact` is that route, because the gateway cannot guess who to ask.
An address becomes a mailto link, an http(s) URL stays a link, and anything
else — a channel, a queue, a person — is shown as text. Other schemes are
never linked: this string comes from configuration, configuration is not
always written by whoever runs the gateway, and a javascript: href on a page
served to every visitor is a short path to a stored XSS. Escaping the text
does not make a scheme safe.

Unconfigured, it now says something rather than nothing, and the page explains
that a key arrives as a one-time link so the reader knows what to expect
before it lands.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 09:22:42 +02:00

5.8 KiB

Onboarding a user

Written for whoever operates an llmgw deployment: the person with a shell on the control plane machine.

Everything here runs as the user the server runs as, so the files it creates stay readable by it:

sudo -u llmgw llmgw keys <command> --store /var/lib/llmgw/llmgw.db

The --store flag is repeated in every example below. Set it once in a shell alias if you do this often; it deliberately has no server-side default beyond /var/lib/llmgw/llmgw.db.

1. Decide where the user belongs

Keys hang off a project, projects off a team, teams off an organization. Limits can be set at any of those levels and compose downward by taking the most restrictive value — a key cannot raise what its team set, and a team cannot raise what its organization set. That is the only rule under which a budget means anything.

If you have no structure yet, one of each is fine:

llmgw keys init --store … --org mynodes --team default --project default

It is idempotent: running it again renames rather than duplicating, so it can live in whatever provisions the machine.

Split it when you want separate limits or separate reporting, not before.

llmgw keys create --store … \
  --project default \
  --band gold \
  --base-url https://llmgw.example \
  --claim \
  "ayse-laptop"

https://llmgw.example/claim/3mKq…

Send the link, not the key.

A key pasted into a chat window stays in that chat window forever, searchable by everyone who joins the channel later. A link works once and then expires, so the worst case is someone opening it before the intended person — which you find out about, because they will tell you the link is dead.

The name is for you, not the user: it appears in keys list and is how you will know which key to revoke.

Why the link is safe to store. The key is sealed with the link's own token, and the token is never written down — only its hash is. A database leak yields two useless hashes and a blob nothing in the database can open.

Without --claim the key is printed to stdout instead, for when you are provisioning something that is not a person:

llmgw keys create --store … --project ci "github-runner" > /run/secrets/llmgw-key

3. Set limits

Limits are optional. A key with none is unlimited, which is the right default on a private cluster and the wrong one the moment the gateway faces anything else.

# Everyone in this project
llmgw keys limits --store … --scope project --id default \
  --rpm 60 --tpm 200000

# One key, tighter
llmgw keys limits --store … --scope key --id key-mpmip77twpt7qaxc \
  --rpm 10

# Restrict which models
llmgw keys limits --store … --scope project --id default \
  --models qwen-0.5b,llama-3.3-70b

An absent model list permits everything. An empty one permits nothing — --models "" is a way to suspend a project without revoking its keys.

Allowlists compose by intersection: a key cannot add a model its team withheld. A model's canonical name and any of its aliases name the same thing, so either form works.

Budgets are counted in micro-units, and what a unit means is yours to decide. Self-hosted models have no per-token price, so it can be GPU-seconds, a notional rate, or 1 000 000 everywhere — which turns a budget into a plain token quota. Models are priced in the server's auth.pricing section; a model priced at zero is exempt from budget checks, which is what lets a key with nothing left still be served a cheap fallback.

llmgw keys limits --store … --scope project --id default \
  --budget 50000000 --period month        # 50 units a month

4. Point people at the right place

Someone who reaches the gateway without a key sees the landing page. Tell it who to send them to, or it can only shrug:

landing:
  contact: "platform@example.com"   # or an https:// URL, or "#platform on Slack"
  notice: "Internal use only."

An address becomes a mailto link, an http(s) URL stays a link, anything else is shown as text. Other schemes are never linked — configuration is not always written by whoever runs the gateway, and a javascript: href on a page served to every visitor is a short path to a stored XSS.

5. What the user does

They open the link, press the button, and get the key once. The page shows them the base URL and a working curl, and points at two endpoints:

  • GET /v1/models — what this key may use
  • GET /me — its limits and what it has spent

/me is there so nobody has to ask you "why was I rate limited" or "how much have I used". It is outside /v1 because it is not part of the OpenAI API.

6. Afterwards

llmgw keys list --store … --project default

LAST USED is written in batches rather than per request, so it lags by up to a minute. It is the column that tells you whether a key is still in use, which is what makes rotation safe:

To rotate, issue the new key, let both work while the user switches, watch LAST USED on the old one stop moving, then revoke it.

llmgw keys revoke --store … key-mpmip77twpt7qaxc

Revocation is a soft delete: the key stops working, the row stays, and its usage history keeps its subject. A running server keeps accepting a revoked key until its cache expires — auth.cache_ttl, 30 seconds by default. That is the price of keeping the database off the request path, and it is the number to lower if you need revocation to bite faster.

What this does not do

  • No self-service. Every key needs someone with a shell here. That is deliberate: an endpoint that mints credentials needs a login system, and this project does not have one. If you outgrow it, that is the thing to build, not a password field on the dashboard.
  • No SSO. Same reason.
  • Rate limits are per gateway instance. With two instances each counts its own, so the effective limit doubles. v1 is single-instance by design.