llmgw/docs/onboarding.md

# Onboarding a user

Written for whoever operates an llmgw deployment: the person with a shell on the
control plane machine.

There are two ways a person gets a key. Pick one and put it in the config.

| | Self-service | Operator-issued |
|---|---|---|
| Config | `auth.signup_open: true` | `auth.signup_open: false` |
| They do | sign up, wait, then make their own keys | wait for a link |
| You do | approve once | issue every key |
| Good for | more than a handful of people | a few, or machines |

Self-service does not mean open. A signup produces an account that can do
nothing at all; **approval is what makes it work**, and approval needs a shell
here rather than a session cookie, because it is the step that decides who gets
to spend the cluster.

Everything below runs as the user the server runs as, so the files it creates
stay readable by it:

```
sudo -u llmgw llmgw <command> --store /var/lib/llmgw/llmgw.db
```

## Self-service

Turn on the form:

```yaml
auth:
  enabled: true
  store: /var/lib/llmgw/llmgw.db
  signup_open: true
```

The landing page then links to it, and the rest is:

```
llmgw users list --pending --store …
ID          EMAIL              STATE    ADMIN  MEMBER OF     SIGNED UP
usr-k4mq…   ayse@example.com   pending  -      on their own  4m12s ago

llmgw users approve ayse@example.com --by ilker --store …
llmgw users approve ayse@example.com --project Acme/ml/training --store …   # or straight into a project
```

That is the whole operator loop. From then on they sign in at `/login`, and
their account page (`/profile`) is where they create keys, see when each was
last used, revoke the ones they are done with, and see their own use over the
last thirty days against every budget that applies to them. A key is shown once
on the page that creates it and never again.

### Connecting an editor

The account page's **Connect your editor** makes a key for Continue, Codex or
Cline and shows what the tool needs with that key, this gateway's address and a
model already in it:

- **Continue** — an assistant file of its own, `~/.continue/assistants/llmgw.yaml`,
  so their `config.yaml` is not touched.
- **Codex** — a provider and a profile in `~/.codex/config.toml`, run with
  `codex --profile llmgw`; their default model stays as it was. Codex speaks the
  Responses API, which the gateway translates at `/v1/responses`.
- **Cline** — four settings to paste into its own screen, since it keeps them
  in VS Code's secret storage where no file reaches.

For Continue and Codex there is a one-line command,
`curl -fsSL <gateway>/setup/<tool>?model=… | sh`. Its script is served without
signing in and holds no secret — the key comes from the environment — and it
keeps a copy of anything it replaces. Revoking the key disconnects the tool.

How well an editor's agent mode works depends on the model: it needs one that
calls tools well, which the coder models do and small general ones often do not.

**This gateway sends no email.** There is no "your account was approved" message
and no password reset, so tell people what to expect: they sign up, they tell
you, you approve, they sign in. If that loop is too manual for your size, it is
the thing to automate — not the approval step.

To close an account:

```
llmgw users suspend usr-k4mq… --store …
```

which closes their sessions immediately. Their **API keys keep working** — those
are separate credentials, and revoking them is a separate step:

```
llmgw keys list --project default --store …
llmgw keys revoke key-… --store …
```

That separation is deliberate: a suspended person's running jobs should not all
fail at once without someone deciding they should.

## Operator-issued

With `signup_open: false` there is no form, and you issue keys yourself. This is
also the path for things that are not people — CI, a notebook server, a script.

### Decide where the key belongs

Keys hang off a project, projects off a team, teams off an organization. Limits
can be set at any of those levels and compose downward by taking the **most
restrictive** value — a key cannot raise what its team set, and a team cannot
raise what its organization set. That is the only rule under which a budget
means anything.

If you have no structure yet, one of each is fine:

```
llmgw org add org mynodes
llmgw org add team default --in mynodes
llmgw org add project default --in mynodes/default
llmgw org tree --people
```

Everything that takes an organization, team or project — here, in `users` and
in `keys` — accepts its id, its path (`mynodes/default/default`) or its name
when the name is unique, so an id never has to be copied off a page. When there
is only one project, `--project` can be left out.

The tree is not fixed once made:

```
llmgw org rename project Acme/platform/gateway api-gateway
llmgw org move project Acme/platform/gateway --to Scania/ml
llmgw org remove team Acme/old                 # only when nothing is in it
llmgw users delete ayse@example.com --yes      # signs out, revokes their keys
llmgw keys list --all                          # every key, where it spends, and whose
```

A project that ever had a key cannot be removed, because usage history refers
to it — rename it instead.

### Where people belong

A person can be a member of any number of organizations, teams and projects, or
of none:

```
llmgw users join ayse@example.com --team Acme/ml          # every project in ml
llmgw users join ayse@example.com --project Scania/research/vision
llmgw users leave ayse@example.com --project Acme/ml/training
```

A member of a team or organization may use every project in it. Leaving revokes
the keys the person made in projects they can no longer reach; a project they
still reach through another membership keeps them.

Someone who is a member of nothing still uses the gateway, on their own:
personal keys, and the chat page, within their personal limits. When they make
a key on `/profile` they choose what it spends against — one of their projects,
or "Personal". An operator can do the same:

```
llmgw keys create laptop --for ayse@example.com                      # personal
llmgw keys create ci --for ayse@example.com --project Acme/ml/training
```

Personal use can be limited by default, or turned off so people use the gateway
only through their projects:

```yaml
auth:
  personal:
    rpm: 20
    tpm: 50000
    budget: 10        # whole units
    period: month
    # disabled: true
```

A personal key has no project, so it keeps no conversations or agent notes;
those are kept per project.

### Organization managers

A member of an organization can be made its manager:

```
llmgw users join boss@acme.com --org Acme --role manager
```

or with the star on their organization membership on `/admin/users`. A manager
gets an **Organization** link and a page at `/org`: usage over time by team,
project or person, the teams and projects with their limits and spend, and every
member. Only what was spent in that organization's projects is on it; what its
people do elsewhere is not. The page is read-only — a manager is not an
administrator of the cluster, and limits and memberships stay with those who
are. An administrator following `/org` lands on the same organization's panel
on the people page instead, where all of that can be changed.

`llmgw keys init` still works for provisioning scripts and is idempotent, but it
will no longer move a team or project that already exists under another parent;
it says so and stops.

Split it when you want separate limits or separate reporting, not before.

### Issue the key as a one-time link

```
llmgw keys create --store … \
  --project default \
  --band gold \
  --base-url https://llmgw.example \
  --claim \
  "ayse-laptop"

https://llmgw.example/claim/3mKq…
```

Send the link, not the key.

A key pasted into a chat window stays in that chat window forever, searchable by
everyone who joins the channel later. A link works once and then expires, so the
worst case is someone opening it before the intended person — which you find out
about, because they will tell you the link is dead.

The name is for you, not the user: it appears in `keys list` and is how you will
know which key to revoke.

**Why the link is safe to store.** The key is sealed with the link's own token,
and the token is never written down — only its hash is. A database leak yields
two useless hashes and a blob nothing in the database can open.

Without `--claim` the key is printed to stdout instead, for when you are
provisioning something that is not a person:

```
llmgw keys create --store … --project ci "github-runner" > /run/secrets/llmgw-key
```

## Limits

Limits are optional. A key with none is unlimited, which is the right default on
a private cluster and the wrong one the moment the gateway faces anything else.

```
# Everyone in this project
llmgw keys limits --store … --scope project --id default \
  --rpm 60 --tpm 200000

# One key, tighter
llmgw keys limits --store … --scope key --id key-mpmip77twpt7qaxc \
  --rpm 10

# Restrict which models
llmgw keys limits --store … --scope project --id default \
  --models qwen-0.5b,llama-3.3-70b
```

An **absent** model list permits everything. An **empty** one permits nothing —
`--models ""` is a way to suspend a project without revoking its keys.

Allowlists compose by intersection: a key cannot add a model its team withheld.
A model's canonical name and any of its aliases name the same thing, so either
form works.

Budgets are given in whole units, as the Limits tab shows them, and what a unit
means is yours to decide. Self-hosted models have no per-token price, so it can
be GPU-seconds, a notional rate, or one unit per thousand tokens everywhere —
a price of 1 000 000 in `auth.pricing`, which is in micro-units — and a budget
becomes a plain quota of thousands of tokens.
Models are priced in the server's `auth.pricing` section; a model priced at zero
is exempt from budget checks, which is what lets a key with nothing left still be
served a cheap fallback.

```
llmgw keys limits --store … --scope project --id Acme/platform/gateway \
  --budget 50 --period month              # 50 units a month
```

Two more kinds of limit follow people rather than the tree:

```
# A person, everywhere: every key they hold, every project, the chat page
llmgw keys limits --scope user --id ayse@example.com --budget 25

# Each member of a project separately — a fair share, not one shared pool
llmgw keys limits --scope project --id Acme/ml/training --per-member --rpm 20
```

A request is held to all of them at once: the key's own, its project's, team's
and organization's, each per-member limit on those for its owner, and the
owner's own. The strictest wins, and each is counted in its own pool.

## The landing page

Someone who reaches the gateway without a key sees the landing page. Tell it who
to send them to, or it can only shrug:

```yaml
landing:
  contact: "platform@example.com"   # or an https:// URL, or "#platform on Slack"
  notice: "Internal use only."
```

An address becomes a mailto link, an `http(s)` URL stays a link, anything else is
shown as text. Other schemes are never linked — configuration is not always
written by whoever runs the gateway, and a `javascript:` href on a page served to
every visitor is a short path to a stored XSS.

## The docs portal

The gateway also serves its manual at `/docs`: these pages, rendered, with
[clients.md](clients.md) — how to use the gateway from the official OpenAI
and Anthropic SDKs — first on its index. The landing page and the account
page link to it, so a user's questions about streaming, tools or the Anthropic
SDK have an answer that is not you.

It is public, like the landing page. The client guide is for the person who
has no account yet, and the operator documents describe llmgw in general —
no address, model, node or key of this installation — which is what the
project's repository publishes anyway. A site that would rather not say even
that turns sign-in on:

```yaml
landing:
  docs_sign_in: true
```

A visitor is then sent to sign in first. On a gateway with API keys and no
accounts, the portal asks for a key instead, like the API.

## What the user sees

They open the link, press the button, and get the key once. The page shows them
the base URL and a working `curl`, and points at two endpoints:

- `GET /v1/models` — what this key may use
- `GET /me` — its limits and what it has spent

`/me` is there so nobody has to ask you "why was I rate limited" or "how much
have I used". It is outside `/v1` because it is not part of the OpenAI API.

## Rotation and revocation

```
llmgw keys list --store … --project default
```

`LAST USED` is written in batches rather than per request, so it lags by up to a
minute. It is the column that tells you whether a key is still in use, which is
what makes rotation safe:

**To rotate**, `llmgw keys rotate <key-id> --overlap 48h` issues a new key with
the same project, owner and limits and lets the old one work for the overlap,
then stop. Watch `LAST USED` on the old one stop moving before it runs out.

**To cap how long keys live**, `llmgw keys ttl project:Acme/ml/train 720h` (or
`team:`, `org:`; `off` removes it), and `auth.max_key_ttl` in server.yaml for
every key. A key is made within the shortest limit that applies, and one asked
for without an expiry gets that limit. `keys list --all` shows each key's kind
— service (a project's), member, personal — and when it expires.

```
llmgw keys revoke --store … key-mpmip77twpt7qaxc
```

Revocation is a soft delete: the key stops working, the row stays, and its usage
history keeps its subject. A running server keeps accepting a revoked key until
its cache expires — `auth.cache_ttl`, 30 seconds by default. That is the price of
keeping the database off the request path, and it is the number to lower if you
need revocation to bite faster.

## Passwords

People change their own from `/profile`. It asks for the current one — a session
is what someone gets from a borrowed laptop, and the password is what they do not
have — and it signs every other browser out, which is the point if they are
changing it because something went wrong.

There is no self-service reset, because this gateway sends no email and a reset
link with nowhere to send it is not a reset. When someone loses theirs:

```
llmgw users reset-password usr-k4mq… --password "a temporary passphrase" --store …
```

Tell them out of band and have them change it from their profile. Their sessions
are closed — if they lost the password, the reason might be that someone else has
it. Their API keys keep working; those are separate credentials.

## Trying a model in the browser

`/chat` is a playground for anyone signed in. It exists so somebody can see
whether a model works without setting up a client, and — more often — so they
can tell whether a problem is the gateway or their code.

**The browser never holds a key.** The page carries a session; the server turns
that into the same identity an API key would produce, resolved from the person's
project, and the request goes down the ordinary path: admission, model
allowlist, rate limits, budget, accounting. Anything else would make the page a
hole in every limit the rest of the gateway enforces.

Usage from the page is attributed to `session:<user-id>` rather than a key, so
it is billed to the right project and visibly did not come from a credential
somebody holds. One turn is capped at 512 tokens: a playground should not be the
thing that ties up a replica for ten minutes, and anyone who wants a longer
answer has an API key for that.

Nothing is stored. The conversation lives in the page and is gone when it is
closed.

## Seeing the cluster

The dashboard — nodes, replicas, models, why something did not get placed, usage
per key — is served twice, and the difference matters.

On the **admin listener** (`127.0.0.1:9090` by default) it needs no
authentication: having a shell on the machine is the credential.

```
ssh -L 9090:127.0.0.1:9090 <control-plane>   # then http://127.0.0.1:9090/
```

That listener also serves pprof, which can dump the process heap — where API
keys sit in plaintext. **Never expose that port.** It is bound to loopback for
that reason.

On the **client-facing port** the same page is at `/admin`, behind a session and
an admin flag, so it can be reached from a browser without the port carrying
pprof along with it:

```
llmgw users promote usr-k4mq… --store …
```

Someone without the flag gets a 404 rather than a 403, because 403 confirms the
page exists.

## Managing accounts from the browser

`/admin/users` is the tree: organizations, teams and projects, each project
opening onto its members and its keys. From there an admin approves people
(into a project or on their own), adds them to and takes them out of
organizations, teams and projects, suspends and removes them, revokes keys, and
adds, renames, moves and removes organizations, teams and projects. A
searchable list of every account is at the bottom. Choosing any organization,
team, project or person opens a panel beside the tree with its limits, its
capacity quota and its use over the last week, and its limits can be changed
there.

The same page has two more tabs, each at the address it had as a page of its
own: **Limits** (`/admin/quotas`) is every limit in the tree at once — per
branch, per member of any part of it, and per person — with the capacity quotas;
**Usage** (`/admin/usage`) is the cluster's use over time by organization, team,
project or person, and the GPU time each owner held. The page works as plain
forms; with scripts on, a change updates it in place and the projects you had
open stay open.

Three things are deliberately not there:

| | Why |
|---|---|
| `llmgw users promote` | An admin who can make admins turns a stolen session into permanent access |
| `llmgw users reset-password` | Setting someone's password is taking over their account |
| `llmgw keys create` | Issuing a credential on another person's behalf |

Each of those turns a stolen admin cookie into more than it was, and each is
rare enough that a shell is no hardship. Cordon and drain are CLI-only for the
same reason; what the pages do change on the cluster — serving models through
the mirror, a node's engines — needs the `operator` or `admin` staff role.

Every administrative change is recorded in the audit log — who, when, through
what, the fields given with secrets taken out, and how it ended, refusals
included: `llmgw ctl audit`, or **Audit** in the console.

## What this does not do

- **No email.** No approval notice, no self-service password reset, no address
  verification. The loop is manual on purpose at this size; if it stops being
  the right size, that is the thing to automate.
- **Single sign-on is optional.** With `auth.oidc` people sign in through
  your identity provider, and SCIM 2.0 (`auth.scim`) creates, updates and
  suspends accounts from it; see [install.md](install.md#single-sign-on).
  Without them accounts are local, with local passwords.
- **Staff roles are coarse.** `viewer`, `operator`, `billing` and `admin`
  (`llmgw users role`) decide what someone may see and do under /admin; an
  organization's manager can view it but not change it. Every change is in
  the audit log (`llmgw ctl audit`).
- **Rate limits are per gateway instance.** With two instances each counts its own,
  so the effective limit doubles. v1 is single-instance by design.