llmgw/docs/admin-api.md

# The admin API

Written for whoever automates a cluster rather than typing at it: how to reach
the control plane's management API, what it asks of a request, and where its
description is.

`llmgw ctl` and the console are clients of this API and nothing more, so
anything they do can be done with `curl`, a generated client or the
[Terraform provider](../terraform-provider-llmgw/README.md). It adds and
removes models, scales them, cordons and upgrades nodes, sets capacity
quotas, attaches guardrail policies, and reads the fleet, usage and the audit
log.

## Reaching it

The API is under `/api/v1` on the admin listener, `admin_listen` in
`/etc/llmgw/server.yaml`: `127.0.0.1:9090` unless changed. It is on loopback
on purpose. The same listener serves pprof when it is turned on, and the
server refuses to start with it on any other address unless
`admin_remote: true` — the token would cross the network in clear.

From another machine, tunnel to it rather than moving it:

```sh
ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
curl -s -H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" http://127.0.0.1:9090/api/v1/summary
```

With `admin_remote: true` and `admin_listen: 10.0.0.5:9090`, call that address
directly — on a network you trust, or behind a TLS proxy that keeps the
`Host` header an address.

## The token

Every path under `/api` needs the admin token as a bearer token:

```
Authorization: Bearer 6f1c…
```

The server writes it the first time it starts, 0600, to
`/var/lib/llmgw/admin-token` under its systemd unit (`admin_token_file`
moves it; a server run by hand keeps it in `secrets/admin-token` beside its
configuration). `llmgw ctl` reads it from there, which is why it runs with
sudo, or from `LLMGW_ADMIN_TOKEN`. To give it to automation:

```sh
sudo cat /var/lib/llmgw/admin-token
```

The token is the whole of the API's authorisation: whoever holds it can do
everything the API does. Keep it where you keep other root credentials.

## What a request must look like

Besides the token, the listener refuses three things before a handler sees
them, as defences against a browser being turned against it:

- **A name in `Host`.** The listener answers to an address or `localhost`,
  not a DNS name, so a page on some other site cannot reach it by rebinding
  a name to 127.0.0.1. Call it as `http://127.0.0.1:9090`, not
  `http://cp.example.com:9090`; behind a proxy, have the proxy send an
  address.
- **A cross-site request.** One whose `Origin` is another host, or whose
  `Sec-Fetch-Site` says another site sent it.
- **A change that is not JSON.** Every method but `GET` and `HEAD` must carry
  `Content-Type: application/json`, even a `DELETE` or a `POST` with no body.
  A form post from a page cannot set that header.

A change is recorded in the audit log with who made it, as the request says
in `X-Llmgw-Actor` (letters, digits and `._-@`, at most 64; `unknown` when
absent). `llmgw ctl` sends the name of whoever ran it; automation should send
its own, `terraform` or `ci@deploy`, so `llmgw ctl audit` can tell them apart.

A refusal is a 4xx or 5xx with `{"error": "…"}` saying why. 501 means this
control plane lacks what the endpoint needs — a store, a mirror, guard
policies — and says which.

## Some calls

```sh
H=(-H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" -H "Content-Type: application/json" -H "X-Llmgw-Actor: $USER")
API=http://127.0.0.1:9090/api/v1

# The fleet in one answer, and every model.
curl -s "${H[@]}" $API/summary
curl -s "${H[@]}" $API/models

# Add a model, read back its definition with the defaults filled in, scale it.
curl -s "${H[@]}" -X POST $API/models -d '{
  "name": "qwen-small",
  "source": "/srv/models/qwen2.5-1.5b-instruct-q4_k_m.gguf",
  "profiles": [{"name": "cpu", "engine": "llamacpp", "min_memory_gb": 4, "cpu_cores": 4}],
  "scaling": {"min": 1, "max": 2}
}'
curl -s "${H[@]}" $API/models/qwen-small
curl -s "${H[@]}" -X POST $API/models/qwen-small/scale -d '{"min": 0, "max": 2, "idle_timeout": 1800000000000}'

# A team's capacity quota.
curl -s "${H[@]}" -X PUT $API/quotas -d '{"scope": "team:Acme/ml", "limit": {"gpus": 8}}'

# Remove the model; a DELETE carries the content type too.
curl -s "${H[@]}" -X DELETE $API/models/qwen-small
```

Durations in bodies are integer nanoseconds, as Go encodes them: half an hour
is `1800000000000`.

## The description

The API is described in OpenAPI 3.1:

- served by every control plane at `GET /api/v1/openapi.json` (with the
  token, like the rest), as built into that binary, so it describes the
  server you are talking to;
- committed as [admin-api.openapi.json](admin-api.openapi.json) for reading
  without a cluster.

```sh
curl -s "${H[@]}" $API/openapi.json > llmgw-admin.json
```

It lists every endpoint, its path and query parameters, and the schema of
each request and answer, with the meaning of each field. Load it into any
OpenAPI tool — Swagger UI, Redoc, an editor — or generate a client from it:

```sh
docker run --rm -v "$PWD:/w" openapitools/openapi-generator-cli generate \
  -i /w/llmgw-admin.json -g python -o /w/llmgw-admin-client
```

### How it is kept true

The document is generated, not written. `internal/control/adminapi/routes.go`
holds a table of every endpoint: its method and path, a summary, its query
parameters, and Go values of the types it reads and answers with. The
generator in `internal/docgen/openapi` reads those types by reflection, the
way `encoding/json` encodes them, and takes each field's description from its
comment in the source.

Two tests keep it from drifting. One registers the API on a recording mux and
fails when an endpoint is served and not in the table, or in the table and not
served. The other regenerates the document and fails when it differs from the
committed one. After changing an endpoint or a type it carries:

```sh
go generate ./internal/docgen
```

## What it does not cover

- **Organizations, teams, projects, people and API keys.** These live in the
  store and are managed with `llmgw org`, `llmgw users` and `llmgw keys` on the
  control plane, or in the web console's `/admin` pages on the client port
  behind a session. The admin API reads the tree only where a quota or a
  policy names a part of it.
- **The client-facing API** (`/v1/chat/completions` and the rest), which is
  OpenAI's and Anthropic's, on `listen`, with API keys.
- **The rest of the admin listener**: `/metrics`, `/health`, the dashboard
  pages and `/debug/pprof`, which are not under `/api`.