llmgw/docs/admin-api.md

The admin API

Written for whoever automates a cluster rather than typing at it: how to reach the control plane's management API, what it asks of a request, and where its description is.

llmgw ctl and the console are clients of this API and nothing more, so anything they do can be done with curl, a generated client or the Terraform provider. It adds and removes models, scales them, cordons and upgrades nodes, sets capacity quotas, attaches guardrail policies, and reads the fleet, usage and the audit log.

Reaching it

The API is under /api/v1 on the admin listener, admin_listen in /etc/llmgw/server.yaml: 127.0.0.1:9090 unless changed. It is on loopback on purpose. The same listener serves pprof when it is turned on, and the server refuses to start with it on any other address unless admin_remote: true — the token would cross the network in clear.

From another machine, tunnel to it rather than moving it:

ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
curl -s -H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" http://127.0.0.1:9090/api/v1/summary

With admin_remote: true and admin_listen: 10.0.0.5:9090, call that address directly — on a network you trust, or behind a TLS proxy that keeps the Host header an address.

The token

Every path under /api needs the admin token as a bearer token:

Authorization: Bearer 6f1c…

The server writes it the first time it starts, 0600, to /var/lib/llmgw/admin-token under its systemd unit (admin_token_file moves it; a server run by hand keeps it in secrets/admin-token beside its configuration). llmgw ctl reads it from there, which is why it runs with sudo, or from LLMGW_ADMIN_TOKEN. To give it to automation:

sudo cat /var/lib/llmgw/admin-token

The token is the whole of the API's authorisation: whoever holds it can do everything the API does. Keep it where you keep other root credentials.

What a request must look like

Besides the token, the listener refuses three things before a handler sees them, as defences against a browser being turned against it:

• A name in Host. The listener answers to an address or localhost, not a DNS name, so a page on some other site cannot reach it by rebinding a name to 127.0.0.1. Call it as http://127.0.0.1:9090, not http://cp.example.com:9090; behind a proxy, have the proxy send an address.
• A cross-site request. One whose Origin is another host, or whose Sec-Fetch-Site says another site sent it.
• A change that is not JSON. Every method but GET and HEAD must carry Content-Type: application/json, even a DELETE or a POST with no body. A form post from a page cannot set that header.

A change is recorded in the audit log with who made it, as the request says in X-Llmgw-Actor (letters, digits and ._-@, at most 64; unknown when absent). llmgw ctl sends the name of whoever ran it; automation should send its own, terraform or ci@deploy, so llmgw ctl audit can tell them apart.

A refusal is a 4xx or 5xx with {"error": "…"} saying why. 501 means this control plane lacks what the endpoint needs — a store, a mirror, guard policies — and says which.

Some calls

H=(-H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" -H "Content-Type: application/json" -H "X-Llmgw-Actor: $USER")
API=http://127.0.0.1:9090/api/v1

# The fleet in one answer, and every model.
curl -s "${H[@]}" $API/summary
curl -s "${H[@]}" $API/models

# Add a model, read back its definition with the defaults filled in, scale it.
curl -s "${H[@]}" -X POST $API/models -d '{
  "name": "qwen-small",
  "source": "/srv/models/qwen2.5-1.5b-instruct-q4_k_m.gguf",
  "profiles": [{"name": "cpu", "engine": "llamacpp", "min_memory_gb": 4, "cpu_cores": 4}],
  "scaling": {"min": 1, "max": 2}
}'
curl -s "${H[@]}" $API/models/qwen-small
curl -s "${H[@]}" -X POST $API/models/qwen-small/scale -d '{"min": 0, "max": 2, "idle_timeout": 1800000000000}'

# A team's capacity quota.
curl -s "${H[@]}" -X PUT $API/quotas -d '{"scope": "team:Acme/ml", "limit": {"gpus": 8}}'

# Remove the model; a DELETE carries the content type too.
curl -s "${H[@]}" -X DELETE $API/models/qwen-small

Durations in bodies are integer nanoseconds, as Go encodes them: half an hour is 1800000000000.

The description

The API is described in OpenAPI 3.1:

• served by every control plane at GET /api/v1/openapi.json (with the token, like the rest), as built into that binary, so it describes the server you are talking to;
• committed as admin-api.openapi.json for reading without a cluster.

curl -s "${H[@]}" $API/openapi.json > llmgw-admin.json

It lists every endpoint, its path and query parameters, and the schema of each request and answer, with the meaning of each field. Load it into any OpenAPI tool — Swagger UI, Redoc, an editor — or generate a client from it:

docker run --rm -v "$PWD:/w" openapitools/openapi-generator-cli generate \
  -i /w/llmgw-admin.json -g python -o /w/llmgw-admin-client

How it is kept true

The document is generated, not written. internal/control/adminapi/routes.go holds a table of every endpoint: its method and path, a summary, its query parameters, and Go values of the types it reads and answers with. The generator in internal/docgen/openapi reads those types by reflection, the way encoding/json encodes them, and takes each field's description from its comment in the source.

Two tests keep it from drifting. One registers the API on a recording mux and fails when an endpoint is served and not in the table, or in the table and not served. The other regenerates the document and fails when it differs from the committed one. After changing an endpoint or a type it carries:

go generate ./internal/docgen

What it does not cover

• Organizations, teams, projects, people and API keys. These live in the store and are managed with llmgw org, llmgw users and llmgw keys on the control plane, or in the web console's /admin pages on the client port behind a session. The admin API reads the tree only where a quota or a policy names a part of it.
• The client-facing API (/v1/chat/completions and the rest), which is OpenAI's and Anthropic's, on listen, with API keys.
• The rest of the admin listener: /metrics, /health, the dashboard pages and /debug/pprof, which are not under /api.