Files
llmgw/terraform-provider-llmgw/README.md
T
Ilker ManapandClaude Opus 5.5 15a209e652 Terraform: organizations, teams, projects, keys and memberships
llmgw_org, llmgw_team and llmgw_project rename and move in place;
llmgw_key holds the key in a sensitive secret and is made again when
revoked or expired outside Terraform; llmgw_membership puts a person in
the tree. Tried with OpenTofu against the fake admin API, drift and
import included.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-09 06:42:07 +02:00

145 lines
7.2 KiB
Markdown

# terraform-provider-llmgw
A Terraform (and OpenTofu) provider for llmgw: the models in a cluster's
catalog, capacity quotas on its teams and projects, and where its guardrail
policies apply, kept in configuration and reviewed like the rest of it.
It talks to the control plane's admin API with the admin token, the same API
`llmgw ctl` uses; see [docs/admin-api.md](../docs/admin-api.md) for how that
API is reached and what it asks of a request.
## What it manages
| | |
|---|---|
| `llmgw_model` | A model added to the catalog at runtime, as `llmgw ctl model add` adds one. Scaling, class, owner and adapters change in place; any other change to the definition replaces the model, which stops its replicas while it is added again. |
| `llmgw_quota` | A capacity quota on an organization, team or project: a guarantee, a limit, or both, in GPUs, CPU cores and memory. |
| `llmgw_guard_attachment` | A guardrail policy attached to an organization, team, project, key or person. The policies are defined in `server.yaml`'s `guard:` section; this says where they apply. |
| `llmgw_org`, `llmgw_team`, `llmgw_project` | The tenancy tree. A team names its organization and a project its team, by path or ID; a new name renames in place, a new parent moves it with everything under it. Only something empty is removed, and a project that has ever had a key is never empty (the revoked key stays for usage history): rename it rather than destroy it. |
| `llmgw_key` | An API key in a project, for a person in a project they belong to, or a person's own. The key is in `secret` (sensitive) from the apply that made it; the gateway keeps only a hash. Any change makes a new key, and one revoked or expired outside Terraform is made again. Destroying it revokes it. |
| `llmgw_membership` | A person in an organization, team or project, as member or manager. People are not made here: they sign up, are invited, or come from the identity provider. Taking a membership away revokes their keys in projects they no longer reach. |
| `data.llmgw_models` | Every model in the catalog, with its replicas now. |
| `data.llmgw_nodes` | Every node, with its cards and state now. |
Not here: people's accounts (sign-up, invitation, OIDC/SCIM, and `llmgw
users` to approve, disable or delete), and key lifetime and spending limits
(`llmgw keys ttl`, `llmgw keys limits`). A quota or an attachment names a
part of the tree by its path or its ID; refer to an `llmgw_team`'s `id` to
have Terraform make the team first.
Models declared in the control plane's `server.yaml` are not managed here
either: the cluster refuses to remove or change them through the API, since
the configuration file would put them back on the next restart.
## Configuring it
```hcl
terraform {
required_providers {
llmgw = { source = "gitea.mynodes.xyz/ilker/llmgw" }
}
}
provider "llmgw" {
endpoint = "http://127.0.0.1:9090" # or LLMGW_ADMIN_URL
# token = "..." # or LLMGW_ADMIN_TOKEN
token_file = "/var/lib/llmgw/admin-token" # read when no token is given
actor = "terraform" # who the audit log says made the change
}
```
All four are optional and default to what is shown. The admin listener is on
loopback, so run Terraform on the control plane — as a user that can read the
token file, which is root's — or from elsewhere through a tunnel:
```sh
ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
export LLMGW_ADMIN_TOKEN=$(ssh admin@control-plane sudo cat /var/lib/llmgw/admin-token)
terraform apply
```
The endpoint must be an address, not a name: the admin listener refuses a
`Host` that is a DNS name.
[examples/main.tf](examples/main.tf) has a model on CPUs, one on GPUs, a team's
quota, a policy attachment and the two data sources.
## Installing it
It is not in a public registry. Build it and point Terraform at it, either as
a development override:
```sh
cd terraform-provider-llmgw
go build -o ~/.terraform.d/llmgw/ .
cat >> ~/.terraformrc <<'EOF'
provider_installation {
dev_overrides {
"gitea.mynodes.xyz/ilker/llmgw" = "/home/you/.terraform.d/llmgw"
}
direct {}
}
EOF
```
or in a filesystem mirror, which `terraform init` installs from like a
registry and records in the lock file:
```sh
v=0.1.0
d=~/.terraform.d/plugins/gitea.mynodes.xyz/ilker/llmgw/$v/$(go env GOOS)_$(go env GOARCH)
mkdir -p $d
go build -ldflags "-X main.version=$v" -o $d/terraform-provider-llmgw_v$v .
```
with `version = "0.1.0"` beside `source` in `required_providers`.
## How the state behaves
- **A model's definition is kept as written.** The catalog fills in what a
definition leaves out — `tp = 1`, `engine = "vllm"`, a 20-minute start-up
timeout — and the state does not take those values, so a plan does not show
them as differences. What can change in place is read back from the
cluster: a model scaled or reclassified from a terminal shows as a change to
put right on the next plan, and one removed from a terminal is added again.
- **`scaling.max` left out follows `min`**, as in the configuration file, and
is read back. Leaving the whole `scaling` block out means no replicas: the
model is in the catalog and does not run.
- **Durations are written as Go writes them**, `"30m"` or `"1h30m"`.
- **An owner or a quota's scope is kept as written** (`team:Acme/ml`) beside
the ID the cluster resolved it to (`owner_id`, and the quota's `id`), and is
found again by the ID, so renaming the team in the store does not lose it.
- **Import** works for quotas, by `team:ID` as `GET /api/v1/quotas` lists it
(the `scope` is then that ID; write it so in the configuration, or the next
plan replaces the quota with itself), for attachments, by
`scope:SCOPE-ID:policy` as their `id` shows, for organizations, teams,
projects and keys by their IDs (an imported key has no `secret`), and for
memberships by `USER-ID/scope:SCOPE-ID`. Models cannot be imported yet.
- **A project's `path` follows a move of its team at the next refresh**: the
project did not change, so the apply that moved the team does not rewrite
it.
## Developing it
A module of its own, so the gateway's binary does not carry Terraform's
dependencies; `go test ./...` at the repository's root does not include it.
```sh
cd terraform-provider-llmgw
go vet ./... && go test ./... # the client, against an admin API in memory and the OpenAPI document
TF_ACC=1 go test ./internal/provider/ # Terraform itself, plan and apply, against the same fake
```
- `internal/client` is the HTTP client. Its `Endpoints` lists every endpoint it
calls and it refuses any other; a test checks each of them, and every field
of every body it sends and reads, against the gateway's generated
[OpenAPI document](../docs/admin-api.openapi.json). A change to the admin API
that breaks the provider fails here.
- `internal/fakeapi` is an admin API in memory: the token, the content type,
the catalog's defaults and the tenancy tree's names, enough to run the
provider without a cluster.
- The acceptance tests need `TF_ACC=1` and a `terraform` on `PATH` (or
`TF_ACC_TERRAFORM_PATH`); without one, terraform-plugin-testing downloads the
latest. They pass with OpenTofu too:
`TF_ACC_TERRAFORM_PATH=$(which tofu) TF_ACC_PROVIDER_HOST=registry.opentofu.org`.