llmgw/terraform-provider-llmgw

terraform-provider-llmgw

A Terraform (and OpenTofu) provider for llmgw: the models in a cluster's catalog, capacity quotas on its teams and projects, and where its guardrail policies apply, kept in configuration and reviewed like the rest of it.

It talks to the control plane's admin API with the admin token, the same API llmgw ctl uses; see docs/admin-api.md for how that API is reached and what it asks of a request.

What it manages

llmgw_model	A model added to the catalog at runtime, as llmgw ctl model add adds one. Scaling, class, owner and adapters change in place; any other change to the definition replaces the model, which stops its replicas while it is added again.
llmgw_quota	A capacity quota on an organization, team or project: a guarantee, a limit, or both, in GPUs, CPU cores and memory.
llmgw_guard_attachment	A guardrail policy attached to an organization, team, project, key or person. The policies are defined in server.yaml's guard: section; this says where they apply.
data.llmgw_models	Every model in the catalog, with its replicas now.
data.llmgw_nodes	Every node, with its cards and state now.

Not here, because the admin API does not serve them: organizations, teams, projects, people and API keys. Those live in the store and are managed with llmgw org, llmgw users and llmgw keys on the control plane. A quota or an attachment names a team that already exists, by its path or its ID.

Models declared in the control plane's server.yaml are not managed here either: the cluster refuses to remove or change them through the API, since the configuration file would put them back on the next restart.

Configuring it

terraform {
  required_providers {
    llmgw = { source = "gitea.mynodes.xyz/ilker/llmgw" }
  }
}

provider "llmgw" {
  endpoint   = "http://127.0.0.1:9090"      # or LLMGW_ADMIN_URL
  # token    = "..."                         # or LLMGW_ADMIN_TOKEN
  token_file = "/var/lib/llmgw/admin-token"  # read when no token is given
  actor      = "terraform"                   # who the audit log says made the change
}

All four are optional and default to what is shown. The admin listener is on loopback, so run Terraform on the control plane — as a user that can read the token file, which is root's — or from elsewhere through a tunnel:

ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
export LLMGW_ADMIN_TOKEN=$(ssh admin@control-plane sudo cat /var/lib/llmgw/admin-token)
terraform apply

The endpoint must be an address, not a name: the admin listener refuses a Host that is a DNS name.

examples/main.tf has a model on CPUs, one on GPUs, a team's quota, a policy attachment and the two data sources.

Installing it

It is not in a public registry. Build it and point Terraform at it, either as a development override:

cd terraform-provider-llmgw
go build -o ~/.terraform.d/llmgw/ .
cat >> ~/.terraformrc <<'EOF'
provider_installation {
  dev_overrides {
    "gitea.mynodes.xyz/ilker/llmgw" = "/home/you/.terraform.d/llmgw"
  }
  direct {}
}
EOF

or in a filesystem mirror, which terraform init installs from like a registry and records in the lock file:

v=0.1.0
d=~/.terraform.d/plugins/gitea.mynodes.xyz/ilker/llmgw/$v/$(go env GOOS)_$(go env GOARCH)
mkdir -p $d
go build -ldflags "-X main.version=$v" -o $d/terraform-provider-llmgw_v$v .

with version = "0.1.0" beside source in required_providers.

How the state behaves

• A model's definition is kept as written. The catalog fills in what a definition leaves out — tp = 1, engine = "vllm", a 20-minute start-up timeout — and the state does not take those values, so a plan does not show them as differences. What can change in place is read back from the cluster: a model scaled or reclassified from a terminal shows as a change to put right on the next plan, and one removed from a terminal is added again.
• scaling.max left out follows min, as in the configuration file, and is read back. Leaving the whole scaling block out means no replicas: the model is in the catalog and does not run.
• Durations are written as Go writes them, "30m" or "1h30m".
• An owner or a quota's scope is kept as written (team:Acme/ml) beside the ID the cluster resolved it to (owner_id, and the quota's id), and is found again by the ID, so renaming the team in the store does not lose it.
• Import works for quotas, by team:ID as GET /api/v1/quotas lists it (the scope is then that ID; write it so in the configuration, or the next plan replaces the quota with itself), and for attachments, by scope:SCOPE-ID:policy as their id shows. Models cannot be imported yet.

Developing it

A module of its own, so the gateway's binary does not carry Terraform's dependencies; go test ./... at the repository's root does not include it.

cd terraform-provider-llmgw
go vet ./... && go test ./...           # the client, against an admin API in memory and the OpenAPI document
TF_ACC=1 go test ./internal/provider/   # Terraform itself, plan and apply, against the same fake

• internal/client is the HTTP client. Its Endpoints lists every endpoint it calls and it refuses any other; a test checks each of them, and every field of every body it sends and reads, against the gateway's generated OpenAPI document. A change to the admin API that breaks the provider fails here.
• internal/fakeapi is an admin API in memory: the token, the content type, the catalog's defaults and the tenancy tree's names, enough to run the provider without a cluster.
• The acceptance tests need TF_ACC=1 and a terraform on PATH (or TF_ACC_TERRAFORM_PATH); without one, terraform-plugin-testing downloads the latest. They pass with OpenTofu too: TF_ACC_TERRAFORM_PATH=$(which tofu) TF_ACC_PROVIDER_HOST=registry.opentofu.org.