A Terraform provider for llmgw: models, quotas and guard attachments through the admin API
terraform-provider-llmgw/ is a module of its own, built on terraform-plugin-framework, so the gateway does not carry its dependencies. llmgw_model is a model added at runtime: scaling, class, owner and adapters change in place, anything else replaces it; the definition is kept as written rather than as the catalog filled it in. llmgw_quota and llmgw_guard_attachment keep the scope as written beside the ID it resolved to, and import by that ID. Data sources llmgw_models and llmgw_nodes. The client lists every endpoint it calls and refuses any other; a test checks each, and every field it sends and reads, against docs/admin-api.openapi.json, so a change to the admin API that breaks the provider fails here. Acceptance tests run Terraform against an admin API in memory when TF_ACC is set. CI vets and tests the module. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed
1 parent
a95c519bc5
commit
11b5d44702
20 files changed
+3398
No files matched your search
@@ -39,6 +39,16 @@ jobs:
|
||||
- name: Build
|
||||
run: go build ./...
|
||||
|
||||
# The Terraform provider is a module of its own, which ./... above does
|
||||
# not reach. Its tests check its client against docs/admin-api.openapi.json,
|
||||
# so a change to the admin API that breaks the provider fails here. The
|
||||
# acceptance tests (TF_ACC=1) are left out: they need Terraform itself.
|
||||
- name: Test the Terraform provider
|
||||
working-directory: terraform-provider-llmgw
|
||||
run: |
|
||||
go vet ./...
|
||||
go test ./...
|
||||
|
||||
# The NVML backend needs cgo and cannot run without a driver, but it must
|
||||
# at least keep compiling: it is the only source of NVLink topology and a
|
||||
# break would otherwise surface on a GPU node, far from here.
|
||||
|
||||
@@ -5,3 +5,7 @@
|
||||
*.db-shm
|
||||
.DS_Store
|
||||
coverage.out
|
||||
# The provider built in place, and what Terraform leaves beside an example.
|
||||
/terraform-provider-llmgw/terraform-provider-llmgw
|
||||
/terraform-provider-llmgw/examples/.terraform*
|
||||
/terraform-provider-llmgw/examples/terraform.tfstate*
|
||||
@@ -0,0 +1,57 @@
|
||||
# C19: OpenAPI for the admin API, and a Terraform provider
|
||||
|
||||
## CHANGELOG
|
||||
|
||||
### The admin API described, and a Terraform provider
|
||||
|
||||
- **The admin API in OpenAPI 3.1.** Every endpoint under `/api/v1`, its path
|
||||
and query parameters, and the schema of each request and answer with the
|
||||
meaning of each field, in
|
||||
[docs/admin-api.openapi.json](docs/admin-api.openapi.json) and served by
|
||||
every control plane at `GET /api/v1/openapi.json` (with the token, like the
|
||||
rest). The bearer token, the `X-Llmgw-Actor` header and the error shape are
|
||||
in it. Generated by `go generate ./internal/docgen` from a route table in
|
||||
`internal/control/adminapi/routes.go` and the Go types the handlers use, with
|
||||
the fields' comments as descriptions; a test fails when an endpoint is
|
||||
served and not in the table, another when the committed document is stale.
|
||||
Checked with openapi-spec-validator and openapi-generator-cli's validate.
|
||||
- **[docs/admin-api.md](docs/admin-api.md)**: how to reach the API (loopback,
|
||||
an SSH tunnel, `admin_remote`), the token at `/var/lib/llmgw/admin-token`,
|
||||
what a request must carry — an address in `Host`, `application/json` on
|
||||
every change, even a `DELETE` — and some calls with `curl`.
|
||||
- **`GET /api/v1/models/{name}`**: one model's whole definition as it is in
|
||||
force, with the defaults the catalog filled in. The listing has only its
|
||||
state and its profiles' names.
|
||||
- **A Terraform provider**, in `terraform-provider-llmgw/`, a module of its
|
||||
own so the gateway does not carry its dependencies. `llmgw_model` (a model
|
||||
added at runtime: scaling, class, owner and adapters change in place, the
|
||||
rest replaces it), `llmgw_quota`, `llmgw_guard_attachment`, and the
|
||||
`llmgw_models` and `llmgw_nodes` data sources. Its client is checked
|
||||
against the OpenAPI document, endpoint by endpoint and field by field, so a
|
||||
change to the admin API that breaks it fails its tests; CI runs them.
|
||||
Built and installed by hand for now: see its
|
||||
[README](terraform-provider-llmgw/README.md).
|
||||
|
||||
## BACKLOG
|
||||
|
||||
**Done (branch `features-c2`):** `adminapi.Routes` and
|
||||
`internal/docgen/openapi` generating `docs/admin-api.openapi.json` (3.1,
|
||||
bearer security, every route, query parameters, schemas from the Go types
|
||||
with their comments), served embedded at `GET /api/v1/openapi.json`, kept true
|
||||
by a route-table test and a staleness test; validated with
|
||||
openapi-spec-validator and openapi-generator-cli. `GET /api/v1/models/{name}`.
|
||||
`docs/admin-api.md`. `terraform-provider-llmgw/` (terraform-plugin-framework):
|
||||
`llmgw_model`, `llmgw_quota`, `llmgw_guard_attachment`, data sources
|
||||
`llmgw_models` and `llmgw_nodes`; client tests against a fake admin API and
|
||||
the OpenAPI document; acceptance tests with Terraform and OpenTofu against the
|
||||
fake; one apply, empty plan, in-place change and destroy against a real
|
||||
`llmgw server` run by hand. Left: organizations, teams, projects, people and
|
||||
keys are not in the admin API (they are `llmgw org`/`users`/`keys` against the
|
||||
store), so not in the provider; a model's definition beyond scaling, class,
|
||||
owner and adapters is changed by replacing it (the API has no edit, and
|
||||
`resources` is not used yet); `canary`, `slo`, `scaling.schedule` and
|
||||
`prewarm`, `guard` role and `origin` are not in `llmgw_model`; models cannot be
|
||||
imported; the provider is not published to a registry and has no release
|
||||
build; the acceptance tests are not run in CI (they need Terraform); the
|
||||
document's schemas mark nothing required and give no enums; `catalog.SLO`'s
|
||||
fields have no JSON tags, so they are `TTFTp99` and `TPOTp99` on the wire.
|
||||
@@ -0,0 +1,135 @@
|
||||
# terraform-provider-llmgw
|
||||
|
||||
A Terraform (and OpenTofu) provider for llmgw: the models in a cluster's
|
||||
catalog, capacity quotas on its teams and projects, and where its guardrail
|
||||
policies apply, kept in configuration and reviewed like the rest of it.
|
||||
|
||||
It talks to the control plane's admin API with the admin token, the same API
|
||||
`llmgw ctl` uses; see [docs/admin-api.md](../docs/admin-api.md) for how that
|
||||
API is reached and what it asks of a request.
|
||||
|
||||
## What it manages
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `llmgw_model` | A model added to the catalog at runtime, as `llmgw ctl model add` adds one. Scaling, class, owner and adapters change in place; any other change to the definition replaces the model, which stops its replicas while it is added again. |
|
||||
| `llmgw_quota` | A capacity quota on an organization, team or project: a guarantee, a limit, or both, in GPUs, CPU cores and memory. |
|
||||
| `llmgw_guard_attachment` | A guardrail policy attached to an organization, team, project, key or person. The policies are defined in `server.yaml`'s `guard:` section; this says where they apply. |
|
||||
| `data.llmgw_models` | Every model in the catalog, with its replicas now. |
|
||||
| `data.llmgw_nodes` | Every node, with its cards and state now. |
|
||||
|
||||
Not here, because the admin API does not serve them: organizations, teams,
|
||||
projects, people and API keys. Those live in the store and are managed with
|
||||
`llmgw org`, `llmgw users` and `llmgw keys` on the control plane. A quota or
|
||||
an attachment names a team that already exists, by its path or its ID.
|
||||
|
||||
Models declared in the control plane's `server.yaml` are not managed here
|
||||
either: the cluster refuses to remove or change them through the API, since
|
||||
the configuration file would put them back on the next restart.
|
||||
|
||||
## Configuring it
|
||||
|
||||
```hcl
|
||||
terraform {
|
||||
required_providers {
|
||||
llmgw = { source = "gitea.mynodes.xyz/ilker/llmgw" }
|
||||
}
|
||||
}
|
||||
|
||||
provider "llmgw" {
|
||||
endpoint = "http://127.0.0.1:9090" # or LLMGW_ADMIN_URL
|
||||
# token = "..." # or LLMGW_ADMIN_TOKEN
|
||||
token_file = "/var/lib/llmgw/admin-token" # read when no token is given
|
||||
actor = "terraform" # who the audit log says made the change
|
||||
}
|
||||
```
|
||||
|
||||
All four are optional and default to what is shown. The admin listener is on
|
||||
loopback, so run Terraform on the control plane — as a user that can read the
|
||||
token file, which is root's — or from elsewhere through a tunnel:
|
||||
|
||||
```sh
|
||||
ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
|
||||
export LLMGW_ADMIN_TOKEN=$(ssh admin@control-plane sudo cat /var/lib/llmgw/admin-token)
|
||||
terraform apply
|
||||
```
|
||||
|
||||
The endpoint must be an address, not a name: the admin listener refuses a
|
||||
`Host` that is a DNS name.
|
||||
|
||||
[examples/main.tf](examples/main.tf) has a model on CPUs, one on GPUs, a team's
|
||||
quota, a policy attachment and the two data sources.
|
||||
|
||||
## Installing it
|
||||
|
||||
It is not in a public registry. Build it and point Terraform at it, either as
|
||||
a development override:
|
||||
|
||||
```sh
|
||||
cd terraform-provider-llmgw
|
||||
go build -o ~/.terraform.d/llmgw/ .
|
||||
cat >> ~/.terraformrc <<'EOF'
|
||||
provider_installation {
|
||||
dev_overrides {
|
||||
"gitea.mynodes.xyz/ilker/llmgw" = "/home/you/.terraform.d/llmgw"
|
||||
}
|
||||
direct {}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
or in a filesystem mirror, which `terraform init` installs from like a
|
||||
registry and records in the lock file:
|
||||
|
||||
```sh
|
||||
v=0.1.0
|
||||
d=~/.terraform.d/plugins/gitea.mynodes.xyz/ilker/llmgw/$v/$(go env GOOS)_$(go env GOARCH)
|
||||
mkdir -p $d
|
||||
go build -ldflags "-X main.version=$v" -o $d/terraform-provider-llmgw_v$v .
|
||||
```
|
||||
|
||||
with `version = "0.1.0"` beside `source` in `required_providers`.
|
||||
|
||||
## How the state behaves
|
||||
|
||||
- **A model's definition is kept as written.** The catalog fills in what a
|
||||
definition leaves out — `tp = 1`, `engine = "vllm"`, a 20-minute start-up
|
||||
timeout — and the state does not take those values, so a plan does not show
|
||||
them as differences. What can change in place is read back from the
|
||||
cluster: a model scaled or reclassified from a terminal shows as a change to
|
||||
put right on the next plan, and one removed from a terminal is added again.
|
||||
- **`scaling.max` left out follows `min`**, as in the configuration file, and
|
||||
is read back. Leaving the whole `scaling` block out means no replicas: the
|
||||
model is in the catalog and does not run.
|
||||
- **Durations are written as Go writes them**, `"30m"` or `"1h30m"`.
|
||||
- **An owner or a quota's scope is kept as written** (`team:Acme/ml`) beside
|
||||
the ID the cluster resolved it to (`owner_id`, and the quota's `id`), and is
|
||||
found again by the ID, so renaming the team in the store does not lose it.
|
||||
- **Import** works for quotas, by `team:ID` as `GET /api/v1/quotas` lists it
|
||||
(the `scope` is then that ID; write it so in the configuration, or the next
|
||||
plan replaces the quota with itself), and for attachments, by
|
||||
`scope:SCOPE-ID:policy` as their `id` shows. Models cannot be imported yet.
|
||||
|
||||
## Developing it
|
||||
|
||||
A module of its own, so the gateway's binary does not carry Terraform's
|
||||
dependencies; `go test ./...` at the repository's root does not include it.
|
||||
|
||||
```sh
|
||||
cd terraform-provider-llmgw
|
||||
go vet ./... && go test ./... # the client, against an admin API in memory and the OpenAPI document
|
||||
TF_ACC=1 go test ./internal/provider/ # Terraform itself, plan and apply, against the same fake
|
||||
```
|
||||
|
||||
- `internal/client` is the HTTP client. Its `Endpoints` lists every endpoint it
|
||||
calls and it refuses any other; a test checks each of them, and every field
|
||||
of every body it sends and reads, against the gateway's generated
|
||||
[OpenAPI document](../docs/admin-api.openapi.json). A change to the admin API
|
||||
that breaks the provider fails here.
|
||||
- `internal/fakeapi` is an admin API in memory: the token, the content type,
|
||||
the catalog's defaults and the tenancy tree's names, enough to run the
|
||||
provider without a cluster.
|
||||
- The acceptance tests need `TF_ACC=1` and a `terraform` on `PATH` (or
|
||||
`TF_ACC_TERRAFORM_PATH`); without one, terraform-plugin-testing downloads the
|
||||
latest. They pass with OpenTofu too:
|
||||
`TF_ACC_TERRAFORM_PATH=$(which tofu) TF_ACC_PROVIDER_HOST=registry.opentofu.org`.
|
||||
@@ -0,0 +1,94 @@
|
||||
# A model, a team's quota and a guardrail policy, managed through the control
|
||||
# plane's admin API. Run where the admin listener is reachable: on the control
|
||||
# plane itself, or through `ssh -N -L 9090:127.0.0.1:9090 control-plane`.
|
||||
|
||||
terraform {
|
||||
required_providers {
|
||||
llmgw = {
|
||||
source = "gitea.mynodes.xyz/ilker/llmgw"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "llmgw" {
|
||||
# The defaults, written out. The token comes from LLMGW_ADMIN_TOKEN when it
|
||||
# is set, and otherwise from token_file, which only root can read.
|
||||
endpoint = "http://127.0.0.1:9090"
|
||||
token_file = "/var/lib/llmgw/admin-token"
|
||||
actor = "terraform"
|
||||
}
|
||||
|
||||
# A small model on CPUs, loaded when asked for and stopped after half an hour
|
||||
# unused.
|
||||
resource "llmgw_model" "qwen_small" {
|
||||
name = "qwen-small"
|
||||
description = "Qwen 2.5 1.5B instruct, for quick questions"
|
||||
source = "/srv/models/qwen2.5-1.5b-instruct-q4_k_m.gguf"
|
||||
aliases = ["qwen"]
|
||||
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 4
|
||||
cpu_cores = 4
|
||||
}]
|
||||
|
||||
scaling = {
|
||||
min = 0
|
||||
max = 2
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
|
||||
class = "opportunistic"
|
||||
owner = "team:Acme/ml"
|
||||
}
|
||||
|
||||
# A larger one on GPUs, always running.
|
||||
resource "llmgw_model" "llama_70b" {
|
||||
name = "llama-3.3-70b"
|
||||
source = "meta-llama/Llama-3.3-70B-Instruct"
|
||||
|
||||
profiles = [{
|
||||
name = "h100"
|
||||
engine = "vllm"
|
||||
gpu_class = "h100-80g"
|
||||
tp = 4
|
||||
}]
|
||||
|
||||
scaling = {
|
||||
min = 1
|
||||
max = 3
|
||||
}
|
||||
|
||||
capabilities = {
|
||||
tools = true
|
||||
}
|
||||
fallbacks = [llmgw_model.qwen_small.name]
|
||||
}
|
||||
|
||||
# The ml team may hold at most eight GPUs, and is always given two.
|
||||
resource "llmgw_quota" "ml" {
|
||||
scope = "team:Acme/ml"
|
||||
guarantee = { gpus = 2 }
|
||||
limit = { gpus = 8 }
|
||||
}
|
||||
|
||||
# The finance policy, from server.yaml's guard.policies, on the ml team.
|
||||
resource "llmgw_guard_attachment" "ml_finance" {
|
||||
target = "team:Acme/ml"
|
||||
policy = "finance"
|
||||
}
|
||||
|
||||
data "llmgw_nodes" "all" {}
|
||||
|
||||
data "llmgw_models" "all" {
|
||||
depends_on = [llmgw_model.qwen_small, llmgw_model.llama_70b]
|
||||
}
|
||||
|
||||
output "free_gpus" {
|
||||
value = sum([for n in data.llmgw_nodes.all.nodes : n.gpu_free if n.state == "ready"])
|
||||
}
|
||||
|
||||
output "models_not_ready" {
|
||||
value = [for m in data.llmgw_models.all.models : m.name if m.ready_replicas < m.min_replicas]
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
module gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw
|
||||
|
||||
go 1.26.6
|
||||
|
||||
require (
|
||||
github.com/hashicorp/terraform-plugin-framework v1.19.0
|
||||
github.com/hashicorp/terraform-plugin-go v0.31.0
|
||||
github.com/hashicorp/terraform-plugin-testing v1.16.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/ProtonMail/go-crypto v1.4.1 // indirect
|
||||
github.com/agext/levenshtein v1.2.3 // indirect
|
||||
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
|
||||
github.com/cloudflare/circl v1.6.3 // indirect
|
||||
github.com/fatih/color v1.18.0 // indirect
|
||||
github.com/golang/protobuf v1.5.4 // indirect
|
||||
github.com/google/go-cmp v0.7.0 // indirect
|
||||
github.com/hashicorp/errwrap v1.1.0 // indirect
|
||||
github.com/hashicorp/go-checkpoint v0.5.0 // indirect
|
||||
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
|
||||
github.com/hashicorp/go-cty v1.5.0 // indirect
|
||||
github.com/hashicorp/go-hclog v1.6.3 // indirect
|
||||
github.com/hashicorp/go-multierror v1.1.1 // indirect
|
||||
github.com/hashicorp/go-plugin v1.7.0 // indirect
|
||||
github.com/hashicorp/go-retryablehttp v0.7.8 // indirect
|
||||
github.com/hashicorp/go-uuid v1.0.3 // indirect
|
||||
github.com/hashicorp/go-version v1.9.0 // indirect
|
||||
github.com/hashicorp/hc-install v0.9.4 // indirect
|
||||
github.com/hashicorp/hcl/v2 v2.24.0 // indirect
|
||||
github.com/hashicorp/logutils v1.0.0 // indirect
|
||||
github.com/hashicorp/terraform-exec v0.25.1 // indirect
|
||||
github.com/hashicorp/terraform-json v0.27.2 // indirect
|
||||
github.com/hashicorp/terraform-plugin-log v0.10.0 // indirect
|
||||
github.com/hashicorp/terraform-plugin-sdk/v2 v2.40.0 // indirect
|
||||
github.com/hashicorp/terraform-registry-address v0.4.0 // indirect
|
||||
github.com/hashicorp/terraform-svchost v0.2.1 // indirect
|
||||
github.com/hashicorp/yamux v0.1.2 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mitchellh/copystructure v1.2.0 // indirect
|
||||
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
|
||||
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
|
||||
github.com/mitchellh/mapstructure v1.5.0 // indirect
|
||||
github.com/mitchellh/reflectwalk v1.0.2 // indirect
|
||||
github.com/oklog/run v1.2.0 // indirect
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible // indirect
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect
|
||||
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
|
||||
github.com/zclconf/go-cty v1.18.1 // indirect
|
||||
golang.org/x/crypto v0.50.0 // indirect
|
||||
golang.org/x/mod v0.35.0 // indirect
|
||||
golang.org/x/net v0.52.0 // indirect
|
||||
golang.org/x/sync v0.20.0 // indirect
|
||||
golang.org/x/sys v0.43.0 // indirect
|
||||
golang.org/x/text v0.36.0 // indirect
|
||||
golang.org/x/tools v0.43.0 // indirect
|
||||
google.golang.org/appengine v1.6.8 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect
|
||||
google.golang.org/grpc v1.79.3 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
)
|
||||
@@ -0,0 +1,242 @@
|
||||
dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
|
||||
dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
|
||||
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
|
||||
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
|
||||
github.com/ProtonMail/go-crypto v1.4.1 h1:9RfcZHqEQUvP8RzecWEUafnZVtEvrBVL9BiF67IQOfM=
|
||||
github.com/ProtonMail/go-crypto v1.4.1/go.mod h1:e1OaTyu5SYVrO9gKOEhTc+5UcXtTUa+P3uLudwcgPqo=
|
||||
github.com/agext/levenshtein v1.2.3 h1:YB2fHEn0UJagG8T1rrWknE3ZQzWM06O8AMAatNn7lmo=
|
||||
github.com/agext/levenshtein v1.2.3/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558=
|
||||
github.com/apparentlymart/go-textseg/v12 v12.0.0/go.mod h1:S/4uRK2UtaQttw1GenVJEynmyUenKwP++x/+DdGV/Ec=
|
||||
github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
|
||||
github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
|
||||
github.com/bufbuild/protocompile v0.14.1 h1:iA73zAf/fyljNjQKwYzUHD6AD4R8KMasmwa/FBatYVw=
|
||||
github.com/bufbuild/protocompile v0.14.1/go.mod h1:ppVdAIhbr2H8asPk6k4pY7t9zB1OU5DoEw9xY/FUi1c=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/cloudflare/circl v1.6.3 h1:9GPOhQGF9MCYUeXyMYlqTR6a5gTrgR/fBLXvUgtVcg8=
|
||||
github.com/cloudflare/circl v1.6.3/go.mod h1:2eXP6Qfat4O/Yhh8BznvKnJ+uzEoTQ6jVKJRn81BiS4=
|
||||
github.com/cyphar/filepath-securejoin v0.4.1 h1:JyxxyPEaktOD+GAnqIqTf9A8tHyAG22rowi7HkoSU1s=
|
||||
github.com/cyphar/filepath-securejoin v0.4.1/go.mod h1:Sdj7gXlvMcPZsbhwhQ33GguGLDGQL7h7bg04C/+u9jI=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
|
||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
|
||||
github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
|
||||
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
|
||||
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
|
||||
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
|
||||
github.com/go-git/go-billy/v5 v5.8.0 h1:I8hjc3LbBlXTtVuFNJuwYuMiHvQJDq1AT6u4DwDzZG0=
|
||||
github.com/go-git/go-billy/v5 v5.8.0/go.mod h1:RpvI/rw4Vr5QA+Z60c6d6LXH0rYJo0uD5SqfmrrheCY=
|
||||
github.com/go-git/go-git/v5 v5.18.0 h1:O831KI+0PR51hM2kep6T8k+w0/LIAD490gvqMCvL5hM=
|
||||
github.com/go-git/go-git/v5 v5.18.0/go.mod h1:pW/VmeqkanRFqR6AljLcs7EA7FbZaN5MQqO7oZADXpo=
|
||||
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-test/deep v1.0.3 h1:ZrJSEWsXzPOxaZnFteGEfooLba+ju3FYIbOrS+rQd68=
|
||||
github.com/go-test/deep v1.0.3/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
|
||||
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ=
|
||||
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw=
|
||||
github.com/golang/protobuf v1.1.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
|
||||
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
|
||||
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
|
||||
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
|
||||
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/go-checkpoint v0.5.0 h1:MFYpPZCnQqQTE18jFwSII6eUQrD/oxMFp3mlgcqk5mU=
|
||||
github.com/hashicorp/go-checkpoint v0.5.0/go.mod h1:7nfLNL10NsxqO4iWuW6tWW0HjZuDrwkBuEQsVcpCOgg=
|
||||
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
|
||||
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
|
||||
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
|
||||
github.com/hashicorp/go-cty v1.5.0 h1:EkQ/v+dDNUqnuVpmS5fPqyY71NXVgT5gf32+57xY8g0=
|
||||
github.com/hashicorp/go-cty v1.5.0/go.mod h1:lFUCG5kd8exDobgSfyj4ONE/dc822kiYMguVKdHGMLM=
|
||||
github.com/hashicorp/go-hclog v1.6.3 h1:Qr2kF+eVWjTiYmU7Y31tYlP1h0q/X3Nl3tPGdaB11/k=
|
||||
github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M=
|
||||
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
|
||||
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
|
||||
github.com/hashicorp/go-plugin v1.7.0 h1:YghfQH/0QmPNc/AZMTFE3ac8fipZyZECHdDPshfk+mA=
|
||||
github.com/hashicorp/go-plugin v1.7.0/go.mod h1:BExt6KEaIYx804z8k4gRzRLEvxKVb+kn0NMcihqOqb8=
|
||||
github.com/hashicorp/go-retryablehttp v0.7.8 h1:ylXZWnqa7Lhqpk0L1P1LzDtGcCR0rPVUrx/c8Unxc48=
|
||||
github.com/hashicorp/go-retryablehttp v0.7.8/go.mod h1:rjiScheydd+CxvumBsIrFKlx3iS0jrZ7LvzFGFmuKbw=
|
||||
github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
|
||||
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
|
||||
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
|
||||
github.com/hashicorp/go-version v1.9.0 h1:CeOIz6k+LoN3qX9Z0tyQrPtiB1DFYRPfCIBtaXPSCnA=
|
||||
github.com/hashicorp/go-version v1.9.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
|
||||
github.com/hashicorp/hc-install v0.9.4 h1:KKWOpUG0EqIV63Qk2GGFrZ0s275NVs5lKf9N5vjBNoc=
|
||||
github.com/hashicorp/hc-install v0.9.4/go.mod h1:4LRYeEN2bMIFfIv57ldMWt9awfuZhvpbRt0vWmv51WU=
|
||||
github.com/hashicorp/hcl/v2 v2.24.0 h1:2QJdZ454DSsYGoaE6QheQZjtKZSUs9Nh2izTWiwQxvE=
|
||||
github.com/hashicorp/hcl/v2 v2.24.0/go.mod h1:oGoO1FIQYfn/AgyOhlg9qLC6/nOJPX3qGbkZpYAcqfM=
|
||||
github.com/hashicorp/logutils v1.0.0 h1:dLEQVugN8vlakKOUE3ihGLTZJRB4j+M2cdTm/ORI65Y=
|
||||
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
|
||||
github.com/hashicorp/terraform-exec v0.25.1 h1:PRutYRGM8pixV3B8812NYoBK5O+yuf3qcB/70KFKGiU=
|
||||
github.com/hashicorp/terraform-exec v0.25.1/go.mod h1:+izOYrs9sKMQK4OYvGDnrSSJHY/pm4e4eXFqSL2Q5mA=
|
||||
github.com/hashicorp/terraform-json v0.27.2 h1:BwGuzM6iUPqf9JYM/Z4AF1OJ5VVJEEzoKST/tRDBJKU=
|
||||
github.com/hashicorp/terraform-json v0.27.2/go.mod h1:GzPLJ1PLdUG5xL6xn1OXWIjteQRT2CNT9o/6A9mi9hE=
|
||||
github.com/hashicorp/terraform-plugin-framework v1.19.0 h1:q0bwyhxAOR3vfdgbk9iplv3MlTv/dhBHTXjQOtQDoBA=
|
||||
github.com/hashicorp/terraform-plugin-framework v1.19.0/go.mod h1:YRXOBu0jvs7xp4AThBbX4mAzYaMJ1JgtFH//oGKxwLc=
|
||||
github.com/hashicorp/terraform-plugin-go v0.31.0 h1:0Fz2r9DQ+kNNl6bx8HRxFd1TfMKUvnrOtvJPmp3Z0q8=
|
||||
github.com/hashicorp/terraform-plugin-go v0.31.0/go.mod h1:A88bDhd/cW7FnwqxQRz3slT+QY6yzbHKc6AOTtmdeS8=
|
||||
github.com/hashicorp/terraform-plugin-log v0.10.0 h1:eu2kW6/QBVdN4P3Ju2WiB2W3ObjkAsyfBsL3Wh1fj3g=
|
||||
github.com/hashicorp/terraform-plugin-log v0.10.0/go.mod h1:/9RR5Cv2aAbrqcTSdNmY1NRHP4E3ekrXRGjqORpXyB0=
|
||||
github.com/hashicorp/terraform-plugin-sdk/v2 v2.40.0 h1:MKS/2URqeJRwJdbOfcbdsZCq/IRrNkqJNN0GtVIsuGs=
|
||||
github.com/hashicorp/terraform-plugin-sdk/v2 v2.40.0/go.mod h1:PuG4P97Ju3QXW6c6vRkRadWJbvnEu2Xh+oOuqcYOqX4=
|
||||
github.com/hashicorp/terraform-plugin-testing v1.16.0 h1:GB97nGnJ1hESpDrCjqZig38RodSF0gdRzxlDupLXP38=
|
||||
github.com/hashicorp/terraform-plugin-testing v1.16.0/go.mod h1:eQPYAy9xFMV7xtIFX8Y+wJGtUB++HBl329zCF6PBMZk=
|
||||
github.com/hashicorp/terraform-registry-address v0.4.0 h1:S1yCGomj30Sao4l5BMPjTGZmCNzuv7/GDTDX99E9gTk=
|
||||
github.com/hashicorp/terraform-registry-address v0.4.0/go.mod h1:LRS1Ay0+mAiRkUyltGT+UHWkIqTFvigGn/LbMshfflE=
|
||||
github.com/hashicorp/terraform-svchost v0.2.1 h1:ubvrTFw3Q7CsoEaX7V06PtCTKG3wu7GyyobAoN4eF3Q=
|
||||
github.com/hashicorp/terraform-svchost v0.2.1/go.mod h1:zDMheBLvNzu7Q6o9TBvPqiZToJcSuCLXjAXxBslSky4=
|
||||
github.com/hashicorp/yamux v0.1.2 h1:XtB8kyFOyHXYVFnwT5C3+Bdo8gArse7j2AQ0DA0Uey8=
|
||||
github.com/hashicorp/yamux v0.1.2/go.mod h1:C+zze2n6e/7wshOZep2A70/aQU6QBRWJO/G6FT1wIns=
|
||||
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
|
||||
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
|
||||
github.com/jhump/protoreflect v1.17.0 h1:qOEr613fac2lOuTgWN4tPAtLL7fUSbuJL5X5XumQh94=
|
||||
github.com/jhump/protoreflect v1.17.0/go.mod h1:h9+vUUL38jiBzck8ck+6G/aeMX8Z4QUY/NiJPwPNi+8=
|
||||
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
|
||||
github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
|
||||
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
|
||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
||||
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
|
||||
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
|
||||
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
|
||||
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
|
||||
github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
|
||||
github.com/mitchellh/go-testing-interface v1.14.1 h1:jrgshOhYAUVNMAJiKbEu7EqAwgJJ2JqpQmpLJOu07cU=
|
||||
github.com/mitchellh/go-testing-interface v1.14.1/go.mod h1:gfgS7OtZj6MA4U1UrDRp04twqAjfvlZyCfX3sDjEym8=
|
||||
github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0=
|
||||
github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
|
||||
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
|
||||
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
|
||||
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
|
||||
github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
|
||||
github.com/oklog/run v1.2.0 h1:O8x3yXwah4A73hJdlrwo/2X6J62gE5qTMusH0dvz60E=
|
||||
github.com/oklog/run v1.2.0/go.mod h1:mgDbKRSwPhJfesJ4PntqFUbKQRZ50NgmZTSPlFA0YFk=
|
||||
github.com/pjbgf/sha1cd v0.3.2 h1:a9wb0bp1oC2TGwStyn0Umc/IGKQnEgF0vVaZ8QF8eo4=
|
||||
github.com/pjbgf/sha1cd v0.3.2/go.mod h1:zQWigSxVmsHEZow5qaLtPYxpcKMMQpa09ixqBxuCS6A=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
|
||||
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
||||
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
|
||||
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
|
||||
github.com/skeema/knownhosts v1.3.1 h1:X2osQ+RAjK76shCbvhHHHVl3ZlgDm8apHEHFqRjnBY8=
|
||||
github.com/skeema/knownhosts v1.3.1/go.mod h1:r7KTdC8l4uxWRyK2TpQZ/1o5HaSzh06ePQNxPwTcfiY=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
|
||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/vmihailenco/msgpack v3.3.3+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible h1:dSLoQfGFAo3F6OoNhwUmLwVgaUXK79GlxNBwueZn0xI=
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1 h1:cQriyiUvjTwOHg8QZaPihLWeRAAVoCpE00IUPn0Bjt8=
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1/go.mod h1:GaZTsDaehaPpQVyxrf5mtQlH+pc21PIudVV/E3rRQok=
|
||||
github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
|
||||
github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
|
||||
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
|
||||
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
github.com/zclconf/go-cty v1.18.1 h1:yEGE8M4iIZlyKQURZNb2SnEyZlZHUcBCnx6KF81KuwM=
|
||||
github.com/zclconf/go-cty v1.18.1/go.mod h1:qpnV6EDNgC1sns/AleL1fvatHw72j+S+nS+MJ+T2CSg=
|
||||
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940 h1:4r45xpDWB6ZMSMNJFMOjqrGHynW3DIBuR2H9j0ug+Mo=
|
||||
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940/go.mod h1:CmBdvvj3nqzfzJ6nTCIwDTPZ56aVGvDrmztiO5g3qrM=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||
go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48=
|
||||
go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8=
|
||||
go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0=
|
||||
go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs=
|
||||
go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18=
|
||||
go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew=
|
||||
go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI=
|
||||
go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
|
||||
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
|
||||
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
|
||||
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
||||
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
|
||||
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
|
||||
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
|
||||
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
|
||||
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
|
||||
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
|
||||
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
|
||||
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
|
||||
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
|
||||
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
|
||||
golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
|
||||
golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
|
||||
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
|
||||
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
|
||||
google.golang.org/appengine v1.6.8 h1:IhEN5q69dyKagZPYMSdIjS2HqprW324FRQZJcGqPAsM=
|
||||
google.golang.org/appengine v1.6.8/go.mod h1:1jJ3jBArFh5pcgW8gCtRJnepW8FzD1V44FJffLiz/Ds=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 h1:gRkg/vSppuSQoDjxyiGfN4Upv/h/DQmIR10ZU8dh4Ww=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
|
||||
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
|
||||
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
|
||||
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
|
||||
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
|
||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
|
||||
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
|
||||
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
|
||||
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
@@ -0,0 +1,438 @@
|
||||
// Package client talks to llmgw's admin API over HTTP with the admin token.
|
||||
//
|
||||
// Its own types rather than the gateway's: the provider is a separate module
|
||||
// so the gateway does not carry Terraform's dependencies, and the gateway's
|
||||
// packages are internal to it. What keeps the two in step is the OpenAPI
|
||||
// document the gateway generates, docs/admin-api.openapi.json, which a test
|
||||
// here checks every endpoint and field this client uses against.
|
||||
package client
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Endpoints are the admin API's endpoints this client calls, as the OpenAPI
|
||||
// document names them. A request for any other is refused before it is
|
||||
// sent, so this list is the whole of what the contract test has to check.
|
||||
var Endpoints = []string{
|
||||
"GET /api/v1/models",
|
||||
"POST /api/v1/models",
|
||||
"GET /api/v1/models/{name}",
|
||||
"DELETE /api/v1/models/{name}",
|
||||
"POST /api/v1/models/{name}/scale",
|
||||
"POST /api/v1/models/{name}/class",
|
||||
"POST /api/v1/models/{name}/owner",
|
||||
"POST /api/v1/models/{name}/adapters",
|
||||
"GET /api/v1/nodes",
|
||||
"GET /api/v1/quotas",
|
||||
"PUT /api/v1/quotas",
|
||||
"DELETE /api/v1/quotas/{scope}",
|
||||
"GET /api/v1/guard",
|
||||
"POST /api/v1/guard/attachments",
|
||||
"DELETE /api/v1/guard/attachments",
|
||||
}
|
||||
|
||||
// Client is a connection to one control plane's admin API.
|
||||
type Client struct {
|
||||
base *url.URL
|
||||
token string
|
||||
actor string
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
// New builds a client for the admin listener at endpoint, such as
|
||||
// http://127.0.0.1:9090. Actor is who changes are recorded as made by.
|
||||
func New(endpoint, token, actor string) (*Client, error) {
|
||||
u, err := url.Parse(strings.TrimRight(endpoint, "/"))
|
||||
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
|
||||
return nil, fmt.Errorf("endpoint %q is not an http or https address", endpoint)
|
||||
}
|
||||
if token == "" {
|
||||
return nil, errors.New("no admin token: set token, token_file or LLMGW_ADMIN_TOKEN")
|
||||
}
|
||||
return &Client{base: u, token: token, actor: actor, http: &http.Client{Timeout: 2 * time.Minute}}, nil
|
||||
}
|
||||
|
||||
// Error is the API refusing a request, with what it said.
|
||||
type Error struct {
|
||||
Method, Path string
|
||||
Status int
|
||||
Message string
|
||||
}
|
||||
|
||||
func (e *Error) Error() string {
|
||||
return fmt.Sprintf("%s %s: %d %s", e.Method, e.Path, e.Status, e.Message)
|
||||
}
|
||||
|
||||
// IsNotFound reports whether err is the API saying there is no such thing.
|
||||
func IsNotFound(err error) bool {
|
||||
var e *Error
|
||||
return errors.As(err, &e) && e.Status == http.StatusNotFound
|
||||
}
|
||||
|
||||
// do sends one request to an endpoint named as in Endpoints, with args put
|
||||
// in its path parameters in order.
|
||||
func (c *Client) do(ctx context.Context, endpoint string, args []string, in, out any) error {
|
||||
if !slices.Contains(Endpoints, endpoint) {
|
||||
return fmt.Errorf("the client does not call %s: add it to Endpoints", endpoint)
|
||||
}
|
||||
method, tmpl, _ := strings.Cut(endpoint, " ")
|
||||
p := tmpl
|
||||
for _, a := range args {
|
||||
i, j := strings.Index(p, "{"), strings.Index(p, "}")
|
||||
if i < 0 || j < i {
|
||||
return fmt.Errorf("%s takes fewer parameters than given", endpoint)
|
||||
}
|
||||
p = p[:i] + url.PathEscape(a) + p[j+1:]
|
||||
}
|
||||
if strings.Contains(p, "{") {
|
||||
return fmt.Errorf("%s takes more parameters than given", endpoint)
|
||||
}
|
||||
|
||||
var body io.Reader
|
||||
if in != nil {
|
||||
b, err := json.Marshal(in)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body = bytes.NewReader(b)
|
||||
}
|
||||
plain, err := url.PathUnescape(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
u := *c.base
|
||||
u.Path = strings.TrimRight(u.Path, "/") + plain
|
||||
// RawPath keeps an escaped slash in a name escaped: a model or a scope
|
||||
// called Acme/ml is one segment, not two.
|
||||
u.RawPath = strings.TrimRight(c.base.EscapedPath(), "/") + p
|
||||
req, err := http.NewRequestWithContext(ctx, method, u.String(), body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
// Every change must say it is JSON, even one without a body: the admin
|
||||
// listener refuses any other, as a page's form post cannot set it.
|
||||
if method != http.MethodGet {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
if c.actor != "" {
|
||||
req.Header.Set("X-Llmgw-Actor", c.actor)
|
||||
}
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
b, err := io.ReadAll(io.LimitReader(resp.Body, 32<<20))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.StatusCode >= 300 {
|
||||
var e struct {
|
||||
Error string `json:"error"`
|
||||
}
|
||||
msg := strings.TrimSpace(string(b))
|
||||
if json.Unmarshal(b, &e) == nil && e.Error != "" {
|
||||
msg = e.Error
|
||||
}
|
||||
return &Error{Method: method, Path: tmpl, Status: resp.StatusCode, Message: msg}
|
||||
}
|
||||
if out != nil {
|
||||
if err := json.Unmarshal(b, out); err != nil {
|
||||
return fmt.Errorf("%s %s: the answer is not what was expected: %w", method, tmpl, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- models ---
|
||||
|
||||
// Model is a model's definition, as the catalog holds it.
|
||||
type Model struct {
|
||||
Name string `json:"name,omitempty"`
|
||||
Aliases []string `json:"aliases,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Source string `json:"source,omitempty"`
|
||||
Profiles []Profile `json:"profiles,omitempty"`
|
||||
Scaling Scaling `json:"scaling,omitzero"`
|
||||
Class string `json:"class,omitempty"`
|
||||
Owner string `json:"owner,omitempty"`
|
||||
External *External `json:"external,omitempty"`
|
||||
Adapters []Adapter `json:"adapters,omitempty"`
|
||||
Fallbacks []string `json:"fallbacks,omitempty"`
|
||||
RoutingProfile string `json:"routing_profile,omitempty"`
|
||||
ContextLength int64 `json:"context_length,omitempty"`
|
||||
MaxOutput int64 `json:"max_output,omitempty"`
|
||||
Capabilities *Capabilities `json:"capabilities,omitempty"`
|
||||
}
|
||||
|
||||
// Profile is how a model runs on one kind of node.
|
||||
type Profile struct {
|
||||
Name string `json:"name,omitempty"`
|
||||
Engine string `json:"engine,omitempty"`
|
||||
GPUClass string `json:"gpu_class,omitempty"`
|
||||
TP int64 `json:"tp,omitempty"`
|
||||
PP int64 `json:"pp,omitempty"`
|
||||
GPUsPerNode int64 `json:"gpus_per_node,omitempty"`
|
||||
Nodes int64 `json:"nodes,omitempty"`
|
||||
Args []string `json:"args,omitempty"`
|
||||
Env map[string]string `json:"env,omitempty"`
|
||||
MinMemoryGB int64 `json:"min_memory_gb,omitempty"`
|
||||
CPUCores int64 `json:"cpu_cores,omitempty"`
|
||||
MinDiskGB int64 `json:"min_disk_gb,omitempty"`
|
||||
NodeSelector map[string]string `json:"node_selector,omitempty"`
|
||||
Weight float64 `json:"weight,omitempty"`
|
||||
Priority int64 `json:"priority,omitempty"`
|
||||
StartupTimeout time.Duration `json:"startup_timeout,omitempty"`
|
||||
}
|
||||
|
||||
// Scaling is how many replicas a model runs.
|
||||
type Scaling struct {
|
||||
Min int64 `json:"min,omitempty"`
|
||||
Max int64 `json:"max,omitempty"`
|
||||
IdleTimeout time.Duration `json:"idle_timeout,omitempty"`
|
||||
}
|
||||
|
||||
// External is a model served by a provider outside the cluster.
|
||||
type External struct {
|
||||
URL string `json:"url"`
|
||||
Model string `json:"model"`
|
||||
APIKeyFile string `json:"api_key_file,omitempty"`
|
||||
}
|
||||
|
||||
// Adapter is a LoRA adapter a model serves.
|
||||
type Adapter struct {
|
||||
Name string `json:"name"`
|
||||
Source string `json:"source"`
|
||||
Scale float64 `json:"scale,omitempty"`
|
||||
}
|
||||
|
||||
// Capabilities are what a model can do beyond chat.
|
||||
type Capabilities struct {
|
||||
Tools bool `json:"tools,omitempty"`
|
||||
Vision bool `json:"vision,omitempty"`
|
||||
Embeddings bool `json:"embeddings,omitempty"`
|
||||
}
|
||||
|
||||
// ModelView is a model as the listing shows it.
|
||||
type ModelView struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Aliases []string `json:"aliases,omitempty"`
|
||||
Source string `json:"source"`
|
||||
Min int64 `json:"min_replicas"`
|
||||
Max int64 `json:"max_replicas"`
|
||||
Ready int64 `json:"ready_replicas"`
|
||||
Total int64 `json:"total_replicas"`
|
||||
Desired int64 `json:"desired_replicas"`
|
||||
Parked bool `json:"parked,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Profiles []string `json:"profiles"`
|
||||
Class string `json:"class"`
|
||||
Owner string `json:"owner,omitempty"`
|
||||
External string `json:"external,omitempty"`
|
||||
}
|
||||
|
||||
// ScaleRequest sets a model's replicas.
|
||||
type ScaleRequest struct {
|
||||
Min int64 `json:"min"`
|
||||
Max int64 `json:"max,omitempty"`
|
||||
IdleTimeout time.Duration `json:"idle_timeout,omitempty"`
|
||||
}
|
||||
|
||||
// ClassRequest sets a model's capacity class.
|
||||
type ClassRequest struct {
|
||||
Class string `json:"class"`
|
||||
}
|
||||
|
||||
// OwnerRequest sets a model's owner.
|
||||
type OwnerRequest struct {
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
// OwnerView is a model's owner, as scope:ID.
|
||||
type OwnerView struct {
|
||||
Model string `json:"model"`
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
// AdaptersRequest replaces a model's adapters.
|
||||
type AdaptersRequest struct {
|
||||
Adapters []Adapter `json:"adapters"`
|
||||
}
|
||||
|
||||
// Models lists the catalog.
|
||||
func (c *Client) Models(ctx context.Context) ([]ModelView, error) {
|
||||
var out []ModelView
|
||||
return out, c.do(ctx, "GET /api/v1/models", nil, nil, &out)
|
||||
}
|
||||
|
||||
// AddModel adds a model to the catalog.
|
||||
func (c *Client) AddModel(ctx context.Context, m Model) error {
|
||||
return c.do(ctx, "POST /api/v1/models", nil, m, nil)
|
||||
}
|
||||
|
||||
// Model reads one model's definition, as it is in force.
|
||||
func (c *Client) Model(ctx context.Context, name string) (Model, error) {
|
||||
var out Model
|
||||
return out, c.do(ctx, "GET /api/v1/models/{name}", []string{name}, nil, &out)
|
||||
}
|
||||
|
||||
// RemoveModel removes a model added at runtime.
|
||||
func (c *Client) RemoveModel(ctx context.Context, name string) error {
|
||||
return c.do(ctx, "DELETE /api/v1/models/{name}", []string{name}, nil, nil)
|
||||
}
|
||||
|
||||
// Scale sets how many replicas a model runs.
|
||||
func (c *Client) Scale(ctx context.Context, name string, s ScaleRequest) error {
|
||||
return c.do(ctx, "POST /api/v1/models/{name}/scale", []string{name}, s, nil)
|
||||
}
|
||||
|
||||
// SetClass sets a model's capacity class; empty goes back to its default.
|
||||
func (c *Client) SetClass(ctx context.Context, name, class string) error {
|
||||
return c.do(ctx, "POST /api/v1/models/{name}/class", []string{name}, ClassRequest{Class: class}, nil)
|
||||
}
|
||||
|
||||
// SetOwner sets a model's owner, by path or ID; empty makes it shared. It
|
||||
// answers with the owner as scope:ID.
|
||||
func (c *Client) SetOwner(ctx context.Context, name, owner string) (string, error) {
|
||||
var out OwnerView
|
||||
err := c.do(ctx, "POST /api/v1/models/{name}/owner", []string{name}, OwnerRequest{Owner: owner}, &out)
|
||||
return out.Owner, err
|
||||
}
|
||||
|
||||
// SetAdapters replaces a model's LoRA adapters.
|
||||
func (c *Client) SetAdapters(ctx context.Context, name string, adapters []Adapter) error {
|
||||
if adapters == nil {
|
||||
adapters = []Adapter{}
|
||||
}
|
||||
return c.do(ctx, "POST /api/v1/models/{name}/adapters", []string{name}, AdaptersRequest{Adapters: adapters}, nil)
|
||||
}
|
||||
|
||||
// --- nodes ---
|
||||
|
||||
// NodeView is a node as the listing shows it.
|
||||
type NodeView struct {
|
||||
ID string `json:"id"`
|
||||
Hostname string `json:"hostname,omitempty"`
|
||||
MgmtAddr string `json:"mgmt_addr,omitempty"`
|
||||
State string `json:"state"`
|
||||
Labels map[string]string `json:"labels,omitempty"`
|
||||
GPUTotal int64 `json:"gpu_total"`
|
||||
GPUFree int64 `json:"gpu_free"`
|
||||
MemoryGB int64 `json:"memory_gb,omitempty"`
|
||||
CPUCount int64 `json:"cpu_count,omitempty"`
|
||||
Engines []string `json:"engines,omitempty"`
|
||||
OS string `json:"os,omitempty"`
|
||||
Classes []string `json:"gpu_classes,omitempty"`
|
||||
Fabrics []string `json:"fabrics,omitempty"`
|
||||
AgentVersion string `json:"agent_version,omitempty"`
|
||||
Replicas int64 `json:"replicas"`
|
||||
}
|
||||
|
||||
// Nodes lists the fleet.
|
||||
func (c *Client) Nodes(ctx context.Context) ([]NodeView, error) {
|
||||
var out []NodeView
|
||||
return out, c.do(ctx, "GET /api/v1/nodes", nil, nil, &out)
|
||||
}
|
||||
|
||||
// --- quotas ---
|
||||
|
||||
// Resources is an amount of the cluster; nil is not governed.
|
||||
type Resources struct {
|
||||
GPUs *int64 `json:"gpus,omitempty"`
|
||||
CPUCores *int64 `json:"cpu_cores,omitempty"`
|
||||
MemoryGB *int64 `json:"memory_gb,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaView is one scope's quota.
|
||||
type QuotaView struct {
|
||||
Scope string `json:"scope"`
|
||||
Name string `json:"name"`
|
||||
Guarantee Resources `json:"guarantee"`
|
||||
Limit Resources `json:"limit"`
|
||||
}
|
||||
|
||||
// QuotaRequest sets a scope's quota, the scope by path or ID.
|
||||
type QuotaRequest struct {
|
||||
Scope string `json:"scope"`
|
||||
Guarantee Resources `json:"guarantee"`
|
||||
Limit Resources `json:"limit"`
|
||||
}
|
||||
|
||||
// QuotaSet is the scope a quota was set on, as scope:ID.
|
||||
type QuotaSet struct {
|
||||
Scope string `json:"scope"`
|
||||
}
|
||||
|
||||
// Quotas lists every capacity quota.
|
||||
func (c *Client) Quotas(ctx context.Context) ([]QuotaView, error) {
|
||||
var out []QuotaView
|
||||
return out, c.do(ctx, "GET /api/v1/quotas", nil, nil, &out)
|
||||
}
|
||||
|
||||
// SetQuota sets a scope's quota whole, and answers with the scope as
|
||||
// scope:ID.
|
||||
func (c *Client) SetQuota(ctx context.Context, q QuotaRequest) (string, error) {
|
||||
var out QuotaSet
|
||||
err := c.do(ctx, "PUT /api/v1/quotas", nil, q, &out)
|
||||
return out.Scope, err
|
||||
}
|
||||
|
||||
// ClearQuota removes a scope's quota.
|
||||
func (c *Client) ClearQuota(ctx context.Context, scope string) error {
|
||||
return c.do(ctx, "DELETE /api/v1/quotas/{scope}", []string{scope}, nil, nil)
|
||||
}
|
||||
|
||||
// --- guardrails ---
|
||||
|
||||
// Attachment is a policy attached to a scope. The API writes its fields by
|
||||
// their Go names.
|
||||
type Attachment struct {
|
||||
Scope string `json:"Scope"`
|
||||
ScopeID string `json:"ScopeID"`
|
||||
Policy string `json:"Policy"`
|
||||
}
|
||||
|
||||
// GuardView is the policy library and its attachments.
|
||||
type GuardView struct {
|
||||
Policies []string `json:"policies"`
|
||||
Default []string `json:"default"`
|
||||
Attachments []Attachment `json:"attachments"`
|
||||
}
|
||||
|
||||
// GuardAttachmentRequest names a policy and where it applies.
|
||||
type GuardAttachmentRequest struct {
|
||||
Target string `json:"target"`
|
||||
Policy string `json:"policy"`
|
||||
}
|
||||
|
||||
// Guard reads the policies and where they are attached.
|
||||
func (c *Client) Guard(ctx context.Context) (GuardView, error) {
|
||||
var out GuardView
|
||||
return out, c.do(ctx, "GET /api/v1/guard", nil, nil, &out)
|
||||
}
|
||||
|
||||
// Attach attaches a policy, answering with where it went by ID.
|
||||
func (c *Client) Attach(ctx context.Context, target, policy string) (Attachment, error) {
|
||||
var out Attachment
|
||||
err := c.do(ctx, "POST /api/v1/guard/attachments", nil, GuardAttachmentRequest{Target: target, Policy: policy}, &out)
|
||||
return out, err
|
||||
}
|
||||
|
||||
// Detach takes a policy off.
|
||||
func (c *Client) Detach(ctx context.Context, target, policy string) error {
|
||||
return c.do(ctx, "DELETE /api/v1/guard/attachments", nil, GuardAttachmentRequest{Target: target, Policy: policy}, nil)
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
package client_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/fakeapi"
|
||||
)
|
||||
|
||||
func serve(t *testing.T) (*fakeapi.API, *client.Client) {
|
||||
t.Helper()
|
||||
api := fakeapi.New("secret", client.NodeView{ID: "gpu01", State: "ready", GPUTotal: 8, GPUFree: 6})
|
||||
srv := httptest.NewServer(api)
|
||||
t.Cleanup(srv.Close)
|
||||
c, err := client.New(srv.URL, "secret", "terraform")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return api, c
|
||||
}
|
||||
|
||||
func TestAModelIsAddedReadScaledAndRemoved(t *testing.T) {
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
err := c.AddModel(ctx, client.Model{Name: "qwen", Source: "/m/qwen.gguf",
|
||||
Profiles: []client.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 4}},
|
||||
Scaling: client.Scaling{Min: 1}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m, err := c.Model(ctx, "qwen")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// As the catalog filled it in.
|
||||
if m.Scaling.Max != 1 || m.Profiles[0].TP != 1 || m.Profiles[0].StartupTimeout != 20*time.Minute {
|
||||
t.Errorf("read back %+v", m)
|
||||
}
|
||||
if err := c.Scale(ctx, "qwen", client.ScaleRequest{Min: 0, Max: 2, IdleTimeout: 30 * time.Minute}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if m, _ := api.Model("qwen"); m.Scaling != (client.Scaling{Min: 0, Max: 2, IdleTimeout: 30 * time.Minute}) {
|
||||
t.Errorf("scaled to %+v", m.Scaling)
|
||||
}
|
||||
if err := c.RemoveModel(ctx, "qwen"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := c.Model(ctx, "qwen"); !client.IsNotFound(err) {
|
||||
t.Errorf("a removed model read as %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryChangeSaysItIsJSONAndWhoMadeIt(t *testing.T) {
|
||||
// The admin listener refuses a change without application/json, a DELETE
|
||||
// with no body included; and the audit log names whoever the change says
|
||||
// it is from.
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
if err := c.AddModel(ctx, client.Model{Name: "m", Source: "/m"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := c.RemoveModel(ctx, "m"); err != nil {
|
||||
t.Fatalf("a DELETE with no body: %v", err)
|
||||
}
|
||||
if got := api.Actors(); !slices.Equal(got, []string{"terraform", "terraform"}) {
|
||||
t.Errorf("actors %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestARefusalCarriesWhatTheAPISaid(t *testing.T) {
|
||||
_, c := serve(t)
|
||||
ctx := context.Background()
|
||||
_, err := c.SetQuota(ctx, client.QuotaRequest{Scope: "team:Nobody/x", Limit: client.Resources{GPUs: ptr(1)}})
|
||||
var e *client.Error
|
||||
if err == nil || !strings.Contains(err.Error(), `there is no team "Nobody/x"`) {
|
||||
t.Fatalf("got %v", err)
|
||||
}
|
||||
if !errors.As(err, &e) || e.Status != http.StatusBadRequest {
|
||||
t.Errorf("not a 400: %#v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWrongTokenIsRefused(t *testing.T) {
|
||||
api := fakeapi.New("secret")
|
||||
srv := httptest.NewServer(api)
|
||||
defer srv.Close()
|
||||
c, err := client.New(srv.URL, "wrong", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var e *client.Error
|
||||
if _, err := c.Models(context.Background()); !errors.As(err, &e) || e.Status != http.StatusUnauthorized {
|
||||
t.Errorf("got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestANameWithASlashIsOneSegment(t *testing.T) {
|
||||
// A quota's scope is named by path, Acme/ml, and must not become two
|
||||
// segments of the URL on its way to DELETE /api/v1/quotas/{scope}.
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
id, err := c.SetQuota(ctx, client.QuotaRequest{Scope: "team:Acme/ml", Guarantee: client.Resources{GPUs: ptr(2)}})
|
||||
if err != nil || id != "team:t1" {
|
||||
t.Fatalf("set %q, %v", id, err)
|
||||
}
|
||||
if err := c.ClearQuota(ctx, "team:Acme/ml"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := api.Quota("team:t1"); ok {
|
||||
t.Error("the quota is still there")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPolicyIsAttachedByPathAndDetachedByID(t *testing.T) {
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
a, err := c.Attach(ctx, "team:Acme/ml", "finance")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if a != (client.Attachment{Scope: "team", ScopeID: "t1", Policy: "finance"}) {
|
||||
t.Errorf("attached %+v", a)
|
||||
}
|
||||
g, err := c.Guard(ctx)
|
||||
if err != nil || len(g.Attachments) != 1 || g.Attachments[0] != a {
|
||||
t.Fatalf("guard %+v, %v", g, err)
|
||||
}
|
||||
if err := c.Detach(ctx, "team:t1", "finance"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(api.Attachments()) != 0 {
|
||||
t.Error("still attached")
|
||||
}
|
||||
if err := c.Detach(ctx, "team:t1", "finance"); !client.IsNotFound(err) {
|
||||
t.Errorf("detaching twice: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheNodesAreListed(t *testing.T) {
|
||||
_, c := serve(t)
|
||||
ns, err := c.Nodes(context.Background())
|
||||
if err != nil || len(ns) != 1 || ns[0].ID != "gpu01" || ns[0].GPUFree != 6 {
|
||||
t.Errorf("nodes %+v, %v", ns, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnEndpointNotListedIsNotCalled(t *testing.T) {
|
||||
// Endpoints is what the contract test checks against the API's document;
|
||||
// a call outside it would be one nothing checks.
|
||||
if slices.Contains(client.Endpoints, "POST /api/v1/restart") {
|
||||
t.Skip("restart is listed now")
|
||||
}
|
||||
if !strings.Contains(client.CallUnlisted(), "add it to Endpoints") {
|
||||
t.Error("an unlisted endpoint was called")
|
||||
}
|
||||
}
|
||||
|
||||
func TestABadEndpointOrNoTokenIsRefusedUpFront(t *testing.T) {
|
||||
if _, err := client.New("127.0.0.1:9090", "t", ""); err == nil {
|
||||
t.Error("an address without a scheme was taken")
|
||||
}
|
||||
if _, err := client.New("http://127.0.0.1:9090", "", ""); err == nil {
|
||||
t.Error("no token was taken")
|
||||
}
|
||||
}
|
||||
|
||||
func ptr(n int64) *int64 { return &n }
|
||||
@@ -0,0 +1,201 @@
|
||||
package client
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The gateway's OpenAPI document is generated from its route table and the
|
||||
// Go types its handlers use, and a test there keeps it current. Checking
|
||||
// this client against it is what keeps the two modules in step without the
|
||||
// provider importing the gateway.
|
||||
|
||||
type document struct {
|
||||
Paths map[string]map[string]operation `json:"paths"`
|
||||
Components struct {
|
||||
Schemas map[string]schemaDoc `json:"schemas"`
|
||||
} `json:"components"`
|
||||
}
|
||||
|
||||
type operation struct {
|
||||
RequestBody *struct {
|
||||
Content map[string]struct {
|
||||
Schema schemaDoc `json:"schema"`
|
||||
} `json:"content"`
|
||||
} `json:"requestBody"`
|
||||
Responses map[string]struct {
|
||||
Content map[string]struct {
|
||||
Schema schemaDoc `json:"schema"`
|
||||
} `json:"content"`
|
||||
} `json:"responses"`
|
||||
}
|
||||
|
||||
type schemaDoc struct {
|
||||
Ref string `json:"$ref"`
|
||||
Type string `json:"type"`
|
||||
Items *schemaDoc `json:"items"`
|
||||
Properties map[string]schemaDoc `json:"properties"`
|
||||
}
|
||||
|
||||
func (s schemaDoc) name() string { return strings.TrimPrefix(s.Ref, "#/components/schemas/") }
|
||||
|
||||
func load(t *testing.T) document {
|
||||
t.Helper()
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("cannot locate this test")
|
||||
}
|
||||
p := filepath.Join(filepath.Dir(file), "..", "..", "..", "docs", "admin-api.openapi.json")
|
||||
b, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
t.Fatalf("the gateway's OpenAPI document: %v", err)
|
||||
}
|
||||
var d document
|
||||
if err := json.Unmarshal(b, &d); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func (d document) op(t *testing.T, endpoint string) (operation, bool) {
|
||||
method, p, _ := strings.Cut(endpoint, " ")
|
||||
op, ok := d.Paths[p][strings.ToLower(method)]
|
||||
return op, ok
|
||||
}
|
||||
|
||||
func TestEveryEndpointTheClientCallsIsInTheDocument(t *testing.T) {
|
||||
d := load(t)
|
||||
for _, e := range Endpoints {
|
||||
if _, ok := d.op(t, e); !ok {
|
||||
t.Errorf("the client calls %s, which the admin API does not serve", e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// bodies is each endpoint's request and answer: the client's type and the
|
||||
// schema the document names for it. Empty is none, or one the client
|
||||
// ignores.
|
||||
var bodies = []struct {
|
||||
endpoint string
|
||||
request any
|
||||
requestSchema string
|
||||
answer any
|
||||
answerSchema string
|
||||
}{
|
||||
{"GET /api/v1/models", nil, "", ModelView{}, "ModelView"},
|
||||
{"POST /api/v1/models", Model{}, "catalog.Model", nil, ""},
|
||||
{"GET /api/v1/models/{name}", nil, "", Model{}, "catalog.Model"},
|
||||
{"POST /api/v1/models/{name}/scale", ScaleRequest{}, "ScaleRequest", nil, ""},
|
||||
{"POST /api/v1/models/{name}/class", ClassRequest{}, "ClassRequest", nil, ""},
|
||||
{"POST /api/v1/models/{name}/owner", OwnerRequest{}, "OwnerRequest", OwnerView{}, "OwnerView"},
|
||||
{"POST /api/v1/models/{name}/adapters", AdaptersRequest{}, "AdaptersRequest", nil, ""},
|
||||
{"GET /api/v1/nodes", nil, "", NodeView{}, "NodeView"},
|
||||
{"GET /api/v1/quotas", nil, "", QuotaView{}, "QuotaView"},
|
||||
{"PUT /api/v1/quotas", QuotaRequest{}, "QuotaRequest", QuotaSet{}, "QuotaSet"},
|
||||
{"GET /api/v1/guard", nil, "", GuardView{}, "GuardView"},
|
||||
{"POST /api/v1/guard/attachments", GuardAttachmentRequest{}, "GuardAttachmentRequest", Attachment{}, "guard.Attachment"},
|
||||
{"DELETE /api/v1/guard/attachments", GuardAttachmentRequest{}, "GuardAttachmentRequest", nil, ""},
|
||||
}
|
||||
|
||||
// nested are the client's types inside those bodies, and their schemas.
|
||||
var nested = map[reflect.Type]string{
|
||||
reflect.TypeOf(Profile{}): "catalog.Profile",
|
||||
reflect.TypeOf(Scaling{}): "catalog.Scaling",
|
||||
reflect.TypeOf(External{}): "catalog.External",
|
||||
reflect.TypeOf(Adapter{}): "catalog.Adapter",
|
||||
reflect.TypeOf(Capabilities{}): "catalog.Capabilities",
|
||||
reflect.TypeOf(Resources{}): "store.Resources",
|
||||
reflect.TypeOf(Attachment{}): "GuardAttachmentView",
|
||||
}
|
||||
|
||||
func TestEveryBodyTheClientSendsAndReadsMatchesTheDocument(t *testing.T) {
|
||||
d := load(t)
|
||||
for _, b := range bodies {
|
||||
op, ok := d.op(t, b.endpoint)
|
||||
if !ok {
|
||||
t.Errorf("%s is not in the document", b.endpoint)
|
||||
continue
|
||||
}
|
||||
if b.request != nil {
|
||||
if op.RequestBody == nil {
|
||||
t.Errorf("%s takes no body in the document", b.endpoint)
|
||||
} else if got := op.RequestBody.Content["application/json"].Schema.name(); got != b.requestSchema {
|
||||
t.Errorf("%s takes %s, not %s", b.endpoint, got, b.requestSchema)
|
||||
}
|
||||
checkType(t, d, reflect.TypeOf(b.request), b.requestSchema)
|
||||
}
|
||||
if b.answer != nil {
|
||||
var s schemaDoc
|
||||
for code, r := range op.Responses {
|
||||
if strings.HasPrefix(code, "2") {
|
||||
s = r.Content["application/json"].Schema
|
||||
}
|
||||
}
|
||||
if s.Type == "array" && s.Items != nil {
|
||||
s = *s.Items
|
||||
}
|
||||
if s.name() != b.answerSchema {
|
||||
t.Errorf("%s answers with %s, not %s", b.endpoint, s.name(), b.answerSchema)
|
||||
}
|
||||
checkType(t, d, reflect.TypeOf(b.answer), b.answerSchema)
|
||||
}
|
||||
}
|
||||
for typ, name := range nested {
|
||||
checkType(t, d, typ, name)
|
||||
}
|
||||
}
|
||||
|
||||
// checkType fails for each of a struct's JSON keys the schema does not have,
|
||||
// or has as another kind of value.
|
||||
func checkType(t *testing.T, d document, typ reflect.Type, schema string) {
|
||||
t.Helper()
|
||||
s, ok := d.Components.Schemas[schema]
|
||||
if !ok {
|
||||
t.Errorf("no schema %s in the document", schema)
|
||||
return
|
||||
}
|
||||
for i := range typ.NumField() {
|
||||
f := typ.Field(i)
|
||||
name, _, _ := strings.Cut(f.Tag.Get("json"), ",")
|
||||
if name == "" {
|
||||
name = f.Name
|
||||
}
|
||||
p, ok := s.Properties[name]
|
||||
if !ok {
|
||||
t.Errorf("%s.%s is %q, which %s does not have", typ.Name(), f.Name, name, schema)
|
||||
continue
|
||||
}
|
||||
if want := kindOf(f.Type); want != "" && p.Type != "" && p.Type != want {
|
||||
t.Errorf("%s.%s is a %s here and a %s in %s", typ.Name(), f.Name, want, p.Type, schema)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func kindOf(t reflect.Type) string {
|
||||
if t == reflect.TypeOf(time.Duration(0)) {
|
||||
return "integer"
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Pointer:
|
||||
return kindOf(t.Elem())
|
||||
case reflect.String:
|
||||
return "string"
|
||||
case reflect.Bool:
|
||||
return "boolean"
|
||||
case reflect.Int, reflect.Int64:
|
||||
return "integer"
|
||||
case reflect.Float64:
|
||||
return "number"
|
||||
case reflect.Slice:
|
||||
return "array"
|
||||
case reflect.Map:
|
||||
return "object"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package client
|
||||
|
||||
import "context"
|
||||
|
||||
// CallUnlisted calls an endpoint that is not in Endpoints, and says what
|
||||
// came of it.
|
||||
func CallUnlisted() string {
|
||||
c := &Client{}
|
||||
err := c.do(context.Background(), "POST /api/v1/restart", nil, nil, nil)
|
||||
if err == nil {
|
||||
return ""
|
||||
}
|
||||
return err.Error()
|
||||
}
|
||||
@@ -0,0 +1,459 @@
|
||||
// Package fakeapi is an admin API in memory, for testing the client and the
|
||||
// provider without a cluster.
|
||||
//
|
||||
// It does what the real one does at the edges the provider depends on: it
|
||||
// demands the token, refuses a change not sent as JSON, fills in the defaults
|
||||
// the catalog fills in, resolves a scope named by path to its ID, and answers
|
||||
// with the same shapes and statuses. It is not the gateway; the OpenAPI
|
||||
// contract test is what ties the client to that.
|
||||
package fakeapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// API is the fake. Its fields may be read by a test while it serves.
|
||||
type API struct {
|
||||
Token string
|
||||
|
||||
mu sync.Mutex
|
||||
models map[string]client.Model
|
||||
order []string
|
||||
quotas map[string]client.QuotaView
|
||||
attachments []client.Attachment
|
||||
nodes []client.NodeView
|
||||
actors []string
|
||||
// scopes maps scope:ID to the path people know it by.
|
||||
scopes map[string]string
|
||||
}
|
||||
|
||||
// New builds a fake with a small tenancy tree — Acme, Acme/ml,
|
||||
// Acme/ml/training — two guard policies and the nodes given.
|
||||
func New(token string, nodes ...client.NodeView) *API {
|
||||
return &API{
|
||||
Token: token,
|
||||
models: map[string]client.Model{},
|
||||
quotas: map[string]client.QuotaView{},
|
||||
nodes: nodes,
|
||||
scopes: map[string]string{
|
||||
"org:o1": "Acme",
|
||||
"team:t1": "Acme / ml",
|
||||
"project:p1": "Acme/ml/training",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Actors are who each change said it was made by, in order.
|
||||
func (a *API) Actors() []string {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
return slices.Clone(a.actors)
|
||||
}
|
||||
|
||||
// Model is a model as the fake holds it, for a test to look at.
|
||||
func (a *API) Model(name string) (client.Model, bool) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m, ok := a.models[name]
|
||||
return m, ok
|
||||
}
|
||||
|
||||
// Quota is a quota as the fake holds it, by scope:ID.
|
||||
func (a *API) Quota(scope string) (client.QuotaView, bool) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
q, ok := a.quotas[scope]
|
||||
return q, ok
|
||||
}
|
||||
|
||||
// Attachments are the guard attachments as the fake holds them.
|
||||
func (a *API) Attachments() []client.Attachment {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
return slices.Clone(a.attachments)
|
||||
}
|
||||
|
||||
// ScaleBehindTerraform changes a model's scaling as someone at a terminal
|
||||
// would, for a test of drift.
|
||||
func (a *API) ScaleBehindTerraform(name string, minimum, maximum int64) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m := a.models[name]
|
||||
m.Scaling.Min, m.Scaling.Max = minimum, maximum
|
||||
a.models[name] = m
|
||||
}
|
||||
|
||||
// RemoveBehindTerraform removes a model as someone at a terminal would.
|
||||
func (a *API) RemoveBehindTerraform(name string) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
delete(a.models, name)
|
||||
a.order = slices.DeleteFunc(a.order, func(n string) bool { return n == name })
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(code)
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
|
||||
func writeError(w http.ResponseWriter, code int, msg string) {
|
||||
writeJSON(w, code, map[string]string{"error": msg})
|
||||
}
|
||||
|
||||
// ServeHTTP checks what the admin listener checks, then routes.
|
||||
func (a *API) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer "+a.Token {
|
||||
writeError(w, http.StatusUnauthorized, "the admin API needs its token")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet {
|
||||
if ct, _, _ := strings.Cut(r.Header.Get("Content-Type"), ";"); ct != "application/json" {
|
||||
http.Error(w, "a change to the admin API must be sent as application/json", http.StatusUnsupportedMediaType)
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
a.actors = append(a.actors, r.Header.Get("X-Llmgw-Actor"))
|
||||
a.mu.Unlock()
|
||||
}
|
||||
a.mux().ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
func (a *API) mux() *http.ServeMux {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /api/v1/models", a.listModels)
|
||||
mux.HandleFunc("POST /api/v1/models", a.addModel)
|
||||
mux.HandleFunc("GET /api/v1/models/{name}", a.getModel)
|
||||
mux.HandleFunc("DELETE /api/v1/models/{name}", a.removeModel)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/scale", a.scale)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/class", a.class)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/owner", a.owner)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/adapters", a.adapters)
|
||||
mux.HandleFunc("GET /api/v1/nodes", func(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
out := slices.Clone(a.nodes)
|
||||
if out == nil {
|
||||
out = []client.NodeView{}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
})
|
||||
mux.HandleFunc("GET /api/v1/quotas", a.listQuotas)
|
||||
mux.HandleFunc("PUT /api/v1/quotas", a.putQuota)
|
||||
mux.HandleFunc("DELETE /api/v1/quotas/{scope}", a.deleteQuota)
|
||||
mux.HandleFunc("GET /api/v1/guard", a.guard)
|
||||
mux.HandleFunc("POST /api/v1/guard/attachments", a.attach(true))
|
||||
mux.HandleFunc("DELETE /api/v1/guard/attachments", a.attach(false))
|
||||
return mux
|
||||
}
|
||||
|
||||
// normalize fills in what the catalog fills in.
|
||||
func normalize(m client.Model) client.Model {
|
||||
if m.Scaling.Max == 0 && m.Scaling.Min > 0 {
|
||||
m.Scaling.Max = m.Scaling.Min
|
||||
}
|
||||
for i := range m.Profiles {
|
||||
p := &m.Profiles[i]
|
||||
if p.Engine == "" {
|
||||
p.Engine = "vllm"
|
||||
}
|
||||
if p.TP <= 0 {
|
||||
p.TP = 1
|
||||
}
|
||||
if p.Nodes <= 0 {
|
||||
p.Nodes = 1
|
||||
}
|
||||
if p.StartupTimeout <= 0 {
|
||||
p.StartupTimeout = 20 * time.Minute
|
||||
}
|
||||
if p.Weight <= 0 {
|
||||
p.Weight = 1
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func (a *API) listModels(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
out := []client.ModelView{}
|
||||
for _, n := range a.order {
|
||||
m := a.models[n]
|
||||
v := client.ModelView{Name: m.Name, Description: m.Description, Aliases: m.Aliases, Source: m.Source,
|
||||
Min: m.Scaling.Min, Max: m.Scaling.Max, Desired: m.Scaling.Min, Class: m.Class, Owner: m.Owner}
|
||||
if v.Class == "" {
|
||||
v.Class = "standard"
|
||||
}
|
||||
for _, p := range m.Profiles {
|
||||
v.Profiles = append(v.Profiles, p.Name)
|
||||
}
|
||||
if m.External != nil {
|
||||
v.External = m.External.URL + " · " + m.External.Model
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (a *API) addModel(w http.ResponseWriter, r *http.Request) {
|
||||
var m client.Model
|
||||
if err := json.NewDecoder(r.Body).Decode(&m); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the model definition is not valid JSON: "+err.Error())
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
if m.Name == "" || (m.Source == "" && m.External == nil) {
|
||||
writeError(w, http.StatusBadRequest, "a model needs a name and a source")
|
||||
return
|
||||
}
|
||||
if _, ok := a.models[m.Name]; ok {
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("model %q is already in the catalog", m.Name))
|
||||
return
|
||||
}
|
||||
a.models[m.Name] = normalize(m)
|
||||
a.order = append(a.order, m.Name)
|
||||
writeJSON(w, http.StatusOK, map[string]string{"added": m.Name})
|
||||
}
|
||||
|
||||
func (a *API) getModel(w http.ResponseWriter, r *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m, ok := a.models[r.PathValue("name")]
|
||||
if !ok {
|
||||
writeError(w, http.StatusNotFound, "no model named "+r.PathValue("name")+" is in the catalog")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
}
|
||||
|
||||
func (a *API) removeModel(w http.ResponseWriter, r *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
name := r.PathValue("name")
|
||||
if _, ok := a.models[name]; !ok {
|
||||
// The real API says this with a 400: the model was not added at
|
||||
// runtime, or is not there at all.
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("model %q was not added at runtime", name))
|
||||
return
|
||||
}
|
||||
delete(a.models, name)
|
||||
a.order = slices.DeleteFunc(a.order, func(n string) bool { return n == name })
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": name})
|
||||
}
|
||||
|
||||
// change applies f to a model, answering 400 when there is none.
|
||||
func (a *API) change(w http.ResponseWriter, r *http.Request, body any, f func(*client.Model) error) (client.Model, bool) {
|
||||
if err := json.NewDecoder(r.Body).Decode(body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return client.Model{}, false
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m, ok := a.models[r.PathValue("name")]
|
||||
if !ok {
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("model %q was not added at runtime", r.PathValue("name")))
|
||||
return client.Model{}, false
|
||||
}
|
||||
if err := f(&m); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return client.Model{}, false
|
||||
}
|
||||
m = normalize(m)
|
||||
a.models[m.Name] = m
|
||||
return m, true
|
||||
}
|
||||
|
||||
func (a *API) scale(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.ScaleRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
if req.Min < 0 || req.Max < 0 {
|
||||
return fmt.Errorf("a replica count cannot be negative")
|
||||
}
|
||||
m.Scaling = client.Scaling{Min: req.Min, Max: req.Max, IdleTimeout: req.IdleTimeout}
|
||||
return nil
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, map[string]any{"model": m.Name, "min": m.Scaling.Min, "max": m.Scaling.Max,
|
||||
"idle_timeout": m.Scaling.IdleTimeout, "on_demand": m.Scaling.Min == 0 && m.Scaling.Max > 0})
|
||||
}
|
||||
}
|
||||
|
||||
func (a *API) class(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.ClassRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
switch req.Class {
|
||||
case "", "critical", "standard", "opportunistic":
|
||||
m.Class = req.Class
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("class %q is not critical, standard or opportunistic", req.Class)
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "class": m.Class})
|
||||
}
|
||||
}
|
||||
|
||||
func (a *API) owner(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.OwnerRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
if req.Owner == "" {
|
||||
m.Owner = ""
|
||||
return nil
|
||||
}
|
||||
key, err := a.resolve(req.Owner)
|
||||
m.Owner = key
|
||||
return err
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, client.OwnerView{Model: m.Name, Owner: m.Owner})
|
||||
}
|
||||
}
|
||||
|
||||
func (a *API) adapters(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.AdaptersRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
m.Adapters = req.Adapters
|
||||
if len(m.Adapters) == 0 {
|
||||
m.Adapters = nil
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, client.AdaptersRequest{Adapters: m.Adapters})
|
||||
}
|
||||
}
|
||||
|
||||
// resolve reads org:X, team:X or project:X, X an ID or a path, as scope:ID.
|
||||
// Must be called holding mu.
|
||||
func (a *API) resolve(s string) (string, error) {
|
||||
kind, what, ok := strings.Cut(s, ":")
|
||||
if !ok || what == "" {
|
||||
return "", fmt.Errorf("%q is not org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT", s)
|
||||
}
|
||||
if _, ok := a.scopes[s]; ok {
|
||||
return s, nil
|
||||
}
|
||||
norm := func(p string) string {
|
||||
parts := strings.Split(p, "/")
|
||||
for i := range parts {
|
||||
parts[i] = strings.ToLower(strings.TrimSpace(parts[i]))
|
||||
}
|
||||
return strings.Join(parts, "/")
|
||||
}
|
||||
for key, name := range a.scopes {
|
||||
if strings.HasPrefix(key, kind+":") && norm(name) == norm(what) {
|
||||
return key, nil
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("there is no %s %q", kind, what)
|
||||
}
|
||||
|
||||
func (a *API) listQuotas(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
out := []client.QuotaView{}
|
||||
for _, q := range a.quotas {
|
||||
out = append(out, q)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Scope < out[j].Scope })
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (a *API) putQuota(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.QuotaRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
key, err := a.resolve(req.Scope)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
empty := func(r client.Resources) bool { return r.GPUs == nil && r.CPUCores == nil && r.MemoryGB == nil }
|
||||
if empty(req.Guarantee) && empty(req.Limit) {
|
||||
writeError(w, http.StatusBadRequest, "a quota needs a guarantee or a limit; to remove one, clear it")
|
||||
return
|
||||
}
|
||||
a.quotas[key] = client.QuotaView{Scope: key, Name: a.scopes[key], Guarantee: req.Guarantee, Limit: req.Limit}
|
||||
writeJSON(w, http.StatusOK, client.QuotaSet{Scope: key})
|
||||
}
|
||||
|
||||
func (a *API) deleteQuota(w http.ResponseWriter, r *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
key, err := a.resolve(r.PathValue("scope"))
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
delete(a.quotas, key)
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": key})
|
||||
}
|
||||
|
||||
func (a *API) guard(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
type view struct {
|
||||
client.Attachment
|
||||
Name string `json:"name,omitempty"`
|
||||
}
|
||||
atts := []view{}
|
||||
for _, at := range a.attachments {
|
||||
atts = append(atts, view{Attachment: at, Name: a.scopes[at.Scope+":"+at.ScopeID]})
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{
|
||||
"policies": []string{"baseline", "finance"}, "default": []string{"baseline"}, "attachments": atts,
|
||||
})
|
||||
}
|
||||
|
||||
func (a *API) attach(on bool) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.GuardAttachmentRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
kind, what, _ := strings.Cut(req.Target, ":")
|
||||
at := client.Attachment{Scope: kind, ScopeID: what, Policy: req.Policy}
|
||||
if kind == "org" || kind == "team" || kind == "project" {
|
||||
key, err := a.resolve(req.Target)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
_, at.ScopeID, _ = strings.Cut(key, ":")
|
||||
} else if kind != "key" && kind != "user" {
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("%q: attach to org, team, project, key or user", kind))
|
||||
return
|
||||
}
|
||||
i := slices.Index(a.attachments, at)
|
||||
switch {
|
||||
case on && req.Policy != "baseline" && req.Policy != "finance":
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("no policy %q in server.yaml's guard.policies", req.Policy))
|
||||
return
|
||||
case on && i < 0:
|
||||
a.attachments = append(a.attachments, at)
|
||||
case !on && i < 0:
|
||||
writeError(w, http.StatusNotFound, "no such attachment")
|
||||
return
|
||||
case !on:
|
||||
a.attachments = slices.Delete(a.attachments, i, i+1)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, at)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,209 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/datasource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/datasource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// --- llmgw_models ---
|
||||
|
||||
type modelsDataSource struct{ c *client.Client }
|
||||
|
||||
func newModelsDataSource() datasource.DataSource { return &modelsDataSource{} }
|
||||
|
||||
type modelsData struct {
|
||||
Models []modelSummary `tfsdk:"models"`
|
||||
}
|
||||
|
||||
type modelSummary struct {
|
||||
Name types.String `tfsdk:"name"`
|
||||
Description types.String `tfsdk:"description"`
|
||||
Source types.String `tfsdk:"source"`
|
||||
Aliases []string `tfsdk:"aliases"`
|
||||
Profiles []string `tfsdk:"profiles"`
|
||||
Class types.String `tfsdk:"class"`
|
||||
Owner types.String `tfsdk:"owner"`
|
||||
External types.String `tfsdk:"external"`
|
||||
MinReplicas types.Int64 `tfsdk:"min_replicas"`
|
||||
MaxReplicas types.Int64 `tfsdk:"max_replicas"`
|
||||
ReadyReplicas types.Int64 `tfsdk:"ready_replicas"`
|
||||
TotalReplicas types.Int64 `tfsdk:"total_replicas"`
|
||||
DesiredReplicas types.Int64 `tfsdk:"desired_replicas"`
|
||||
Parked types.Bool `tfsdk:"parked"`
|
||||
Reason types.String `tfsdk:"reason"`
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Metadata(_ context.Context, req datasource.MetadataRequest, resp *datasource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_models"
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Schema(_ context.Context, _ datasource.SchemaRequest, resp *datasource.SchemaResponse) {
|
||||
str := func(desc string) schema.StringAttribute {
|
||||
return schema.StringAttribute{Computed: true, Description: desc}
|
||||
}
|
||||
num := func(desc string) schema.Int64Attribute {
|
||||
return schema.Int64Attribute{Computed: true, Description: desc}
|
||||
}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "Every model in the catalog, from configuration and added at runtime, with its replicas now.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"models": schema.ListNestedAttribute{Computed: true, Description: "The models, in the catalog's order.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"name": str("The name clients ask for."),
|
||||
"description": str("What it is for."),
|
||||
"source": str("Where its weights are."),
|
||||
"aliases": schema.ListAttribute{Computed: true, ElementType: types.StringType, Description: "Other names it answers to."},
|
||||
"profiles": schema.ListAttribute{Computed: true, ElementType: types.StringType, Description: "Its profiles' names."},
|
||||
"class": str("Its capacity class."),
|
||||
"owner": str("Its owner, scope:ID; empty is shared."),
|
||||
"external": str("Where an external model is served: the provider's host and its name for it."),
|
||||
"min_replicas": num("Replicas kept running."),
|
||||
"max_replicas": num("The most replicas."),
|
||||
"ready_replicas": num("Replicas serving now."),
|
||||
"total_replicas": num("Replicas in any state."),
|
||||
"desired_replicas": num("Replicas it should have now."),
|
||||
"parked": schema.BoolAttribute{Computed: true, Description: "It failed to start too often and is no longer retried."},
|
||||
"reason": str("The last error, when parked."),
|
||||
}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Configure(_ context.Context, req datasource.ConfigureRequest, _ *datasource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
d.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Read(ctx context.Context, _ datasource.ReadRequest, resp *datasource.ReadResponse) {
|
||||
ms, err := d.c.Models(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Listing the models", err.Error())
|
||||
return
|
||||
}
|
||||
out := modelsData{Models: []modelSummary{}}
|
||||
for _, m := range ms {
|
||||
out.Models = append(out.Models, modelSummary{
|
||||
Name: types.StringValue(m.Name), Description: types.StringValue(m.Description), Source: types.StringValue(m.Source),
|
||||
Aliases: nonNil(m.Aliases), Profiles: nonNil(m.Profiles), Class: types.StringValue(m.Class),
|
||||
Owner: types.StringValue(m.Owner), External: types.StringValue(m.External),
|
||||
MinReplicas: types.Int64Value(m.Min), MaxReplicas: types.Int64Value(m.Max), ReadyReplicas: types.Int64Value(m.Ready),
|
||||
TotalReplicas: types.Int64Value(m.Total), DesiredReplicas: types.Int64Value(m.Desired),
|
||||
Parked: types.BoolValue(m.Parked), Reason: types.StringValue(m.Reason),
|
||||
})
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &out)...)
|
||||
}
|
||||
|
||||
// nonNil is an empty list for none, which reads better in an expression than
|
||||
// a null one: length() of it is 0 rather than an error.
|
||||
func nonNil(s []string) []string {
|
||||
if s == nil {
|
||||
return []string{}
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// --- llmgw_nodes ---
|
||||
|
||||
type nodesDataSource struct{ c *client.Client }
|
||||
|
||||
func newNodesDataSource() datasource.DataSource { return &nodesDataSource{} }
|
||||
|
||||
type nodesData struct {
|
||||
Nodes []nodeSummary `tfsdk:"nodes"`
|
||||
}
|
||||
|
||||
type nodeSummary struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Hostname types.String `tfsdk:"hostname"`
|
||||
Address types.String `tfsdk:"address"`
|
||||
State types.String `tfsdk:"state"`
|
||||
Labels map[string]string `tfsdk:"labels"`
|
||||
GPUTotal types.Int64 `tfsdk:"gpu_total"`
|
||||
GPUFree types.Int64 `tfsdk:"gpu_free"`
|
||||
GPUClasses []string `tfsdk:"gpu_classes"`
|
||||
MemoryGB types.Int64 `tfsdk:"memory_gb"`
|
||||
CPUCount types.Int64 `tfsdk:"cpu_count"`
|
||||
Engines []string `tfsdk:"engines"`
|
||||
Fabrics []string `tfsdk:"fabrics"`
|
||||
OS types.String `tfsdk:"os"`
|
||||
AgentVersion types.String `tfsdk:"agent_version"`
|
||||
Replicas types.Int64 `tfsdk:"replicas"`
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Metadata(_ context.Context, req datasource.MetadataRequest, resp *datasource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_nodes"
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Schema(_ context.Context, _ datasource.SchemaRequest, resp *datasource.SchemaResponse) {
|
||||
str := func(desc string) schema.StringAttribute {
|
||||
return schema.StringAttribute{Computed: true, Description: desc}
|
||||
}
|
||||
num := func(desc string) schema.Int64Attribute {
|
||||
return schema.Int64Attribute{Computed: true, Description: desc}
|
||||
}
|
||||
list := func(desc string) schema.ListAttribute {
|
||||
return schema.ListAttribute{Computed: true, ElementType: types.StringType, Description: desc}
|
||||
}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "Every node the control plane knows, with its cards and state now.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"nodes": schema.ListNestedAttribute{Computed: true, Description: "The nodes.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"id": str("The node's ID."),
|
||||
"hostname": str("Its hostname."),
|
||||
"address": str("Its management address."),
|
||||
"state": str("ready, cordoned, draining, unreachable or lost."),
|
||||
"labels": schema.MapAttribute{Computed: true, ElementType: types.StringType, Description: "Its labels."},
|
||||
"gpu_total": num("Its GPUs."),
|
||||
"gpu_free": num("GPUs not allocated."),
|
||||
"gpu_classes": list("The classes of its GPUs, h100-80g."),
|
||||
"memory_gb": num("Its memory, 0 when not reported."),
|
||||
"cpu_count": num("Its CPUs, 0 when not reported."),
|
||||
"engines": list("The inference engines installed."),
|
||||
"fabrics": list("The fabrics its adapters are on."),
|
||||
"os": str("Its operating system and architecture."),
|
||||
"agent_version": str("The agent's version."),
|
||||
"replicas": num("Replicas with a shard on it."),
|
||||
}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Configure(_ context.Context, req datasource.ConfigureRequest, _ *datasource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
d.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Read(ctx context.Context, _ datasource.ReadRequest, resp *datasource.ReadResponse) {
|
||||
ns, err := d.c.Nodes(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Listing the nodes", err.Error())
|
||||
return
|
||||
}
|
||||
out := nodesData{Nodes: []nodeSummary{}}
|
||||
for _, n := range ns {
|
||||
labels := n.Labels
|
||||
if labels == nil {
|
||||
labels = map[string]string{}
|
||||
}
|
||||
out.Nodes = append(out.Nodes, nodeSummary{
|
||||
ID: types.StringValue(n.ID), Hostname: types.StringValue(n.Hostname), Address: types.StringValue(n.MgmtAddr),
|
||||
State: types.StringValue(n.State), Labels: labels,
|
||||
GPUTotal: types.Int64Value(n.GPUTotal), GPUFree: types.Int64Value(n.GPUFree), GPUClasses: nonNil(n.Classes),
|
||||
MemoryGB: types.Int64Value(n.MemoryGB), CPUCount: types.Int64Value(n.CPUCount),
|
||||
Engines: nonNil(n.Engines), Fabrics: nonNil(n.Fabrics), OS: types.StringValue(n.OS),
|
||||
AgentVersion: types.StringValue(n.AgentVersion), Replicas: types.Int64Value(n.Replicas),
|
||||
})
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &out)...)
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
// Package provider is the llmgw Terraform provider: models, quotas and
|
||||
// guardrail attachments managed through a control plane's admin API.
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/datasource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/provider"
|
||||
"github.com/hashicorp/terraform-plugin-framework/provider/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// DefaultEndpoint is the admin listener's default address.
|
||||
const DefaultEndpoint = "http://127.0.0.1:9090"
|
||||
|
||||
// DefaultTokenFile is where the server writes the admin token under systemd.
|
||||
const DefaultTokenFile = "/var/lib/llmgw/admin-token"
|
||||
|
||||
type llmgwProvider struct{ version string }
|
||||
|
||||
// New returns the provider, for providerserver.
|
||||
func New(version string) func() provider.Provider {
|
||||
return func() provider.Provider { return &llmgwProvider{version: version} }
|
||||
}
|
||||
|
||||
type providerConfig struct {
|
||||
Endpoint types.String `tfsdk:"endpoint"`
|
||||
Token types.String `tfsdk:"token"`
|
||||
TokenFile types.String `tfsdk:"token_file"`
|
||||
Actor types.String `tfsdk:"actor"`
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Metadata(_ context.Context, _ provider.MetadataRequest, resp *provider.MetadataResponse) {
|
||||
resp.TypeName = "llmgw"
|
||||
resp.Version = p.version
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Schema(_ context.Context, _ provider.SchemaRequest, resp *provider.SchemaResponse) {
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "Manages an llmgw cluster through its control plane's admin API.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"endpoint": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Description: "The admin listener, " + DefaultEndpoint + " by default, or LLMGW_ADMIN_URL. " +
|
||||
"It answers to an address, not a name; reach a remote one through an SSH tunnel.",
|
||||
},
|
||||
"token": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Sensitive: true,
|
||||
Description: "The admin token, or LLMGW_ADMIN_TOKEN. When neither is set it is read from token_file.",
|
||||
},
|
||||
"token_file": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Description: "A file holding the admin token; " + DefaultTokenFile + " by default.",
|
||||
},
|
||||
"actor": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Description: "Who changes are recorded as made by in the audit log; terraform by default.",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Configure(ctx context.Context, req provider.ConfigureRequest, resp *provider.ConfigureResponse) {
|
||||
var cfg providerConfig
|
||||
resp.Diagnostics.Append(req.Config.Get(ctx, &cfg)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
pick := func(v types.String, env, def string) string {
|
||||
if !v.IsNull() && !v.IsUnknown() && v.ValueString() != "" {
|
||||
return v.ValueString()
|
||||
}
|
||||
if s := os.Getenv(env); env != "" && s != "" {
|
||||
return s
|
||||
}
|
||||
return def
|
||||
}
|
||||
endpoint := pick(cfg.Endpoint, "LLMGW_ADMIN_URL", DefaultEndpoint)
|
||||
token := pick(cfg.Token, "LLMGW_ADMIN_TOKEN", "")
|
||||
if token == "" {
|
||||
file := pick(cfg.TokenFile, "", DefaultTokenFile)
|
||||
b, err := os.ReadFile(file) //nolint:gosec // the file the configuration names
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("No admin token",
|
||||
"Set token or LLMGW_ADMIN_TOKEN, or make "+file+" readable (it is root's): "+err.Error())
|
||||
return
|
||||
}
|
||||
token = strings.TrimSpace(string(b))
|
||||
}
|
||||
c, err := client.New(endpoint, token, pick(cfg.Actor, "", "terraform"))
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Cannot reach the admin API", err.Error())
|
||||
return
|
||||
}
|
||||
resp.ResourceData = c
|
||||
resp.DataSourceData = c
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Resources(context.Context) []func() resource.Resource {
|
||||
return []func() resource.Resource{newModelResource, newQuotaResource, newGuardAttachmentResource}
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) DataSources(context.Context) []func() datasource.DataSource {
|
||||
return []func() datasource.DataSource{newModelsDataSource, newNodesDataSource}
|
||||
}
|
||||
|
||||
// clientFrom takes the client Configure built, which is absent while
|
||||
// Terraform validates a configuration before the provider is configured.
|
||||
func clientFrom(data any) (*client.Client, bool) {
|
||||
c, ok := data.(*client.Client)
|
||||
return c, ok && c != nil
|
||||
}
|
||||
@@ -0,0 +1,307 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/providerserver"
|
||||
"github.com/hashicorp/terraform-plugin-go/tfprotov6"
|
||||
"github.com/hashicorp/terraform-plugin-testing/helper/resource"
|
||||
"github.com/hashicorp/terraform-plugin-testing/plancheck"
|
||||
"github.com/hashicorp/terraform-plugin-testing/terraform"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/fakeapi"
|
||||
)
|
||||
|
||||
// These run Terraform itself against the provider and an admin API in
|
||||
// memory, and so need TF_ACC=1 and a terraform binary (on PATH, or
|
||||
// TF_ACC_TERRAFORM_PATH); without TF_ACC they are skipped. Each step's apply
|
||||
// is followed by a plan that must be empty, which is what shows the state
|
||||
// read back does not differ from the configuration on every run.
|
||||
|
||||
var factories = map[string]func() (tfprotov6.ProviderServer, error){
|
||||
"llmgw": providerserver.NewProtocol6WithError(New("test")()),
|
||||
}
|
||||
|
||||
func fake(t *testing.T) (*fakeapi.API, string) {
|
||||
t.Helper()
|
||||
api := fakeapi.New("secret",
|
||||
client.NodeView{ID: "gpu01", State: "ready", GPUTotal: 8, GPUFree: 8, Classes: []string{"h100-80g"}},
|
||||
client.NodeView{ID: "cpu01", State: "cordoned", CPUCount: 32, MemoryGB: 128})
|
||||
srv := httptest.NewServer(api)
|
||||
t.Cleanup(srv.Close)
|
||||
return api, fmt.Sprintf(`
|
||||
provider "llmgw" {
|
||||
endpoint = %q
|
||||
token = "secret"
|
||||
}
|
||||
`, srv.URL)
|
||||
}
|
||||
|
||||
const modelV1 = `
|
||||
resource "llmgw_model" "qwen" {
|
||||
name = "qwen-small"
|
||||
source = "/srv/models/qwen.gguf"
|
||||
aliases = ["qwen"]
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 4
|
||||
cpu_cores = 4
|
||||
}]
|
||||
scaling = {
|
||||
min = 1
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
}
|
||||
`
|
||||
|
||||
// In place: scaling, class and owner.
|
||||
const modelV2 = `
|
||||
resource "llmgw_model" "qwen" {
|
||||
name = "qwen-small"
|
||||
source = "/srv/models/qwen.gguf"
|
||||
aliases = ["qwen"]
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 4
|
||||
cpu_cores = 4
|
||||
}]
|
||||
scaling = {
|
||||
min = 0
|
||||
max = 2
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
class = "opportunistic"
|
||||
owner = "team:Acme/ml"
|
||||
}
|
||||
`
|
||||
|
||||
// A profile changed: the model is replaced.
|
||||
const modelV3 = `
|
||||
resource "llmgw_model" "qwen" {
|
||||
name = "qwen-small"
|
||||
source = "/srv/models/qwen.gguf"
|
||||
aliases = ["qwen"]
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 6
|
||||
cpu_cores = 4
|
||||
}]
|
||||
scaling = {
|
||||
min = 0
|
||||
max = 2
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
class = "opportunistic"
|
||||
owner = "team:Acme/ml"
|
||||
}
|
||||
`
|
||||
|
||||
func TestAccAModelIsAddedChangedInPlaceReplacedAndRemoved(t *testing.T) {
|
||||
api, cfg := fake(t)
|
||||
inCluster := func(check func(client.Model) error) resource.TestCheckFunc {
|
||||
return func(*terraform.State) error {
|
||||
m, ok := api.Model("qwen-small")
|
||||
if !ok {
|
||||
return fmt.Errorf("qwen-small is not in the catalog")
|
||||
}
|
||||
return check(m)
|
||||
}
|
||||
}
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
CheckDestroy: func(*terraform.State) error {
|
||||
if _, ok := api.Model("qwen-small"); ok {
|
||||
return fmt.Errorf("qwen-small is still in the catalog")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + modelV1,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "id", "qwen-small"),
|
||||
// Filled in by the catalog: a max not written follows min.
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "scaling.max", "1"),
|
||||
// As written, not as the catalog defaulted it.
|
||||
resource.TestCheckNoResourceAttr("llmgw_model.qwen", "profiles.0.tp"),
|
||||
inCluster(func(m client.Model) error {
|
||||
if m.Scaling.IdleTimeout != 30*time.Minute || m.Profiles[0].MinMemoryGB != 4 {
|
||||
return fmt.Errorf("added as %+v", m)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
),
|
||||
},
|
||||
{
|
||||
Config: cfg + modelV2,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionUpdate),
|
||||
}},
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "owner", "team:Acme/ml"),
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "owner_id", "team:t1"),
|
||||
inCluster(func(m client.Model) error {
|
||||
if m.Scaling.Min != 0 || m.Scaling.Max != 2 || m.Class != "opportunistic" || m.Owner != "team:t1" {
|
||||
return fmt.Errorf("changed to %+v", m)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
),
|
||||
},
|
||||
{
|
||||
// Scaled from a terminal: the next plan puts it back, in place.
|
||||
PreConfig: func() { api.ScaleBehindTerraform("qwen-small", 3, 3) },
|
||||
Config: cfg + modelV2,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionUpdate),
|
||||
}},
|
||||
Check: inCluster(func(m client.Model) error {
|
||||
if m.Scaling.Min != 0 || m.Scaling.Max != 2 {
|
||||
return fmt.Errorf("not put back: %+v", m.Scaling)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
},
|
||||
{
|
||||
Config: cfg + modelV3,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionDestroyBeforeCreate),
|
||||
}},
|
||||
Check: inCluster(func(m client.Model) error {
|
||||
if m.Profiles[0].MinMemoryGB != 6 || m.Owner != "team:t1" {
|
||||
return fmt.Errorf("replaced as %+v", m)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
},
|
||||
{
|
||||
// Removed from a terminal: planned again.
|
||||
PreConfig: func() { api.RemoveBehindTerraform("qwen-small") },
|
||||
Config: cfg + modelV3,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionCreate),
|
||||
}},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestAccAQuotaIsSetChangedImportedAndCleared(t *testing.T) {
|
||||
api, cfg := fake(t)
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
CheckDestroy: func(*terraform.State) error {
|
||||
if _, ok := api.Quota("team:t1"); ok {
|
||||
return fmt.Errorf("the quota is still set")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + `
|
||||
resource "llmgw_quota" "ml" {
|
||||
scope = "team:Acme/ml"
|
||||
limit = { gpus = 8 }
|
||||
}
|
||||
`,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_quota.ml", "id", "team:t1"),
|
||||
resource.TestCheckResourceAttr("llmgw_quota.ml", "name", "Acme / ml"),
|
||||
resource.TestCheckResourceAttr("llmgw_quota.ml", "limit.gpus", "8"),
|
||||
),
|
||||
},
|
||||
{
|
||||
Config: cfg + `
|
||||
resource "llmgw_quota" "ml" {
|
||||
scope = "team:Acme/ml"
|
||||
guarantee = { gpus = 2 }
|
||||
limit = { gpus = 8, memory_gb = 256 }
|
||||
}
|
||||
`,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_quota.ml", plancheck.ResourceActionUpdate),
|
||||
}},
|
||||
Check: func(*terraform.State) error {
|
||||
q, ok := api.Quota("team:t1")
|
||||
if !ok || q.Guarantee.GPUs == nil || *q.Guarantee.GPUs != 2 || q.Limit.MemoryGB == nil || *q.Limit.MemoryGB != 256 {
|
||||
return fmt.Errorf("quota %+v", q)
|
||||
}
|
||||
return nil
|
||||
},
|
||||
},
|
||||
{
|
||||
ResourceName: "llmgw_quota.ml",
|
||||
ImportState: true,
|
||||
ImportStateId: "team:t1",
|
||||
ImportStateVerify: true,
|
||||
// Imported by its ID, which is then the only name it has.
|
||||
ImportStateVerifyIgnore: []string{"scope"},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestAccAPolicyIsAttachedImportedAndDetached(t *testing.T) {
|
||||
api, cfg := fake(t)
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
CheckDestroy: func(*terraform.State) error {
|
||||
if n := len(api.Attachments()); n != 0 {
|
||||
return fmt.Errorf("%d attachments left", n)
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + `
|
||||
resource "llmgw_guard_attachment" "ml" {
|
||||
target = "team:Acme/ml"
|
||||
policy = "finance"
|
||||
}
|
||||
`,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_guard_attachment.ml", "id", "team:t1:finance"),
|
||||
resource.TestCheckResourceAttr("llmgw_guard_attachment.ml", "scope_id", "t1"),
|
||||
),
|
||||
},
|
||||
{
|
||||
ResourceName: "llmgw_guard_attachment.ml",
|
||||
ImportState: true,
|
||||
ImportStateVerify: true,
|
||||
ImportStateVerifyIgnore: []string{"target"},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestAccTheModelsAndNodesAreRead(t *testing.T) {
|
||||
_, cfg := fake(t)
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + modelV1 + `
|
||||
data "llmgw_nodes" "all" {}
|
||||
data "llmgw_models" "all" {
|
||||
depends_on = [llmgw_model.qwen]
|
||||
}
|
||||
`,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("data.llmgw_nodes.all", "nodes.#", "2"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_nodes.all", "nodes.0.gpu_classes.0", "h100-80g"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_nodes.all", "nodes.1.state", "cordoned"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_models.all", "models.#", "1"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_models.all", "models.0.name", "qwen-small"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_models.all", "models.0.profiles.0", "cpu"),
|
||||
),
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// The guard attachment resource is a guardrail policy attached to a part of
|
||||
// the tenancy tree, a key or a person. The policies themselves are defined
|
||||
// in server.yaml's guard section, not here: what this manages is where they
|
||||
// apply.
|
||||
|
||||
type guardAttachmentResource struct{ c *client.Client }
|
||||
|
||||
func newGuardAttachmentResource() resource.Resource { return &guardAttachmentResource{} }
|
||||
|
||||
type guardAttachmentModel struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Target types.String `tfsdk:"target"`
|
||||
Policy types.String `tfsdk:"policy"`
|
||||
Scope types.String `tfsdk:"scope"`
|
||||
ScopeID types.String `tfsdk:"scope_id"`
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_guard_attachment"
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Schema(_ context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {
|
||||
replace := []planmodifier.String{stringplanmodifier.RequiresReplace()}
|
||||
keep := []planmodifier.String{stringplanmodifier.UseStateForUnknown()}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "A guardrail policy from server.yaml's guard.policies, attached to an organization, team, project, key or person.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"id": schema.StringAttribute{Computed: true, PlanModifiers: keep,
|
||||
Description: "scope:SCOPE-ID:policy. Import by it."},
|
||||
"target": schema.StringAttribute{Required: true, PlanModifiers: replace,
|
||||
Description: "org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT (or the ID after the colon), key:KEY-ID or user:USER-ID."},
|
||||
"policy": schema.StringAttribute{Required: true, PlanModifiers: replace,
|
||||
Description: "The policy's name in server.yaml's guard.policies."},
|
||||
"scope": schema.StringAttribute{Computed: true, PlanModifiers: keep,
|
||||
Description: "org, team, project, key or user."},
|
||||
"scope_id": schema.StringAttribute{Computed: true, PlanModifiers: keep,
|
||||
Description: "The ID the target resolved to."},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Configure(_ context.Context, req resource.ConfigureRequest, _ *resource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
r.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func attachmentID(a client.Attachment) string { return a.Scope + ":" + a.ScopeID + ":" + a.Policy }
|
||||
|
||||
func (r *guardAttachmentResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
|
||||
var plan guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
a, err := r.c.Attach(ctx, plan.Target.ValueString(), plan.Policy.ValueString())
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Attaching "+plan.Policy.ValueString()+" to "+plan.Target.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
plan.ID = types.StringValue(attachmentID(a))
|
||||
plan.Scope = types.StringValue(a.Scope)
|
||||
plan.ScopeID = types.StringValue(a.ScopeID)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
|
||||
var st guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
g, err := r.c.Guard(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the guard attachments", err.Error())
|
||||
return
|
||||
}
|
||||
for _, a := range g.Attachments {
|
||||
if attachmentID(a) == st.ID.ValueString() {
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
return
|
||||
}
|
||||
}
|
||||
resp.State.RemoveResource(ctx)
|
||||
}
|
||||
|
||||
// Update is never called: every attribute replaces the attachment.
|
||||
func (r *guardAttachmentResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
|
||||
var plan guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
|
||||
var st guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
// By the ID it resolved to, which still names it if the team has since
|
||||
// been renamed.
|
||||
err := r.c.Detach(ctx, st.Scope.ValueString()+":"+st.ScopeID.ValueString(), st.Policy.ValueString())
|
||||
if err != nil && !client.IsNotFound(err) {
|
||||
resp.Diagnostics.AddError("Detaching "+st.Policy.ValueString()+" from "+st.Target.ValueString(), err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) {
|
||||
// scope:SCOPE-ID:policy. The policy is after the last colon and the scope
|
||||
// before the first; an ID between them is whatever is left.
|
||||
scope, rest, ok1 := strings.Cut(req.ID, ":")
|
||||
i := strings.LastIndex(rest, ":")
|
||||
if !ok1 || i <= 0 || i == len(rest)-1 {
|
||||
resp.Diagnostics.AddError("Not an attachment ID", fmt.Sprintf("%q is not scope:SCOPE-ID:policy, as id shows it", req.ID))
|
||||
return
|
||||
}
|
||||
id, policy := rest[:i], rest[i+1:]
|
||||
st := guardAttachmentModel{
|
||||
ID: types.StringValue(req.ID), Target: types.StringValue(scope + ":" + id), Policy: types.StringValue(policy),
|
||||
Scope: types.StringValue(scope), ScopeID: types.StringValue(id),
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
}
|
||||
@@ -0,0 +1,424 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/diag"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/int64planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/listplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/objectplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// The model resource is a catalog entry added at runtime, as `llmgw ctl
|
||||
// model add` adds one.
|
||||
//
|
||||
// What the admin API can change in place — scaling, class, owner, adapters —
|
||||
// is changed in place; anything else in the definition replaces the model,
|
||||
// because the API has no way to edit it and removing and adding is what an
|
||||
// operator would do by hand. A replacement stops the model's replicas.
|
||||
//
|
||||
// The state keeps the definition as it was written rather than as the
|
||||
// catalog filled it in, so a profile written without tp does not differ on
|
||||
// every plan from one the catalog gave tp 1. Only what can change in place is
|
||||
// read back from the cluster, where a change made from a terminal shows as a
|
||||
// difference to put right. A model removed from the cluster is planned again.
|
||||
|
||||
type modelResource struct{ c *client.Client }
|
||||
|
||||
func newModelResource() resource.Resource { return &modelResource{} }
|
||||
|
||||
type modelResourceModel struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Name types.String `tfsdk:"name"`
|
||||
Source types.String `tfsdk:"source"`
|
||||
Description types.String `tfsdk:"description"`
|
||||
Aliases types.List `tfsdk:"aliases"`
|
||||
Fallbacks types.List `tfsdk:"fallbacks"`
|
||||
RoutingProfile types.String `tfsdk:"routing_profile"`
|
||||
ContextLength types.Int64 `tfsdk:"context_length"`
|
||||
MaxOutput types.Int64 `tfsdk:"max_output"`
|
||||
Class types.String `tfsdk:"class"`
|
||||
Owner types.String `tfsdk:"owner"`
|
||||
OwnerID types.String `tfsdk:"owner_id"`
|
||||
Capabilities *capabilitiesModel `tfsdk:"capabilities"`
|
||||
External *externalModel `tfsdk:"external"`
|
||||
Profiles []profileModel `tfsdk:"profiles"`
|
||||
Scaling *scalingModel `tfsdk:"scaling"`
|
||||
Adapters []adapterModel `tfsdk:"adapters"`
|
||||
}
|
||||
|
||||
type capabilitiesModel struct {
|
||||
Tools types.Bool `tfsdk:"tools"`
|
||||
Vision types.Bool `tfsdk:"vision"`
|
||||
Embeddings types.Bool `tfsdk:"embeddings"`
|
||||
}
|
||||
|
||||
type externalModel struct {
|
||||
URL types.String `tfsdk:"url"`
|
||||
Model types.String `tfsdk:"model"`
|
||||
APIKeyFile types.String `tfsdk:"api_key_file"`
|
||||
}
|
||||
|
||||
type profileModel struct {
|
||||
Name types.String `tfsdk:"name"`
|
||||
Engine types.String `tfsdk:"engine"`
|
||||
GPUClass types.String `tfsdk:"gpu_class"`
|
||||
TP types.Int64 `tfsdk:"tp"`
|
||||
PP types.Int64 `tfsdk:"pp"`
|
||||
GPUsPerNode types.Int64 `tfsdk:"gpus_per_node"`
|
||||
Nodes types.Int64 `tfsdk:"nodes"`
|
||||
Args types.List `tfsdk:"args"`
|
||||
Env types.Map `tfsdk:"env"`
|
||||
MinMemoryGB types.Int64 `tfsdk:"min_memory_gb"`
|
||||
CPUCores types.Int64 `tfsdk:"cpu_cores"`
|
||||
MinDiskGB types.Int64 `tfsdk:"min_disk_gb"`
|
||||
NodeSelector types.Map `tfsdk:"node_selector"`
|
||||
Weight types.Float64 `tfsdk:"weight"`
|
||||
Priority types.Int64 `tfsdk:"priority"`
|
||||
StartupTimeout types.String `tfsdk:"startup_timeout"`
|
||||
}
|
||||
|
||||
type scalingModel struct {
|
||||
Min types.Int64 `tfsdk:"min"`
|
||||
Max types.Int64 `tfsdk:"max"`
|
||||
IdleTimeout types.String `tfsdk:"idle_timeout"`
|
||||
}
|
||||
|
||||
type adapterModel struct {
|
||||
Name types.String `tfsdk:"name"`
|
||||
Source types.String `tfsdk:"source"`
|
||||
Scale types.Float64 `tfsdk:"scale"`
|
||||
}
|
||||
|
||||
func (r *modelResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_model"
|
||||
}
|
||||
|
||||
func (r *modelResource) Schema(_ context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {
|
||||
replace := []planmodifier.String{stringplanmodifier.RequiresReplace()}
|
||||
replaceInt := []planmodifier.Int64{int64planmodifier.RequiresReplace()}
|
||||
replaceList := []planmodifier.List{listplanmodifier.RequiresReplace()}
|
||||
replaceObject := []planmodifier.Object{objectplanmodifier.RequiresReplace()}
|
||||
str := func(desc string) schema.StringAttribute {
|
||||
return schema.StringAttribute{Optional: true, Description: desc}
|
||||
}
|
||||
num := func(desc string) schema.Int64Attribute {
|
||||
return schema.Int64Attribute{Optional: true, Description: desc}
|
||||
}
|
||||
strMap := func(desc string) schema.MapAttribute {
|
||||
return schema.MapAttribute{Optional: true, ElementType: types.StringType, Description: desc}
|
||||
}
|
||||
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "A model in the catalog, added at runtime and kept in the store, as `llmgw ctl model add` adds one. " +
|
||||
"Scaling, class, owner and adapters change in place; any other change replaces the model, stopping its replicas. " +
|
||||
"A model declared in the control plane's configuration cannot be managed here.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"id": schema.StringAttribute{Computed: true, Description: "The model's name.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
|
||||
"name": schema.StringAttribute{Required: true, PlanModifiers: replace,
|
||||
Description: "The name clients ask for."},
|
||||
"source": schema.StringAttribute{Optional: true, PlanModifiers: replace,
|
||||
Description: "Where the weights are: a path on the nodes, an https URL, or a registry repository. Absent for an external model."},
|
||||
"description": schema.StringAttribute{Optional: true, PlanModifiers: replace,
|
||||
Description: "What the model is for."},
|
||||
"aliases": schema.ListAttribute{Optional: true, ElementType: types.StringType, PlanModifiers: replaceList,
|
||||
Description: "Other names it answers to."},
|
||||
"fallbacks": schema.ListAttribute{Optional: true, ElementType: types.StringType, PlanModifiers: replaceList,
|
||||
Description: "Models tried in order when this one cannot answer."},
|
||||
"routing_profile": schema.StringAttribute{Optional: true, PlanModifiers: replace,
|
||||
Description: "The routing profile, from the router's configuration."},
|
||||
"context_length": schema.Int64Attribute{Optional: true, PlanModifiers: replaceInt,
|
||||
Description: "The context window, in tokens."},
|
||||
"max_output": schema.Int64Attribute{Optional: true, PlanModifiers: replaceInt,
|
||||
Description: "The most tokens one answer may have."},
|
||||
"class": schema.StringAttribute{Optional: true,
|
||||
Description: "The capacity class: critical, standard or opportunistic. Absent is what the scaling implies."},
|
||||
"owner": schema.StringAttribute{Optional: true,
|
||||
Description: "The part of the tenancy tree the model's capacity counts against: org:NAME, team:ORG/TEAM, " +
|
||||
"project:ORG/TEAM/PROJECT, or their IDs. Absent is shared."},
|
||||
"owner_id": schema.StringAttribute{Computed: true,
|
||||
Description: "The owner as the cluster records it, scope:ID."},
|
||||
"capabilities": schema.SingleNestedAttribute{Optional: true, PlanModifiers: replaceObject,
|
||||
Description: "What the model can do beyond chat.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"tools": schema.BoolAttribute{Optional: true, Description: "It calls tools."},
|
||||
"vision": schema.BoolAttribute{Optional: true, Description: "It reads images."},
|
||||
"embeddings": schema.BoolAttribute{Optional: true, Description: "It answers /v1/embeddings."},
|
||||
}},
|
||||
"external": schema.SingleNestedAttribute{Optional: true, PlanModifiers: replaceObject,
|
||||
Description: "A model served by a provider outside the cluster, in place of source and profiles.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"url": schema.StringAttribute{Required: true, Description: "The provider's API, https://api.openai.com/v1."},
|
||||
"model": schema.StringAttribute{Required: true, Description: "The provider's name for the model."},
|
||||
"api_key_file": schema.StringAttribute{Optional: true, Description: "The provider's key: a file on the control plane, or an env: or vault: reference."},
|
||||
}},
|
||||
"profiles": schema.ListNestedAttribute{Optional: true, PlanModifiers: replaceList,
|
||||
Description: "How the model runs: one profile per kind of node, tried in order. " +
|
||||
"What is left out takes the catalog's default, as in server.yaml.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"name": str("The profile's name."),
|
||||
"engine": str("vllm, sglang or llamacpp; vllm when absent."),
|
||||
"gpu_class": str("The GPU class it runs on, h100-80g; absent runs on CPUs."),
|
||||
"tp": num("Tensor parallelism."),
|
||||
"pp": num("Pipeline parallelism."),
|
||||
"gpus_per_node": num("GPUs on each node."),
|
||||
"nodes": num("Nodes one replica spans."),
|
||||
"args": schema.ListAttribute{Optional: true, ElementType: types.StringType, Description: "Arguments added to the engine's."},
|
||||
"env": strMap("Environment for the engine."),
|
||||
"min_memory_gb": num("Host memory one replica reserves, in GB."),
|
||||
"cpu_cores": num("CPU cores one replica without a GPU computes on."),
|
||||
"min_disk_gb": num("Free disk a node needs for the weights, in GB."),
|
||||
"node_selector": strMap("Labels a node must have."),
|
||||
"weight": schema.Float64Attribute{Optional: true, Description: "The share of requests this profile's replicas take."},
|
||||
"priority": num("Placement order among profiles."),
|
||||
"startup_timeout": str("How long a replica may take to start, 20m when absent."),
|
||||
}},
|
||||
},
|
||||
"scaling": schema.SingleNestedAttribute{Optional: true,
|
||||
Description: "How many replicas run. Absent is none: the model is in the catalog and not running. " +
|
||||
"Changes in place.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"min": schema.Int64Attribute{Optional: true, Computed: true, Description: "Replicas kept running; 0 is on demand."},
|
||||
"max": schema.Int64Attribute{Optional: true, Computed: true, Description: "The most replicas; min when absent."},
|
||||
"idle_timeout": schema.StringAttribute{Optional: true, Computed: true,
|
||||
Description: "How long an on-demand model may go unasked-for before it is stopped, 30m; the cluster's default when absent."},
|
||||
}},
|
||||
"adapters": schema.ListNestedAttribute{Optional: true,
|
||||
Description: "LoRA adapters it serves, each a name clients ask for. Changes in place; each new file is checked first.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"name": schema.StringAttribute{Required: true, Description: "The name clients ask for."},
|
||||
"source": schema.StringAttribute{Required: true, Description: "Where the adapter's file is."},
|
||||
"scale": schema.Float64Attribute{Optional: true, Description: "How strongly it applies, 1 when absent."},
|
||||
}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (r *modelResource) Configure(_ context.Context, req resource.ConfigureRequest, _ *resource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
r.c = c
|
||||
}
|
||||
}
|
||||
|
||||
// definition is the model to add, from a plan. The owner is left out: it is
|
||||
// set afterwards through its own endpoint, which resolves a path to an ID.
|
||||
func definition(ctx context.Context, p modelResourceModel, diags *diag.Diagnostics) client.Model {
|
||||
m := client.Model{
|
||||
Name: p.Name.ValueString(), Source: p.Source.ValueString(), Description: p.Description.ValueString(),
|
||||
Aliases: listOf(ctx, p.Aliases, diags), Fallbacks: listOf(ctx, p.Fallbacks, diags),
|
||||
RoutingProfile: p.RoutingProfile.ValueString(), ContextLength: p.ContextLength.ValueInt64(),
|
||||
MaxOutput: p.MaxOutput.ValueInt64(), Class: p.Class.ValueString(),
|
||||
Adapters: adapters(p.Adapters),
|
||||
}
|
||||
if c := p.Capabilities; c != nil {
|
||||
m.Capabilities = &client.Capabilities{Tools: c.Tools.ValueBool(), Vision: c.Vision.ValueBool(), Embeddings: c.Embeddings.ValueBool()}
|
||||
}
|
||||
if e := p.External; e != nil {
|
||||
m.External = &client.External{URL: e.URL.ValueString(), Model: e.Model.ValueString(), APIKeyFile: e.APIKeyFile.ValueString()}
|
||||
}
|
||||
for _, pr := range p.Profiles {
|
||||
m.Profiles = append(m.Profiles, client.Profile{
|
||||
Name: pr.Name.ValueString(), Engine: pr.Engine.ValueString(), GPUClass: pr.GPUClass.ValueString(),
|
||||
TP: pr.TP.ValueInt64(), PP: pr.PP.ValueInt64(), GPUsPerNode: pr.GPUsPerNode.ValueInt64(), Nodes: pr.Nodes.ValueInt64(),
|
||||
Args: listOf(ctx, pr.Args, diags), Env: mapOf(ctx, pr.Env, diags),
|
||||
MinMemoryGB: pr.MinMemoryGB.ValueInt64(), CPUCores: pr.CPUCores.ValueInt64(), MinDiskGB: pr.MinDiskGB.ValueInt64(),
|
||||
NodeSelector: mapOf(ctx, pr.NodeSelector, diags), Weight: pr.Weight.ValueFloat64(), Priority: pr.Priority.ValueInt64(),
|
||||
StartupTimeout: duration(pr.StartupTimeout, "startup_timeout", diags),
|
||||
})
|
||||
}
|
||||
m.Scaling = client.Scaling(scaleRequest(p.Scaling, diags))
|
||||
return m
|
||||
}
|
||||
|
||||
func adapters(in []adapterModel) []client.Adapter {
|
||||
var out []client.Adapter
|
||||
for _, a := range in {
|
||||
out = append(out, client.Adapter{Name: a.Name.ValueString(), Source: a.Source.ValueString(), Scale: a.Scale.ValueFloat64()})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// scaleRequest is the scaling a plan asks for. What is unknown is left for
|
||||
// the cluster to fill in: a max not written follows min.
|
||||
func scaleRequest(s *scalingModel, diags *diag.Diagnostics) client.ScaleRequest {
|
||||
if s == nil {
|
||||
return client.ScaleRequest{}
|
||||
}
|
||||
return client.ScaleRequest{Min: s.Min.ValueInt64(), Max: s.Max.ValueInt64(),
|
||||
IdleTimeout: duration(s.IdleTimeout, "scaling.idle_timeout", diags)}
|
||||
}
|
||||
|
||||
// refresh takes what can change in place from the cluster's copy. adopt is
|
||||
// set when reading: a scaling block the configuration left out is taken
|
||||
// from the cluster when the cluster has one, so a model scaled from a
|
||||
// terminal shows the difference.
|
||||
func refresh(m client.Model, st *modelResourceModel, adopt bool) {
|
||||
st.ID = types.StringValue(m.Name)
|
||||
switch {
|
||||
case st.Scaling != nil:
|
||||
st.Scaling.Min = types.Int64Value(m.Scaling.Min)
|
||||
st.Scaling.Max = types.Int64Value(m.Scaling.Max)
|
||||
st.Scaling.IdleTimeout = keepDuration(st.Scaling.IdleTimeout, m.Scaling.IdleTimeout)
|
||||
case adopt && m.Scaling != (client.Scaling{}):
|
||||
st.Scaling = &scalingModel{Min: types.Int64Value(m.Scaling.Min), Max: types.Int64Value(m.Scaling.Max),
|
||||
IdleTimeout: types.StringValue(m.Scaling.IdleTimeout.String())}
|
||||
}
|
||||
st.Class = strOrNull(m.Class)
|
||||
if m.Owner != st.OwnerID.ValueString() {
|
||||
st.Owner = strOrNull(m.Owner)
|
||||
}
|
||||
st.OwnerID = strOrNull(m.Owner)
|
||||
switch {
|
||||
case len(m.Adapters) > 0:
|
||||
st.Adapters = nil
|
||||
for _, a := range m.Adapters {
|
||||
st.Adapters = append(st.Adapters, adapterModel{Name: types.StringValue(a.Name),
|
||||
Source: types.StringValue(a.Source), Scale: floatOrNull(a.Scale)})
|
||||
}
|
||||
case st.Adapters != nil:
|
||||
st.Adapters = []adapterModel{}
|
||||
}
|
||||
}
|
||||
|
||||
func (r *modelResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
|
||||
var plan modelResourceModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
m := definition(ctx, plan, &resp.Diagnostics)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
name := m.Name
|
||||
if err := r.c.AddModel(ctx, m); err != nil {
|
||||
resp.Diagnostics.AddError("Adding the model "+name, err.Error())
|
||||
return
|
||||
}
|
||||
plan.OwnerID = types.StringNull()
|
||||
if owner := plan.Owner.ValueString(); owner != "" {
|
||||
id, err := r.c.SetOwner(ctx, name, owner)
|
||||
if err != nil {
|
||||
// Taken out again rather than left half made: Terraform would
|
||||
// not know it exists, and the next apply would fail on the name.
|
||||
_ = r.c.RemoveModel(ctx, name)
|
||||
resp.Diagnostics.AddError("Setting the owner of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
plan.OwnerID = strOrNull(id)
|
||||
}
|
||||
got, err := r.c.Model(ctx, name)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the model "+name+" back", err.Error())
|
||||
return
|
||||
}
|
||||
refresh(got, &plan, false)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *modelResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
|
||||
var st modelResourceModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
m, err := r.c.Model(ctx, st.Name.ValueString())
|
||||
if client.IsNotFound(err) {
|
||||
resp.State.RemoveResource(ctx)
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the model "+st.Name.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
refresh(m, &st, true)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
}
|
||||
|
||||
func (r *modelResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
|
||||
var plan, st modelResourceModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
name := plan.Name.ValueString()
|
||||
|
||||
want, had := scaleRequest(plan.Scaling, &resp.Diagnostics), scaleRequest(st.Scaling, &resp.Diagnostics)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if want != had || (plan.Scaling == nil) != (st.Scaling == nil) {
|
||||
if err := r.c.Scale(ctx, name, want); err != nil {
|
||||
resp.Diagnostics.AddError("Scaling "+name, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
if !plan.Class.Equal(st.Class) {
|
||||
if err := r.c.SetClass(ctx, name, plan.Class.ValueString()); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the class of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
plan.OwnerID = st.OwnerID
|
||||
if !plan.Owner.Equal(st.Owner) {
|
||||
id, err := r.c.SetOwner(ctx, name, plan.Owner.ValueString())
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Setting the owner of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
plan.OwnerID = strOrNull(id)
|
||||
}
|
||||
if !sameAdapters(plan.Adapters, st.Adapters) {
|
||||
if err := r.c.SetAdapters(ctx, name, adapters(plan.Adapters)); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the adapters of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
got, err := r.c.Model(ctx, name)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the model "+name+" back", err.Error())
|
||||
return
|
||||
}
|
||||
refresh(got, &plan, false)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func sameAdapters(a, b []adapterModel) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for i := range a {
|
||||
if !a[i].Name.Equal(b[i].Name) || !a[i].Source.Equal(b[i].Source) || a[i].Scale.ValueFloat64() != b[i].Scale.ValueFloat64() {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (r *modelResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
|
||||
var st modelResourceModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
name := st.Name.ValueString()
|
||||
if err := r.c.RemoveModel(ctx, name); err != nil {
|
||||
// The API refuses removing a model that is not there with the same
|
||||
// status as one it may not remove; asking tells them apart, and one
|
||||
// already gone is what deleting wanted.
|
||||
if _, gerr := r.c.Model(ctx, name); client.IsNotFound(gerr) {
|
||||
return
|
||||
}
|
||||
resp.Diagnostics.AddError("Removing the model "+name, err.Error())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/path"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// The quota resource is one scope's capacity quota: what an organization,
|
||||
// team or project is guaranteed of the cluster and the most it may hold.
|
||||
//
|
||||
// The scope is kept as written — team:Acme/ml — and beside it, as id, the
|
||||
// scope the cluster resolved it to, team:ID. The quota is found again by the
|
||||
// ID, so renaming the team does not lose it.
|
||||
|
||||
type quotaResource struct{ c *client.Client }
|
||||
|
||||
func newQuotaResource() resource.Resource { return "aResource{} }
|
||||
|
||||
type quotaModel struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Scope types.String `tfsdk:"scope"`
|
||||
Name types.String `tfsdk:"name"`
|
||||
Guarantee *resourcesModel `tfsdk:"guarantee"`
|
||||
Limit *resourcesModel `tfsdk:"limit"`
|
||||
}
|
||||
|
||||
type resourcesModel struct {
|
||||
GPUs types.Int64 `tfsdk:"gpus"`
|
||||
CPUCores types.Int64 `tfsdk:"cpu_cores"`
|
||||
MemoryGB types.Int64 `tfsdk:"memory_gb"`
|
||||
}
|
||||
|
||||
func (r *quotaResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_quota"
|
||||
}
|
||||
|
||||
func (r *quotaResource) Schema(_ context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {
|
||||
amount := func(desc string) schema.SingleNestedAttribute {
|
||||
return schema.SingleNestedAttribute{Optional: true, Description: desc,
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"gpus": schema.Int64Attribute{Optional: true, Description: "GPUs; absent is not governed."},
|
||||
"cpu_cores": schema.Int64Attribute{Optional: true, Description: "CPU cores of replicas without a GPU; absent is not governed."},
|
||||
"memory_gb": schema.Int64Attribute{Optional: true, Description: "Host memory of those replicas, in GB; absent is not governed."},
|
||||
}}
|
||||
}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "A capacity quota on an organization, team or project. At least one of guarantee and limit must be set.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"id": schema.StringAttribute{Computed: true,
|
||||
Description: "The scope as the cluster records it, team:ID. Import by it.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
|
||||
"scope": schema.StringAttribute{Required: true,
|
||||
Description: "org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT, or the ID after the colon.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.RequiresReplace()}},
|
||||
"name": schema.StringAttribute{Computed: true, Description: "The scope's path, Acme / ml.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
|
||||
"guarantee": amount("What is held for the scope: while it holds less, what others borrowed beyond their own guarantees is taken back for it."),
|
||||
"limit": amount("The most the scope may hold."),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (r *quotaResource) Configure(_ context.Context, req resource.ConfigureRequest, _ *resource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
r.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func amountOf(m *resourcesModel) client.Resources {
|
||||
if m == nil {
|
||||
return client.Resources{}
|
||||
}
|
||||
return client.Resources{GPUs: ptrOf(m.GPUs), CPUCores: ptrOf(m.CPUCores), MemoryGB: ptrOf(m.MemoryGB)}
|
||||
}
|
||||
|
||||
// amountModel is the cluster's amount for the state: a block left out stays
|
||||
// out while the cluster governs nothing in it.
|
||||
func amountModel(had *resourcesModel, r client.Resources) *resourcesModel {
|
||||
if had == nil && r.GPUs == nil && r.CPUCores == nil && r.MemoryGB == nil {
|
||||
return nil
|
||||
}
|
||||
return &resourcesModel{GPUs: ptrOrNull(r.GPUs), CPUCores: ptrOrNull(r.CPUCores), MemoryGB: ptrOrNull(r.MemoryGB)}
|
||||
}
|
||||
|
||||
func (r *quotaResource) put(ctx context.Context, plan *quotaModel) error {
|
||||
id, err := r.c.SetQuota(ctx, client.QuotaRequest{Scope: plan.Scope.ValueString(),
|
||||
Guarantee: amountOf(plan.Guarantee), Limit: amountOf(plan.Limit)})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
plan.ID = types.StringValue(id)
|
||||
plan.Name = types.StringNull()
|
||||
qs, err := r.c.Quotas(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, q := range qs {
|
||||
if q.Scope == id {
|
||||
plan.Name = strOrNull(q.Name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *quotaResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
|
||||
var plan quotaModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if err := r.put(ctx, &plan); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the quota on "+plan.Scope.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *quotaResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
|
||||
var st quotaModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
qs, err := r.c.Quotas(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the quotas", err.Error())
|
||||
return
|
||||
}
|
||||
for _, q := range qs {
|
||||
if q.Scope != st.ID.ValueString() {
|
||||
continue
|
||||
}
|
||||
if st.Scope.IsNull() {
|
||||
// Imported: the ID is the only name there is.
|
||||
st.Scope = types.StringValue(q.Scope)
|
||||
}
|
||||
st.Name = strOrNull(q.Name)
|
||||
st.Guarantee = amountModel(st.Guarantee, q.Guarantee)
|
||||
st.Limit = amountModel(st.Limit, q.Limit)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
return
|
||||
}
|
||||
resp.State.RemoveResource(ctx)
|
||||
}
|
||||
|
||||
func (r *quotaResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
|
||||
var plan quotaModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if err := r.put(ctx, &plan); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the quota on "+plan.Scope.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *quotaResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
|
||||
var st quotaModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if err := r.c.ClearQuota(ctx, st.ID.ValueString()); err != nil {
|
||||
resp.Diagnostics.AddError("Removing the quota on "+st.Scope.ValueString(), err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func (r *quotaResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) {
|
||||
resource.ImportStatePassthroughID(ctx, path.Root("id"), req, resp)
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/diag"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
)
|
||||
|
||||
// Conversions between Terraform's values and the client's.
|
||||
|
||||
// strOrNull is null for empty: the API leaves an unset field out, and a
|
||||
// configuration that does not set it holds null, not "".
|
||||
func strOrNull(s string) types.String {
|
||||
if s == "" {
|
||||
return types.StringNull()
|
||||
}
|
||||
return types.StringValue(s)
|
||||
}
|
||||
|
||||
func intOrNull(n int64) types.Int64 {
|
||||
if n == 0 {
|
||||
return types.Int64Null()
|
||||
}
|
||||
return types.Int64Value(n)
|
||||
}
|
||||
|
||||
func floatOrNull(f float64) types.Float64 {
|
||||
if f == 0 {
|
||||
return types.Float64Null()
|
||||
}
|
||||
return types.Float64Value(f)
|
||||
}
|
||||
|
||||
func ptrOrNull(p *int64) types.Int64 {
|
||||
if p == nil {
|
||||
return types.Int64Null()
|
||||
}
|
||||
return types.Int64Value(*p)
|
||||
}
|
||||
|
||||
func ptrOf(v types.Int64) *int64 {
|
||||
if v.IsNull() || v.IsUnknown() {
|
||||
return nil
|
||||
}
|
||||
n := v.ValueInt64()
|
||||
return &n
|
||||
}
|
||||
|
||||
func listOf(ctx context.Context, l types.List, diags *diag.Diagnostics) []string {
|
||||
if l.IsNull() || l.IsUnknown() {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
diags.Append(l.ElementsAs(ctx, &out, false)...)
|
||||
return out
|
||||
}
|
||||
|
||||
func mapOf(ctx context.Context, m types.Map, diags *diag.Diagnostics) map[string]string {
|
||||
if m.IsNull() || m.IsUnknown() {
|
||||
return nil
|
||||
}
|
||||
out := map[string]string{}
|
||||
diags.Append(m.ElementsAs(ctx, &out, false)...)
|
||||
return out
|
||||
}
|
||||
|
||||
// duration reads a duration written as Go writes them, "30m" or "1h30m".
|
||||
func duration(v types.String, what string, diags *diag.Diagnostics) time.Duration {
|
||||
if v.IsNull() || v.IsUnknown() || v.ValueString() == "" {
|
||||
return 0
|
||||
}
|
||||
d, err := time.ParseDuration(v.ValueString())
|
||||
if err != nil || d < 0 {
|
||||
diags.AddError("Not a duration", fmt.Sprintf("%s is %q: write a duration like 30m or 1h30m", what, v.ValueString()))
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
// keepDuration is the API's duration as the configuration wrote it when they
|
||||
// mean the same: "30m" and "30m0s" are one value, and showing a difference
|
||||
// between them on every plan would be noise.
|
||||
func keepDuration(had types.String, d time.Duration) types.String {
|
||||
if !had.IsNull() && !had.IsUnknown() {
|
||||
if was, err := time.ParseDuration(had.ValueString()); err == nil && was == d {
|
||||
return had
|
||||
}
|
||||
}
|
||||
return types.StringValue(d.String())
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
// Command terraform-provider-llmgw is the llmgw Terraform provider: models,
|
||||
// capacity quotas and guardrail attachments managed through a control
|
||||
// plane's admin API.
|
||||
//
|
||||
// A module of its own, so the gateway's binary does not carry Terraform's
|
||||
// dependencies.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"log"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/providerserver"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/provider"
|
||||
)
|
||||
|
||||
// version is set when a release is built: -ldflags "-X main.version=v0.1.0".
|
||||
var version = "dev"
|
||||
|
||||
// Address is the provider's source address, as a configuration's
|
||||
// required_providers names it.
|
||||
const Address = "gitea.mynodes.xyz/ilker/llmgw"
|
||||
|
||||
func main() {
|
||||
var debug bool
|
||||
flag.BoolVar(&debug, "debug", false, "run under a debugger, printing what TF_REATTACH_PROVIDERS needs")
|
||||
flag.Parse()
|
||||
|
||||
err := providerserver.Serve(context.Background(), provider.New(version), providerserver.ServeOpts{
|
||||
Address: Address,
|
||||
Debug: debug,
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user