Merge C19 OpenAPI for the admin API and a Terraform provider
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
commit
9dc82f73d4
37 files changed
+11095
-62
No files matched your search
@@ -39,6 +39,16 @@ jobs:
|
||||
- name: Build
|
||||
run: go build ./...
|
||||
|
||||
# The Terraform provider is a module of its own, which ./... above does
|
||||
# not reach. Its tests check its client against docs/admin-api.openapi.json,
|
||||
# so a change to the admin API that breaks the provider fails here. The
|
||||
# acceptance tests (TF_ACC=1) are left out: they need Terraform itself.
|
||||
- name: Test the Terraform provider
|
||||
working-directory: terraform-provider-llmgw
|
||||
run: |
|
||||
go vet ./...
|
||||
go test ./...
|
||||
|
||||
# The NVML backend needs cgo and cannot run without a driver, but it must
|
||||
# at least keep compiling: it is the only source of NVLink topology and a
|
||||
# break would otherwise surface on a GPU node, far from here.
|
||||
|
||||
@@ -5,3 +5,7 @@
|
||||
*.db-shm
|
||||
.DS_Store
|
||||
coverage.out
|
||||
# The provider built in place, and what Terraform leaves beside an example.
|
||||
/terraform-provider-llmgw/terraform-provider-llmgw
|
||||
/terraform-provider-llmgw/examples/.terraform*
|
||||
/terraform-provider-llmgw/examples/terraform.tfstate*
|
||||
@@ -1,4 +1,5 @@
|
||||
// Package llmgw holds what the binary carries besides code: its manual.
|
||||
// Package llmgw holds what the binary carries besides code: its manual, and
|
||||
// the description of its admin API.
|
||||
package llmgw
|
||||
|
||||
import (
|
||||
@@ -87,3 +88,10 @@ func titleOf(src, path string) string {
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
// AdminOpenAPI is the admin API described in OpenAPI 3.1, generated from its
|
||||
// route table by `go generate` in internal/docgen and served at
|
||||
// /api/v1/openapi.json, so a client can be pointed at the server it talks to.
|
||||
//
|
||||
//go:embed docs/admin-api.openapi.json
|
||||
var AdminOpenAPI []byte
|
||||
@@ -0,0 +1,160 @@
|
||||
# The admin API
|
||||
|
||||
Written for whoever automates a cluster rather than typing at it: how to reach
|
||||
the control plane's management API, what it asks of a request, and where its
|
||||
description is.
|
||||
|
||||
`llmgw ctl` and the console are clients of this API and nothing more, so
|
||||
anything they do can be done with `curl`, a generated client or the
|
||||
[Terraform provider](../terraform-provider-llmgw/README.md). It adds and
|
||||
removes models, scales them, cordons and upgrades nodes, sets capacity
|
||||
quotas, attaches guardrail policies, and reads the fleet, usage and the audit
|
||||
log.
|
||||
|
||||
## Reaching it
|
||||
|
||||
The API is under `/api/v1` on the admin listener, `admin_listen` in
|
||||
`/etc/llmgw/server.yaml`: `127.0.0.1:9090` unless changed. It is on loopback
|
||||
on purpose. The same listener serves pprof when it is turned on, and the
|
||||
server refuses to start with it on any other address unless
|
||||
`admin_remote: true` — the token would cross the network in clear.
|
||||
|
||||
From another machine, tunnel to it rather than moving it:
|
||||
|
||||
```sh
|
||||
ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
|
||||
curl -s -H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" http://127.0.0.1:9090/api/v1/summary
|
||||
```
|
||||
|
||||
With `admin_remote: true` and `admin_listen: 10.0.0.5:9090`, call that address
|
||||
directly — on a network you trust, or behind a TLS proxy that keeps the
|
||||
`Host` header an address.
|
||||
|
||||
## The token
|
||||
|
||||
Every path under `/api` needs the admin token as a bearer token:
|
||||
|
||||
```
|
||||
Authorization: Bearer 6f1c…
|
||||
```
|
||||
|
||||
The server writes it the first time it starts, 0600, to
|
||||
`/var/lib/llmgw/admin-token` under its systemd unit (`admin_token_file`
|
||||
moves it; a server run by hand keeps it in `secrets/admin-token` beside its
|
||||
configuration). `llmgw ctl` reads it from there, which is why it runs with
|
||||
sudo, or from `LLMGW_ADMIN_TOKEN`. To give it to automation:
|
||||
|
||||
```sh
|
||||
sudo cat /var/lib/llmgw/admin-token
|
||||
```
|
||||
|
||||
The token is the whole of the API's authorisation: whoever holds it can do
|
||||
everything the API does. Keep it where you keep other root credentials.
|
||||
|
||||
## What a request must look like
|
||||
|
||||
Besides the token, the listener refuses three things before a handler sees
|
||||
them, as defences against a browser being turned against it:
|
||||
|
||||
- **A name in `Host`.** The listener answers to an address or `localhost`,
|
||||
not a DNS name, so a page on some other site cannot reach it by rebinding
|
||||
a name to 127.0.0.1. Call it as `http://127.0.0.1:9090`, not
|
||||
`http://cp.example.com:9090`; behind a proxy, have the proxy send an
|
||||
address.
|
||||
- **A cross-site request.** One whose `Origin` is another host, or whose
|
||||
`Sec-Fetch-Site` says another site sent it.
|
||||
- **A change that is not JSON.** Every method but `GET` and `HEAD` must carry
|
||||
`Content-Type: application/json`, even a `DELETE` or a `POST` with no body.
|
||||
A form post from a page cannot set that header.
|
||||
|
||||
A change is recorded in the audit log with who made it, as the request says
|
||||
in `X-Llmgw-Actor` (letters, digits and `._-@`, at most 64; `unknown` when
|
||||
absent). `llmgw ctl` sends the name of whoever ran it; automation should send
|
||||
its own, `terraform` or `ci@deploy`, so `llmgw ctl audit` can tell them apart.
|
||||
|
||||
A refusal is a 4xx or 5xx with `{"error": "…"}` saying why. 501 means this
|
||||
control plane lacks what the endpoint needs — a store, a mirror, guard
|
||||
policies — and says which.
|
||||
|
||||
## Some calls
|
||||
|
||||
```sh
|
||||
H=(-H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" -H "Content-Type: application/json" -H "X-Llmgw-Actor: $USER")
|
||||
API=http://127.0.0.1:9090/api/v1
|
||||
|
||||
# The fleet in one answer, and every model.
|
||||
curl -s "${H[@]}" $API/summary
|
||||
curl -s "${H[@]}" $API/models
|
||||
|
||||
# Add a model, read back its definition with the defaults filled in, scale it.
|
||||
curl -s "${H[@]}" -X POST $API/models -d '{
|
||||
"name": "qwen-small",
|
||||
"source": "/srv/models/qwen2.5-1.5b-instruct-q4_k_m.gguf",
|
||||
"profiles": [{"name": "cpu", "engine": "llamacpp", "min_memory_gb": 4, "cpu_cores": 4}],
|
||||
"scaling": {"min": 1, "max": 2}
|
||||
}'
|
||||
curl -s "${H[@]}" $API/models/qwen-small
|
||||
curl -s "${H[@]}" -X POST $API/models/qwen-small/scale -d '{"min": 0, "max": 2, "idle_timeout": 1800000000000}'
|
||||
|
||||
# A team's capacity quota.
|
||||
curl -s "${H[@]}" -X PUT $API/quotas -d '{"scope": "team:Acme/ml", "limit": {"gpus": 8}}'
|
||||
|
||||
# Remove the model; a DELETE carries the content type too.
|
||||
curl -s "${H[@]}" -X DELETE $API/models/qwen-small
|
||||
```
|
||||
|
||||
Durations in bodies are integer nanoseconds, as Go encodes them: half an hour
|
||||
is `1800000000000`.
|
||||
|
||||
## The description
|
||||
|
||||
The API is described in OpenAPI 3.1:
|
||||
|
||||
- served by every control plane at `GET /api/v1/openapi.json` (with the
|
||||
token, like the rest), as built into that binary, so it describes the
|
||||
server you are talking to;
|
||||
- committed as [admin-api.openapi.json](admin-api.openapi.json) for reading
|
||||
without a cluster.
|
||||
|
||||
```sh
|
||||
curl -s "${H[@]}" $API/openapi.json > llmgw-admin.json
|
||||
```
|
||||
|
||||
It lists every endpoint, its path and query parameters, and the schema of
|
||||
each request and answer, with the meaning of each field. Load it into any
|
||||
OpenAPI tool — Swagger UI, Redoc, an editor — or generate a client from it:
|
||||
|
||||
```sh
|
||||
docker run --rm -v "$PWD:/w" openapitools/openapi-generator-cli generate \
|
||||
-i /w/llmgw-admin.json -g python -o /w/llmgw-admin-client
|
||||
```
|
||||
|
||||
### How it is kept true
|
||||
|
||||
The document is generated, not written. `internal/control/adminapi/routes.go`
|
||||
holds a table of every endpoint: its method and path, a summary, its query
|
||||
parameters, and Go values of the types it reads and answers with. The
|
||||
generator in `internal/docgen/openapi` reads those types by reflection, the
|
||||
way `encoding/json` encodes them, and takes each field's description from its
|
||||
comment in the source.
|
||||
|
||||
Two tests keep it from drifting. One registers the API on a recording mux and
|
||||
fails when an endpoint is served and not in the table, or in the table and not
|
||||
served. The other regenerates the document and fails when it differs from the
|
||||
committed one. After changing an endpoint or a type it carries:
|
||||
|
||||
```sh
|
||||
go generate ./internal/docgen
|
||||
```
|
||||
|
||||
## What it does not cover
|
||||
|
||||
- **Organizations, teams, projects, people and API keys.** These live in the
|
||||
store and are managed with `llmgw org`, `llmgw users` and `llmgw keys` on the
|
||||
control plane, or in the web console's `/admin` pages on the client port
|
||||
behind a session. The admin API reads the tree only where a quota or a
|
||||
policy names a part of it.
|
||||
- **The client-facing API** (`/v1/chat/completions` and the rest), which is
|
||||
OpenAI's and Anthropic's, on `listen`, with API keys.
|
||||
- **The rest of the admin listener**: `/metrics`, `/health`, the dashboard
|
||||
pages and `/debug/pprof`, which are not under `/api`.
|
||||
File diff suppressed because it is too large.
Load diff
+2
-1
@@ -252,7 +252,8 @@ knowing before you edit it:
|
||||
`LLMGW_ADMIN_TOKEN` gives it from elsewhere. pprof, whose heap holds the API
|
||||
keys requests presented, is off unless `debug_pprof: true`, and then behind
|
||||
the token too. To see the dashboard from elsewhere use `/admin` on the
|
||||
client port — do not move this listener.
|
||||
client port — do not move this listener. The API itself is described in
|
||||
[admin-api.md](admin-api.md).
|
||||
- `default_rpm`, `default_tpm` and `default_price` are what make limits and
|
||||
budgets mean anything. Without a price every request costs zero and a budget
|
||||
never binds; without the defaults, a project created without limits is
|
||||
|
||||
@@ -75,13 +75,26 @@ func (a *API) SetRevoker(f func(ctx context.Context, nodeID string, at time.Time
|
||||
a.revoke = f
|
||||
}
|
||||
|
||||
// Mux is where the endpoints are mounted. An *http.ServeMux is one; a test
|
||||
// that records the patterns is another, and is how the route table is checked
|
||||
// against what is actually served.
|
||||
type Mux interface {
|
||||
HandleFunc(pattern string, handler func(http.ResponseWriter, *http.Request))
|
||||
}
|
||||
|
||||
// Register mounts the endpoints on a mux under /api/v1.
|
||||
func (a *API) Register(mux *http.ServeMux) {
|
||||
//
|
||||
// Every pattern registered here, or by the register helpers it calls, must
|
||||
// also be in Routes: that table is what the OpenAPI document is generated
|
||||
// from, and a test fails when the two disagree.
|
||||
func (a *API) Register(mux Mux) {
|
||||
mux.HandleFunc("GET /api/v1/summary", a.summary)
|
||||
// Adding a model is a write, and the only one this API has. It lives here
|
||||
// rather than on the client-facing listener for the same reason the rest of
|
||||
// it does: reaching this port already means holding the control plane.
|
||||
mux.HandleFunc("GET /api/v1/openapi.json", serveOpenAPI)
|
||||
// Changes to the catalog live here rather than on the client-facing
|
||||
// listener for the same reason the rest of this does: reaching this port
|
||||
// already means holding the control plane.
|
||||
mux.HandleFunc("POST /api/v1/models", a.addModel)
|
||||
mux.HandleFunc("GET /api/v1/models/{name}", a.getModel)
|
||||
mux.HandleFunc("DELETE /api/v1/models/{name}", a.removeModel)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/scale", a.scaleModel)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/resources", a.setResources)
|
||||
@@ -97,9 +110,7 @@ func (a *API) Register(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane has no store")
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Class string `json:"class"`
|
||||
}
|
||||
var req ClassRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
@@ -109,7 +120,7 @@ func (a *API) Register(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "class": m.EffectiveClass()})
|
||||
writeJSON(w, http.StatusOK, ClassView{Model: m.Name, Class: m.EffectiveClass()})
|
||||
})
|
||||
mux.HandleFunc("PUT /api/v1/models/{name}/capacity", a.putCapacity)
|
||||
mux.HandleFunc("GET /api/v1/capacity", a.listCapacity)
|
||||
@@ -997,7 +1008,25 @@ func (a *API) addModel(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"added": m.Name})
|
||||
writeJSON(w, http.StatusOK, ModelAdded{Added: m.Name})
|
||||
}
|
||||
|
||||
// getModel answers with one model's whole definition as it is in force: what
|
||||
// was added, with the defaults the catalog filled in. The listing shows a
|
||||
// model's state and only the names of its profiles; this is what a tool
|
||||
// managing the definition — Terraform, say — compares against what it wants.
|
||||
//
|
||||
// By its name only, not an alias: an alias answering would let a tool believe
|
||||
// a model it manages exists when it is another one.
|
||||
func (a *API) getModel(w http.ResponseWriter, r *http.Request) {
|
||||
name := r.PathValue("name")
|
||||
for _, m := range a.ctl.Catalog().Models() {
|
||||
if m.Name == name {
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
return
|
||||
}
|
||||
}
|
||||
writeError(w, http.StatusNotFound, "no model named "+name+" is in the catalog")
|
||||
}
|
||||
|
||||
// ScaleRequest is how many replicas a model should run.
|
||||
@@ -1083,7 +1112,7 @@ func (a *API) removeModel(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": name})
|
||||
writeJSON(w, http.StatusOK, Removed{Removed: name})
|
||||
}
|
||||
|
||||
func (a *API) summary(w http.ResponseWriter, _ *http.Request) {
|
||||
@@ -1181,15 +1210,13 @@ func (a *API) removeNode(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
inv.Remove(id)
|
||||
a.ctl.Reconciler().Notify()
|
||||
writeJSON(w, http.StatusOK, map[string]string{"id": id, "removed": "true"})
|
||||
writeJSON(w, http.StatusOK, NodeRemoved{ID: id, Removed: "true"})
|
||||
}
|
||||
|
||||
func (a *API) setNodeState(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
|
||||
var body struct {
|
||||
State string `json:"state"`
|
||||
}
|
||||
var body NodeStateRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
@@ -1217,7 +1244,7 @@ func (a *API) setNodeState(w http.ResponseWriter, r *http.Request) {
|
||||
// is usually about to do something to it.
|
||||
a.ctl.Reconciler().Notify()
|
||||
|
||||
writeJSON(w, http.StatusOK, map[string]string{"id": id, "state": st.String()})
|
||||
writeJSON(w, http.StatusOK, NodeStateView{ID: id, State: st.String()})
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
@@ -1227,7 +1254,7 @@ func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
}
|
||||
|
||||
func writeError(w http.ResponseWriter, code int, msg string) {
|
||||
writeJSON(w, code, map[string]string{"error": msg})
|
||||
writeJSON(w, code, ErrorView{Error: msg})
|
||||
}
|
||||
|
||||
// profileChanged reports whether a replica is running a profile the catalog no
|
||||
@@ -1428,7 +1455,7 @@ func (a *API) retryModel(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
was := a.ctl.Reconciler().Retry(name)
|
||||
writeJSON(w, http.StatusOK, map[string]bool{"was_parked": was})
|
||||
writeJSON(w, http.StatusOK, RetryView{WasParked: was})
|
||||
}
|
||||
|
||||
// requestFinder looks a request up by the id it was given. Asked of the usage
|
||||
@@ -1523,7 +1550,7 @@ func (a *API) upgradeNode(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadGateway, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"node": r.PathValue("id"), "version": req.Version})
|
||||
writeJSON(w, http.StatusOK, UpgradeView{Node: r.PathValue("id"), Version: req.Version})
|
||||
}
|
||||
|
||||
// SetServe supplies what serves a model from the registry.
|
||||
@@ -1623,7 +1650,7 @@ func (a *API) rollbackModel(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"result": msg})
|
||||
writeJSON(w, http.StatusOK, Result{Result: msg})
|
||||
}
|
||||
|
||||
// OriginRequest names the repository a model's weights came from.
|
||||
|
||||
@@ -8,7 +8,13 @@ import (
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||
)
|
||||
|
||||
func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
// CanaryPercentRequest changes the share of a model's requests its canary
|
||||
// takes.
|
||||
type CanaryPercentRequest struct {
|
||||
Percent int `json:"percent"`
|
||||
}
|
||||
|
||||
func (a *API) registerCanary(mux Mux) {
|
||||
need := func(w http.ResponseWriter) bool {
|
||||
if a.serve == nil {
|
||||
writeError(w, http.StatusNotImplemented, "a canary needs a mirror and a store, and this control plane lacks one")
|
||||
@@ -45,9 +51,7 @@ func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
if !need(w) {
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Percent int `json:"percent"`
|
||||
}
|
||||
var req CanaryPercentRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
@@ -57,7 +61,7 @@ func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"result": msg})
|
||||
writeJSON(w, http.StatusOK, Result{Result: msg})
|
||||
})
|
||||
mux.HandleFunc("DELETE /api/v1/models/{name}/canary", func(w http.ResponseWriter, r *http.Request) {
|
||||
if !need(w) {
|
||||
@@ -68,7 +72,7 @@ func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"result": msg})
|
||||
writeJSON(w, http.StatusOK, Result{Result: msg})
|
||||
})
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/promote", func(w http.ResponseWriter, r *http.Request) {
|
||||
if !need(w) {
|
||||
|
||||
@@ -34,10 +34,7 @@ type RailView struct {
|
||||
// sends traffic over the fabric for seconds a rail, so it is a POST and only
|
||||
// ever run when asked.
|
||||
func (a *API) fabricTest(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
A, B, Fabric string
|
||||
Seconds int
|
||||
}
|
||||
var body FabricTestRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
|
||||
@@ -66,7 +66,17 @@ func (a *API) guardTarget(ctx context.Context, target string) (guard.Attachment,
|
||||
return guard.Attachment{}, fmt.Errorf("%q: attach to org, team, project, key or user", kind)
|
||||
}
|
||||
|
||||
func (a *API) registerGuard(mux *http.ServeMux) {
|
||||
// GuardAttachmentRequest attaches a policy to a part of the tenancy tree, a
|
||||
// key or a person, or with DELETE takes it off again.
|
||||
type GuardAttachmentRequest struct {
|
||||
// Target is org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT (or the ID
|
||||
// after the colon), key:KEY-ID or user:USER-ID.
|
||||
Target string `json:"target"`
|
||||
// Policy is a policy's name in server.yaml's guard.policies.
|
||||
Policy string `json:"policy"`
|
||||
}
|
||||
|
||||
func (a *API) registerGuard(mux Mux) {
|
||||
mux.HandleFunc("GET /api/v1/guard", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.guard == nil {
|
||||
writeError(w, http.StatusNotImplemented, "no guard policies: define them under guard: in server.yaml")
|
||||
@@ -97,12 +107,7 @@ func (a *API) registerGuard(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusNotImplemented, "no guard policies: define them under guard: in server.yaml")
|
||||
return
|
||||
}
|
||||
var at struct {
|
||||
// Target is org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT
|
||||
// (or the ID after the colon), key:KEY-ID or user:USER-ID.
|
||||
Target string `json:"target"`
|
||||
Policy string `json:"policy"`
|
||||
}
|
||||
var at GuardAttachmentRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&at); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
|
||||
@@ -50,7 +50,26 @@ type QuotaRequest struct {
|
||||
Limit store.Resources `json:"limit"`
|
||||
}
|
||||
|
||||
func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
// QuotaSet is the scope a quota was set on, as org:ID, team:ID or
|
||||
// project:ID whatever name it was asked by.
|
||||
type QuotaSet struct {
|
||||
Scope string `json:"scope"`
|
||||
}
|
||||
|
||||
// OwnerRequest sets the part of the tenancy tree a model's capacity counts
|
||||
// against: org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT, or their IDs.
|
||||
// Empty makes the model shared.
|
||||
type OwnerRequest struct {
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
// OwnerView is a model's owner now, as scope:ID.
|
||||
type OwnerView struct {
|
||||
Model string `json:"model"`
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
func (a *API) registerQuotas(mux Mux) {
|
||||
mux.HandleFunc("GET /api/v1/quotas", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.quotas == nil {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane has no store")
|
||||
@@ -78,7 +97,7 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"scope": key})
|
||||
writeJSON(w, http.StatusOK, QuotaSet{Scope: key})
|
||||
})
|
||||
mux.HandleFunc("DELETE /api/v1/quotas/{scope}", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.quotas == nil {
|
||||
@@ -90,16 +109,14 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": key})
|
||||
writeJSON(w, http.StatusOK, Removed{Removed: key})
|
||||
})
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/owner", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.models == nil || a.quotas == nil {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane has no store")
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
var req OwnerRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
@@ -118,7 +135,7 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "owner": m.Owner})
|
||||
writeJSON(w, http.StatusOK, OwnerView{Model: m.Name, Owner: m.Owner})
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
package adminapi
|
||||
|
||||
// The small bodies the endpoints read and answer with. Named types rather
|
||||
// than maps and anonymous structs, so the OpenAPI document generated from
|
||||
// Routes describes exactly what a handler decodes and writes: a map literal
|
||||
// in a handler is a shape no generator can see.
|
||||
|
||||
// ErrorView is every refusal: a 4xx or 5xx status with what went wrong.
|
||||
type ErrorView struct {
|
||||
Error string `json:"error"`
|
||||
}
|
||||
|
||||
// Result is what a change that has no other answer says it did.
|
||||
type Result struct {
|
||||
Result string `json:"result"`
|
||||
}
|
||||
|
||||
// ModelAdded names the model added to the catalog.
|
||||
type ModelAdded struct {
|
||||
Added string `json:"added"`
|
||||
}
|
||||
|
||||
// Removed names what was taken away: a model, or a quota's scope.
|
||||
type Removed struct {
|
||||
Removed string `json:"removed"`
|
||||
}
|
||||
|
||||
// ClassRequest sets a model's capacity class: critical, standard or
|
||||
// opportunistic, or empty to go back to what its scaling implies.
|
||||
type ClassRequest struct {
|
||||
Class string `json:"class"`
|
||||
}
|
||||
|
||||
// ClassView is a model's capacity class now.
|
||||
type ClassView struct {
|
||||
Model string `json:"model"`
|
||||
Class string `json:"class"`
|
||||
}
|
||||
|
||||
// RetryView says whether the model had been parked, given up on after
|
||||
// failing to start, before it was retried.
|
||||
type RetryView struct {
|
||||
WasParked bool `json:"was_parked"`
|
||||
}
|
||||
|
||||
// NodeStateRequest cordons, drains or readies a node: ready (or uncordon),
|
||||
// cordoned (or cordon), draining (or drain).
|
||||
type NodeStateRequest struct {
|
||||
State string `json:"state"`
|
||||
}
|
||||
|
||||
// NodeStateView is a node's state after the change.
|
||||
type NodeStateView struct {
|
||||
ID string `json:"id"`
|
||||
State string `json:"state"`
|
||||
}
|
||||
|
||||
// NodeRemoved names the node forgotten. Removed is the string "true".
|
||||
type NodeRemoved struct {
|
||||
ID string `json:"id"`
|
||||
Removed string `json:"removed"`
|
||||
}
|
||||
|
||||
// UpgradeView is the release a node was told to run.
|
||||
type UpgradeView struct {
|
||||
Node string `json:"node"`
|
||||
Version string `json:"version"`
|
||||
}
|
||||
|
||||
// FabricTestRequest is a bandwidth test between two nodes.
|
||||
type FabricTestRequest struct {
|
||||
A string `json:"a"`
|
||||
B string `json:"b"`
|
||||
// Fabric is the declared fabric to test over; empty is the one the two
|
||||
// nodes share.
|
||||
Fabric string `json:"fabric,omitempty"`
|
||||
// Seconds each rail is measured for, from 1 to 30.
|
||||
Seconds int `json:"seconds,omitempty"`
|
||||
}
|
||||
|
||||
// ReplayRequest names the model a kept request is sent to again.
|
||||
type ReplayRequest struct {
|
||||
Model string `json:"model"`
|
||||
}
|
||||
@@ -60,9 +60,7 @@ func (a *API) replayRequest(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane does not route requests")
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Model string `json:"model"`
|
||||
}
|
||||
var req ReplayRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil || req.Model == "" {
|
||||
writeError(w, http.StatusBadRequest, "say which model: {\"model\": \"...\"}")
|
||||
return
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
package adminapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
llmgw "gitea.mynodes.xyz/ilker/llmgw"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/bench"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/fabrics"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/modelserve"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/nvlink"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/guard"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/notify"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
||||
)
|
||||
|
||||
// Route is one endpoint as the OpenAPI document describes it.
|
||||
//
|
||||
// A table written by hand beside Register rather than read out of the
|
||||
// handlers, because what a handler decodes and answers is not visible from
|
||||
// outside it. The table is checked against Register by a test, so an endpoint
|
||||
// cannot be added without being described; the bodies are Go values whose
|
||||
// types the generator reads, so a field added to one is in the document the
|
||||
// next time it is generated.
|
||||
type Route struct {
|
||||
Method string
|
||||
Path string
|
||||
// Summary is one line; Description, when there is more to say, the rest.
|
||||
Summary string
|
||||
Description string
|
||||
// Query is the query parameters the handler reads.
|
||||
Query []Param
|
||||
// Request is a value of the body's type; nil reads none.
|
||||
Request any
|
||||
// OptionalBody is set when the body may be left out.
|
||||
OptionalBody bool
|
||||
// Response is a value of the type a success answers with.
|
||||
Response any
|
||||
// Status is the success status; zero is 200.
|
||||
Status int
|
||||
// CSV is set when the answer is CSV unless JSON is asked for.
|
||||
CSV bool
|
||||
}
|
||||
|
||||
// Param is a query parameter.
|
||||
type Param struct {
|
||||
Name string
|
||||
Description string
|
||||
// Type is a JSON Schema type: string when empty, or integer, number,
|
||||
// boolean.
|
||||
Type string
|
||||
Required bool
|
||||
}
|
||||
|
||||
// PathParams describes the parameters in Routes' paths, by the segment
|
||||
// before them and their name.
|
||||
var PathParams = map[string]string{
|
||||
"models/{name}": "The model's name in the catalog.",
|
||||
"nodes/{id}": "The node's ID, as `llmgw ctl nodes` lists it.",
|
||||
"jobs/{id}": "The job's ID, as starting it answered.",
|
||||
"requests/{id}": "The request's ID: the x-request-id a client was answered with.",
|
||||
"fabrics/{id}": "The physical fabric's ID, as GET /api/v1/fabrics lists it.",
|
||||
"quotas/{scope}": "The scope: org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT, or their IDs after the colon.",
|
||||
}
|
||||
|
||||
// Routes is every endpoint Register mounts.
|
||||
var Routes = []Route{
|
||||
// The document itself.
|
||||
{Method: "GET", Path: "/api/v1/openapi.json", Summary: "This document",
|
||||
Description: "The admin API in OpenAPI 3.1, generated from the route table and built into the binary.",
|
||||
Response: map[string]any{}},
|
||||
|
||||
// The fleet.
|
||||
{Method: "GET", Path: "/api/v1/summary", Summary: "The whole fleet in one answer",
|
||||
Response: Summary{}},
|
||||
{Method: "GET", Path: "/api/v1/nodes", Summary: "Every node, with its cards, load and fabrics",
|
||||
Response: []NodeView{}},
|
||||
{Method: "POST", Path: "/api/v1/nodes/{id}/state", Summary: "Cordon, drain or ready a node",
|
||||
Request: NodeStateRequest{}, Response: NodeStateView{}},
|
||||
{Method: "POST", Path: "/api/v1/nodes/{id}/upgrade", Summary: "Have a node run a release",
|
||||
Request: UpgradeRequest{}, Response: UpgradeView{}},
|
||||
{Method: "POST", Path: "/api/v1/nodes/{id}/engines", Summary: "Choose an engine's build for a node",
|
||||
Description: "The node installs the build chosen. Build is auto, a build such as cu129, or none.",
|
||||
Request: NodeEngineRequest{}, Response: NodeEngineRequest{}},
|
||||
{Method: "DELETE", Path: "/api/v1/nodes/{id}", Summary: "Forget a node that has gone",
|
||||
Description: "Only a node that is lost, or unreachable with nothing on it; its certificate is revoked.",
|
||||
Response: NodeRemoved{}},
|
||||
{Method: "GET", Path: "/api/v1/nodes/{id}/logs", Summary: "The last lines of a node's logs",
|
||||
Query: []Param{
|
||||
{Name: "lines", Type: "integer", Description: "How many lines; 300 when absent, at most 2000."},
|
||||
{Name: "source", Description: "Which log: agent when absent, or an engine the node lists in sources."},
|
||||
},
|
||||
Response: LogsView{}},
|
||||
{Method: "GET", Path: "/api/v1/fabrics", Summary: "The physical fabrics and the names nodes give them",
|
||||
Response: fabrics.Report{}},
|
||||
{Method: "GET", Path: "/api/v1/fabrics/{id}/topology", Summary: "One fabric's map, with the cluster's adapters live on it",
|
||||
Response: TopologyView{}},
|
||||
{Method: "GET", Path: "/api/v1/nvlink", Summary: "The multi-node NVLink domains and what keeps nodes out of them",
|
||||
Response: nvlink.Report{}},
|
||||
{Method: "POST", Path: "/api/v1/fabric-test", Summary: "Measure bandwidth between two nodes, rail by rail",
|
||||
Request: FabricTestRequest{}, Response: FabricTestView{}},
|
||||
{Method: "GET", Path: "/api/v1/weights/orphans", Summary: "Weights on nodes that no model uses",
|
||||
Response: []OrphanView{}},
|
||||
{Method: "POST", Path: "/api/v1/weights/collect", Summary: "Delete weights that no model uses",
|
||||
Request: CollectRequest{}, OptionalBody: true, Response: []CollectResult{}},
|
||||
|
||||
// The catalog.
|
||||
{Method: "GET", Path: "/api/v1/models", Summary: "Every model, with its replicas and state",
|
||||
Response: []ModelView{}},
|
||||
{Method: "POST", Path: "/api/v1/models", Summary: "Add a model to the catalog",
|
||||
Description: "Kept in the store across restarts. Refused when the name is taken, a model in " +
|
||||
"configuration has it, or the definition would make the catalog invalid.",
|
||||
Request: catalog.Model{}, Response: ModelAdded{}},
|
||||
{Method: "GET", Path: "/api/v1/models/{name}", Summary: "One model's definition, as it is in force",
|
||||
Description: "With the defaults the catalog filled in. By its name only, not an alias.",
|
||||
Response: catalog.Model{}},
|
||||
{Method: "DELETE", Path: "/api/v1/models/{name}", Summary: "Remove a model added at runtime",
|
||||
Description: "A model declared in configuration is refused: it would come back on the next restart.",
|
||||
Response: Removed{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/scale", Summary: "Set how many replicas a model runs",
|
||||
Request: ScaleRequest{}, Response: ScaleResponse{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/resources", Summary: "Set what a model's profiles reserve of a node",
|
||||
Request: ResourcesRequest{}, Response: catalog.Model{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/class", Summary: "Set a model's capacity class",
|
||||
Request: ClassRequest{}, Response: ClassView{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/owner", Summary: "Set the part of the tenancy tree a model's capacity counts against",
|
||||
Request: OwnerRequest{}, Response: OwnerView{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/adapters", Summary: "Replace a model's LoRA adapters",
|
||||
Description: "Each new file is read first, to check it is an adapter for the model's architecture.",
|
||||
Request: AdaptersRequest{}, Response: AdaptersRequest{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/retry", Summary: "Try a parked model again",
|
||||
Response: RetryView{}},
|
||||
{Method: "GET", Path: "/api/v1/models/{name}/target", Summary: "How many replicas a throughput needs, and whether they fit",
|
||||
Query: []Param{{Name: "tokens_per_sec", Type: "number", Required: true,
|
||||
Description: "The output tokens a second wanted of the model."}},
|
||||
Response: TargetView{}},
|
||||
{Method: "GET", Path: "/api/v1/models/{name}/headroom", Summary: "How many more replicas of a model fit now, and where",
|
||||
Query: []Param{{Name: "limit", Type: "integer",
|
||||
Description: "The most to try placing; 16 when absent or past 64."}},
|
||||
Response: HeadroomView{}},
|
||||
{Method: "PUT", Path: "/api/v1/models/{name}/capacity", Summary: "Record what a replica was measured to serve",
|
||||
Description: "Replaces an earlier measurement of the same model on the same profile.",
|
||||
Request: bench.Capacity{}, Response: bench.Capacity{}},
|
||||
{Method: "GET", Path: "/api/v1/capacity", Summary: "Every recorded measurement",
|
||||
Response: []CapacityView{}},
|
||||
{Method: "GET", Path: "/api/v1/replicas", Summary: "Every replica, with its shards",
|
||||
Query: []Param{{Name: "model", Description: "Only this model's replicas."}},
|
||||
Response: []ReplicaView{}},
|
||||
{Method: "GET", Path: "/api/v1/explain-placement", Summary: "Why a model could not be placed",
|
||||
Query: []Param{{Name: "model", Required: true, Description: "The model's name or an alias."}},
|
||||
Response: ExplainView{}},
|
||||
{Method: "POST", Path: "/api/v1/restart", Summary: "Restart replicas, every replica of some models, or all",
|
||||
Request: RestartRequest{}, Response: RestartResult{}, Status: http.StatusAccepted},
|
||||
|
||||
// Serving from the registry, and newer weights.
|
||||
{Method: "GET", Path: "/api/v1/plan", Summary: "How a registry repository would be served here",
|
||||
Query: []Param{
|
||||
{Name: "repo", Required: true, Description: "The repository, owner/name."},
|
||||
{Name: "format", Description: "gguf or safetensors; gguf when absent."},
|
||||
{Name: "context", Type: "integer", Description: "The context length to plan for, in tokens."},
|
||||
},
|
||||
Response: modelserve.Plan{}},
|
||||
{Method: "POST", Path: "/api/v1/models/serve", Summary: "Fetch a registry repository and serve it",
|
||||
Request: modelserve.Request{}, Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
{Method: "GET", Path: "/api/v1/jobs", Summary: "Every job since the control plane started",
|
||||
Response: []modelserve.Job{}},
|
||||
{Method: "GET", Path: "/api/v1/jobs/{id}", Summary: "One job",
|
||||
Response: modelserve.Job{}},
|
||||
{Method: "GET", Path: "/api/v1/updates", Summary: "Newer weights the registry has for models",
|
||||
Response: []modelserve.UpdateInfo{}},
|
||||
{Method: "POST", Path: "/api/v1/updates/check", Summary: "Ask the registry now, about one model or every one",
|
||||
Request: CheckRequest{}, OptionalBody: true, Response: []modelserve.UpdateInfo{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/update", Summary: "Move a model to its repository's newer weights",
|
||||
Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/rollback", Summary: "Go back to the weights the last update replaced",
|
||||
Response: Result{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/origin", Summary: "Record the repository a model's weights came from",
|
||||
Request: OriginRequest{}, Response: OriginView{}},
|
||||
{Method: "GET", Path: "/api/v1/canaries", Summary: "Every canary, and how its two sides serve",
|
||||
Response: []modelserve.CanaryView{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/canary", Summary: "Start a canary of a model's newer weights",
|
||||
Request: catalog.Canary{}, Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
{Method: "PUT", Path: "/api/v1/models/{name}/canary", Summary: "Change the share of requests a canary takes",
|
||||
Request: CanaryPercentRequest{}, Response: Result{}},
|
||||
{Method: "DELETE", Path: "/api/v1/models/{name}/canary", Summary: "Take a canary out: its share first, then the candidate",
|
||||
Response: Result{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/promote", Summary: "Make a canary's weights the model's",
|
||||
Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
|
||||
// Tenancy: quotas and guardrails.
|
||||
{Method: "GET", Path: "/api/v1/quotas", Summary: "Every capacity quota, with what its scope holds now",
|
||||
Response: []QuotaView{}},
|
||||
{Method: "PUT", Path: "/api/v1/quotas", Summary: "Set a scope's capacity quota",
|
||||
Description: "Replaces the scope's quota whole. The scope may be named by path or by ID; the answer names it by ID.",
|
||||
Request: QuotaRequest{}, Response: QuotaSet{}},
|
||||
{Method: "DELETE", Path: "/api/v1/quotas/{scope}", Summary: "Remove a scope's capacity quota",
|
||||
Response: Removed{}},
|
||||
{Method: "GET", Path: "/api/v1/guard", Summary: "The guardrail policies and where they are attached",
|
||||
Response: GuardView{}},
|
||||
{Method: "POST", Path: "/api/v1/guard/attachments", Summary: "Attach a guardrail policy",
|
||||
Request: GuardAttachmentRequest{}, Response: guard.Attachment{}},
|
||||
{Method: "DELETE", Path: "/api/v1/guard/attachments", Summary: "Detach a guardrail policy",
|
||||
Request: GuardAttachmentRequest{}, Response: guard.Attachment{}},
|
||||
|
||||
// Usage, requests and the record of changes.
|
||||
{Method: "GET", Path: "/api/v1/usage", Summary: "What each group of the tenancy tree spent",
|
||||
Query: []Param{
|
||||
{Name: "by", Description: "org, team or project; team when absent."},
|
||||
{Name: "since", Description: "A duration back from now, like 24h; 24h when absent."},
|
||||
},
|
||||
Response: []ScopeUsageView{}},
|
||||
{Method: "GET", Path: "/api/v1/usage/export", Summary: "Usage totals by period and dimension, as CSV or JSON",
|
||||
Query: []Param{
|
||||
{Name: "from", Description: "A date (2026-09-01) or an RFC 3339 time; the first of this month when absent."},
|
||||
{Name: "to", Description: "A date or an RFC 3339 time; now when absent."},
|
||||
{Name: "period", Description: "hour, day or month; day when absent."},
|
||||
{Name: "by", Description: "Dimensions, comma-separated: org, team, project, user, key, model."},
|
||||
{Name: "format", Description: "json for JSON; CSV otherwise."},
|
||||
},
|
||||
Response: []store.ExportRow{}, CSV: true},
|
||||
{Method: "GET", Path: "/api/v1/requests/{id}", Summary: "What one request went through",
|
||||
Response: RequestView{}},
|
||||
{Method: "GET", Path: "/api/v1/requests/{id}/body", Summary: "A kept request and its answer",
|
||||
Response: store.RequestBody{}},
|
||||
{Method: "POST", Path: "/api/v1/requests/{id}/replay", Summary: "Send a kept request again, to any model",
|
||||
Request: ReplayRequest{}, Response: ReplayView{}},
|
||||
{Method: "GET", Path: "/api/v1/audit", Summary: "Changes made, newest first",
|
||||
Query: []Param{
|
||||
{Name: "since", Description: "How far back: 90m, 24h, 7d; 7d when absent."},
|
||||
{Name: "limit", Type: "integer", Description: "The most events to answer with."},
|
||||
{Name: "actor", Description: "Only changes by this actor."},
|
||||
{Name: "contains", Description: "Only events whose action or target contains this."},
|
||||
},
|
||||
Response: []store.AuditEvent{}},
|
||||
|
||||
// Operations.
|
||||
{Method: "POST", Path: "/api/v1/notify/test", Summary: "Send a test notification to every webhook",
|
||||
Response: []notify.Result{}},
|
||||
{Method: "POST", Path: "/api/v1/metrics/range", Summary: "Several Prometheus range queries at once",
|
||||
Description: "Read from the Prometheus monitoring.prometheus names; configured is false without one.",
|
||||
Request: MetricsRequest{}, Response: MetricsResult{}},
|
||||
}
|
||||
|
||||
// serveOpenAPI answers with the document generated from Routes, as it was
|
||||
// when this binary was built.
|
||||
func serveOpenAPI(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write(llmgw.AdminOpenAPI)
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
package adminapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||
)
|
||||
|
||||
// patterns records what Register mounts.
|
||||
type patterns []string
|
||||
|
||||
func (p *patterns) HandleFunc(pattern string, _ func(http.ResponseWriter, *http.Request)) {
|
||||
*p = append(*p, pattern)
|
||||
}
|
||||
|
||||
func TestEveryRegisteredEndpointIsInTheRouteTable(t *testing.T) {
|
||||
// The OpenAPI document is generated from Routes, so an endpoint missing
|
||||
// from it is one no client generated from the document can call, and one
|
||||
// in it that is not served is a promise the server breaks.
|
||||
a, _ := setup(t, testModels())
|
||||
var got patterns
|
||||
a.Register(&got)
|
||||
|
||||
table := map[string]bool{}
|
||||
for _, r := range Routes {
|
||||
key := r.Method + " " + r.Path
|
||||
if table[key] {
|
||||
t.Errorf("%s is in Routes twice", key)
|
||||
}
|
||||
table[key] = true
|
||||
}
|
||||
served := map[string]bool{}
|
||||
for _, p := range got {
|
||||
served[p] = true
|
||||
if !table[p] {
|
||||
t.Errorf("%s is registered and not in Routes: describe it there", p)
|
||||
}
|
||||
}
|
||||
for key := range table {
|
||||
if !served[key] {
|
||||
t.Errorf("%s is in Routes and not registered", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryRouteSaysWhatItAnswers(t *testing.T) {
|
||||
for _, r := range Routes {
|
||||
if r.Summary == "" || r.Response == nil {
|
||||
t.Errorf("%s %s needs a summary and a response type", r.Method, r.Path)
|
||||
}
|
||||
if r.Request != nil && (r.Method == "GET" || r.Method == "HEAD") {
|
||||
t.Errorf("%s %s reads a body on a GET", r.Method, r.Path)
|
||||
}
|
||||
for _, seg := range strings.Split(strings.Trim(r.Path, "/"), "/") {
|
||||
if !strings.HasPrefix(seg, "{") {
|
||||
continue
|
||||
}
|
||||
found := false
|
||||
for k := range PathParams {
|
||||
if strings.HasSuffix(k, "/"+seg) && strings.Contains(r.Path, k) {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("%s %s: %s is not described in PathParams", r.Method, r.Path, seg)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheAPIServesItsOwnDescription(t *testing.T) {
|
||||
a, _ := setup(t, testModels())
|
||||
var doc struct {
|
||||
OpenAPI string `json:"openapi"`
|
||||
Paths map[string]map[string]any `json:"paths"`
|
||||
}
|
||||
if code := call(t, a, "GET", "/api/v1/openapi.json", "", &doc); code != 200 {
|
||||
t.Fatalf("status %d", code)
|
||||
}
|
||||
if doc.OpenAPI != "3.1.0" {
|
||||
t.Errorf("openapi %q", doc.OpenAPI)
|
||||
}
|
||||
// What is built in is the document generated when it was last committed;
|
||||
// internal/docgen's test is what fails when that is stale. This checks the
|
||||
// two ends meet: every route the table has is in what is served.
|
||||
for _, r := range Routes {
|
||||
if _, ok := doc.Paths[r.Path][strings.ToLower(r.Method)]; !ok {
|
||||
t.Errorf("the served document has no %s %s", r.Method, r.Path)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOneModelsDefinitionIsReadByItsNameAlone(t *testing.T) {
|
||||
a, _ := setup(t, testModels())
|
||||
var m catalog.Model
|
||||
if code := call(t, a, "GET", "/api/v1/models/llama", "", &m); code != 200 {
|
||||
t.Fatalf("status %d", code)
|
||||
}
|
||||
if m.Name != "llama" || m.Source != "/m" || len(m.Profiles) != 1 || m.Profiles[0].TP != 8 {
|
||||
t.Errorf("got %+v", m)
|
||||
}
|
||||
if m.Scaling.Min != 2 || m.Scaling.Max != 4 {
|
||||
t.Errorf("scaling %+v", m.Scaling)
|
||||
}
|
||||
if !slices.Equal(m.Aliases, []string{"l"}) {
|
||||
t.Errorf("aliases %v", m.Aliases)
|
||||
}
|
||||
// An alias answering would let a tool that manages "l" believe it exists
|
||||
// when what it found is llama.
|
||||
if code := call(t, a, "GET", "/api/v1/models/l", "", nil); code != http.StatusNotFound {
|
||||
t.Errorf("an alias answered %d", code)
|
||||
}
|
||||
if code := call(t, a, "GET", "/api/v1/models/nothing", "", nil); code != http.StatusNotFound {
|
||||
t.Errorf("a missing model answered %d", code)
|
||||
}
|
||||
}
|
||||
@@ -65,6 +65,20 @@ func (a *API) listOrphans(w http.ResponseWriter, _ *http.Request) {
|
||||
writeJSON(w, http.StatusOK, a.Orphans())
|
||||
}
|
||||
|
||||
// CollectRequest limits a sweep to one node. Empty means every node, which
|
||||
// is the common case after removing a model from the catalog.
|
||||
type CollectRequest struct {
|
||||
Node string `json:"node,omitempty"`
|
||||
}
|
||||
|
||||
// CollectResult is what was deleted on one node, or why nothing was.
|
||||
type CollectResult struct {
|
||||
Node string `json:"node"`
|
||||
Files int `json:"files"`
|
||||
Bytes int64 `json:"bytes"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// collectWeights deletes them.
|
||||
//
|
||||
// A proposal, not an instruction: each node checks every path again before
|
||||
@@ -79,26 +93,16 @@ func (a *API) collectWeights(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
var body struct {
|
||||
// Node limits it to one node. Empty means every node, which is the
|
||||
// common case after removing a model from the catalog.
|
||||
Node string `json:"node,omitempty"`
|
||||
}
|
||||
var body CollectRequest
|
||||
// An empty body is the "all nodes" case and must not be an error.
|
||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||
|
||||
type result struct {
|
||||
Node string `json:"node"`
|
||||
Files int `json:"files"`
|
||||
Bytes int64 `json:"bytes"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
out := make([]result, 0)
|
||||
out := make([]CollectResult, 0)
|
||||
for _, o := range a.Orphans() {
|
||||
if body.Node != "" && o.Node != body.Node {
|
||||
continue
|
||||
}
|
||||
res := result{Node: o.Node, Files: len(o.Paths), Bytes: o.Bytes}
|
||||
res := CollectResult{Node: o.Node, Files: len(o.Paths), Bytes: o.Bytes}
|
||||
// A node that has gone since the list was built is reported rather than
|
||||
// failing the whole sweep: the other nodes' space is still worth
|
||||
// freeing, and this one will be offered again next time.
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
// Command gen writes docs/reference.md from the configuration structs.
|
||||
// Command gen writes docs/reference.md from the configuration structs, and
|
||||
// docs/admin-api.openapi.json from the admin API route table.
|
||||
//
|
||||
// Run by `go generate ./...`, and checked by a test, so the reference cannot
|
||||
// quietly stop describing the code.
|
||||
@@ -10,7 +11,9 @@ import (
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/docgen"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/docgen/openapi"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -43,4 +46,15 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("docs/reference.md and internal/docgen/reference_gen.go: %d blocks\n", len(blocks))
|
||||
|
||||
spec, err := openapi.Generate(root, adminapi.Routes, adminapi.PathParams)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "docs", "admin-api.openapi.json"), spec, 0o644); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("docs/admin-api.openapi.json: %d routes\n", len(adminapi.Routes))
|
||||
}
|
||||
@@ -0,0 +1,672 @@
|
||||
// Package openapi generates the admin API's OpenAPI 3.1 document from its
|
||||
// route table, adminapi.Routes.
|
||||
//
|
||||
// The schemas are read from the Go types the handlers decode and write, by
|
||||
// reflection, following encoding/json's rules — tag names, omitted fields,
|
||||
// embedded structs flattened — so the document says what goes over the wire
|
||||
// rather than what someone remembered to write down. Their descriptions are
|
||||
// the types' and fields' doc comments, read from the source, which is why
|
||||
// generating needs the repository and is done by `go generate` rather than
|
||||
// at run time: the binary serves the document built into it.
|
||||
package openapi
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
|
||||
)
|
||||
|
||||
// Module is the import path the repository's packages are under; a type
|
||||
// from any other is described without comments.
|
||||
const Module = "gitea.mynodes.xyz/ilker/llmgw"
|
||||
|
||||
// Generate renders the document for routes, reading doc comments from the
|
||||
// repository at root.
|
||||
func Generate(root string, routes []adminapi.Route, pathParams map[string]string) ([]byte, error) {
|
||||
g := &generator{root: root, docs: map[string]map[string]typeDoc{}, names: map[reflect.Type]string{}}
|
||||
|
||||
// Every named struct reachable from a body first, so a name two
|
||||
// packages share can be told apart before either is written: by the name
|
||||
// alone where it is unique, with its package where it is not.
|
||||
for _, r := range routes {
|
||||
for _, v := range []any{r.Request, r.Response} {
|
||||
if v != nil {
|
||||
g.collect(reflect.TypeOf(v))
|
||||
}
|
||||
}
|
||||
}
|
||||
g.collect(errorType)
|
||||
g.assignNames()
|
||||
errorSchema := g.schema(errorType)
|
||||
|
||||
paths := map[string]*object{}
|
||||
var order []string
|
||||
tags := map[string]bool{}
|
||||
for _, r := range routes {
|
||||
item, ok := paths[r.Path]
|
||||
if !ok {
|
||||
item = &object{}
|
||||
paths[r.Path] = item
|
||||
order = append(order, r.Path)
|
||||
}
|
||||
op, err := g.operation(r, pathParams)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tags[tagOf(r.Path)] = true
|
||||
item.set(strings.ToLower(r.Method), op)
|
||||
}
|
||||
slices.Sort(order)
|
||||
var pathsObj object
|
||||
for _, p := range order {
|
||||
pathsObj.set(p, *paths[p])
|
||||
}
|
||||
|
||||
var schemas object
|
||||
names := make([]string, 0, len(g.schemas))
|
||||
for n := range g.schemas {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, n := range names {
|
||||
schemas.set(n, g.schemas[n])
|
||||
}
|
||||
|
||||
var tagList []any
|
||||
for _, t := range sortedKeys(tags) {
|
||||
tagList = append(tagList, obj("name", t))
|
||||
}
|
||||
|
||||
doc := obj(
|
||||
"openapi", "3.1.0",
|
||||
"info", obj(
|
||||
"title", "llmgw admin API",
|
||||
"version", "v1",
|
||||
"description", infoDescription,
|
||||
),
|
||||
"servers", []any{obj("url", "http://127.0.0.1:9090",
|
||||
"description", "The admin listener at its default address, admin_listen.")},
|
||||
"security", []any{obj("adminToken", []any{})},
|
||||
"tags", tagList,
|
||||
"paths", pathsObj,
|
||||
"components", obj(
|
||||
"securitySchemes", obj("adminToken", obj(
|
||||
"type", "http",
|
||||
"scheme", "bearer",
|
||||
"description", "The admin token: the contents of /var/lib/llmgw/admin-token on the "+
|
||||
"control plane (admin_token_file), readable by root and the server. "+
|
||||
"Sent as Authorization: Bearer TOKEN.",
|
||||
)),
|
||||
"parameters", obj("Actor", obj(
|
||||
"name", "X-Llmgw-Actor",
|
||||
"in", "header",
|
||||
"required", false,
|
||||
"description", "Who the change is recorded as made by in the audit log: letters, digits "+
|
||||
"and ._-@, at most 64. unknown when absent.",
|
||||
"schema", obj("type", "string"),
|
||||
)),
|
||||
"responses", obj(
|
||||
"Error", obj("description", "Refused: the error says why.",
|
||||
"content", obj("application/json", obj("schema", errorSchema))),
|
||||
"Unauthorized", obj("description", "The admin token is missing or wrong.",
|
||||
"content", obj("application/json", obj("schema", errorSchema))),
|
||||
),
|
||||
"schemas", schemas,
|
||||
),
|
||||
)
|
||||
b, err := marshal(doc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out bytes.Buffer
|
||||
if err := json.Indent(&out, b, "", " "); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out.WriteByte('\n')
|
||||
return out.Bytes(), nil
|
||||
}
|
||||
|
||||
const infoDescription = "The control plane's management API: what `llmgw ctl` and the console use.\n\n" +
|
||||
"It is served on the admin listener (admin_listen, 127.0.0.1:9090 by default), which stays on " +
|
||||
"loopback unless admin_remote is set: reach it from elsewhere through an SSH tunnel. Every path " +
|
||||
"under /api needs the admin token as a bearer token.\n\n" +
|
||||
"The listener also refuses, before the token is looked at: a Host header that is a name rather " +
|
||||
"than an address or localhost; a request another site's page sent; and any method but GET and " +
|
||||
"HEAD whose Content-Type is not application/json, even one with no body.\n\n" +
|
||||
"Durations in bodies are integer nanoseconds, as Go encodes them. Generated from the route " +
|
||||
"table in internal/control/adminapi; see docs/admin-api.md."
|
||||
|
||||
// operation builds one route's operation object.
|
||||
func (g *generator) operation(r adminapi.Route, pathParams map[string]string) (object, error) {
|
||||
var op object
|
||||
op.set("operationId", operationID(r.Method, r.Path))
|
||||
op.set("summary", r.Summary)
|
||||
if r.Description != "" {
|
||||
op.set("description", r.Description)
|
||||
}
|
||||
op.set("tags", []any{tagOf(r.Path)})
|
||||
|
||||
var params []any
|
||||
segs := strings.Split(strings.Trim(r.Path, "/"), "/")
|
||||
for i, s := range segs {
|
||||
if !strings.HasPrefix(s, "{") {
|
||||
continue
|
||||
}
|
||||
name := strings.Trim(s, "{}")
|
||||
key := s
|
||||
if i > 0 {
|
||||
key = segs[i-1] + "/" + s
|
||||
}
|
||||
desc, ok := pathParams[key]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s %s: no description of %s in adminapi.PathParams", r.Method, r.Path, key)
|
||||
}
|
||||
params = append(params, obj("name", name, "in", "path", "required", true,
|
||||
"description", desc, "schema", obj("type", "string")))
|
||||
}
|
||||
for _, q := range r.Query {
|
||||
t := q.Type
|
||||
if t == "" {
|
||||
t = "string"
|
||||
}
|
||||
p := obj("name", q.Name, "in", "query")
|
||||
if q.Required {
|
||||
p.set("required", true)
|
||||
}
|
||||
p.set("description", q.Description)
|
||||
p.set("schema", obj("type", t))
|
||||
params = append(params, p)
|
||||
}
|
||||
if r.Method != "GET" && r.Method != "HEAD" {
|
||||
params = append(params, obj("$ref", "#/components/parameters/Actor"))
|
||||
}
|
||||
if len(params) > 0 {
|
||||
op.set("parameters", params)
|
||||
}
|
||||
|
||||
if r.Request != nil {
|
||||
body := obj("required", !r.OptionalBody,
|
||||
"content", obj("application/json", obj("schema", g.schema(reflect.TypeOf(r.Request)))))
|
||||
op.set("requestBody", body)
|
||||
}
|
||||
|
||||
status := r.Status
|
||||
if status == 0 {
|
||||
status = 200
|
||||
}
|
||||
content := obj("application/json", obj("schema", g.schema(reflect.TypeOf(r.Response))))
|
||||
if r.CSV {
|
||||
content.set("text/csv", obj("schema", obj("type", "string")))
|
||||
}
|
||||
desc := "OK"
|
||||
if status == 202 {
|
||||
desc = "Accepted: started, and running on."
|
||||
}
|
||||
op.set("responses", obj(
|
||||
fmt.Sprint(status), obj("description", desc, "content", content),
|
||||
"401", obj("$ref", "#/components/responses/Unauthorized"),
|
||||
"default", obj("$ref", "#/components/responses/Error"),
|
||||
))
|
||||
return op, nil
|
||||
}
|
||||
|
||||
// operationID is the method and the path's words, getModelsNameTarget.
|
||||
func operationID(method, p string) string {
|
||||
var b strings.Builder
|
||||
b.WriteString(strings.ToLower(method))
|
||||
for _, s := range strings.Split(strings.TrimPrefix(p, "/api/v1/"), "/") {
|
||||
for _, w := range strings.FieldsFunc(s, func(r rune) bool { return !unicode.IsLetter(r) && !unicode.IsDigit(r) }) {
|
||||
b.WriteString(strings.ToUpper(w[:1]) + w[1:])
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// tagOf groups an operation by the first word of its path.
|
||||
func tagOf(p string) string {
|
||||
first, _, _ := strings.Cut(strings.TrimPrefix(p, "/api/v1/"), "/")
|
||||
return strings.TrimSuffix(first, ".json")
|
||||
}
|
||||
|
||||
// --- schemas ---
|
||||
|
||||
type typeDoc struct {
|
||||
doc string
|
||||
fields map[string]string
|
||||
}
|
||||
|
||||
type generator struct {
|
||||
root string
|
||||
docs map[string]map[string]typeDoc
|
||||
names map[reflect.Type]string
|
||||
schemas map[string]any
|
||||
seen []reflect.Type
|
||||
}
|
||||
|
||||
var (
|
||||
timeType = reflect.TypeOf(time.Time{})
|
||||
durationType = reflect.TypeOf(time.Duration(0))
|
||||
rawType = reflect.TypeOf(json.RawMessage{})
|
||||
errorType = reflect.TypeOf(adminapi.ErrorView{})
|
||||
marshalerType = reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||
textType = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
|
||||
)
|
||||
|
||||
// collect records every named struct reachable from t.
|
||||
func (g *generator) collect(t reflect.Type) {
|
||||
for t.Kind() == reflect.Pointer || t.Kind() == reflect.Slice || t.Kind() == reflect.Array || t.Kind() == reflect.Map {
|
||||
if t == rawType {
|
||||
return
|
||||
}
|
||||
t = t.Elem()
|
||||
}
|
||||
if t.Kind() != reflect.Struct || special(t) {
|
||||
return
|
||||
}
|
||||
if t.Name() != "" {
|
||||
if slices.Contains(g.seen, t) {
|
||||
return
|
||||
}
|
||||
g.seen = append(g.seen, t)
|
||||
}
|
||||
for _, f := range fields(t) {
|
||||
g.collect(f.typ)
|
||||
}
|
||||
}
|
||||
|
||||
// assignNames names each struct's schema: by its name alone when it is the
|
||||
// admin API's own, with its package's otherwise — catalog.Model,
|
||||
// modelserve.Job. Qualified always rather than only on a collision, so a
|
||||
// type added elsewhere one day cannot rename a schema clients already use.
|
||||
func (g *generator) assignNames() {
|
||||
own := reflect.TypeOf(adminapi.Route{}).PkgPath()
|
||||
for _, t := range g.seen {
|
||||
n := t.Name()
|
||||
if t.PkgPath() != own {
|
||||
n = path.Base(t.PkgPath()) + "." + n
|
||||
}
|
||||
g.names[t] = n
|
||||
}
|
||||
g.schemas = map[string]any{}
|
||||
}
|
||||
|
||||
func (g *generator) nameOf(t reflect.Type) string { return g.names[t] }
|
||||
|
||||
// special is a type encoded other than by its fields.
|
||||
func special(t reflect.Type) bool {
|
||||
if t == timeType || t == durationType || t == rawType {
|
||||
return true
|
||||
}
|
||||
return t.Implements(marshalerType) || reflect.PointerTo(t).Implements(marshalerType) ||
|
||||
t.Implements(textType) || reflect.PointerTo(t).Implements(textType)
|
||||
}
|
||||
|
||||
func ref(name string) object { return obj("$ref", "#/components/schemas/"+name) }
|
||||
|
||||
// schema describes t, adding the named structs it reaches to the components.
|
||||
func (g *generator) schema(t reflect.Type) any {
|
||||
switch {
|
||||
case t == timeType:
|
||||
return obj("type", "string", "format", "date-time")
|
||||
case t == durationType:
|
||||
return obj("type", "integer", "format", "int64", "description", "A duration, in nanoseconds.")
|
||||
case t == rawType:
|
||||
return obj("description", "Any JSON value.")
|
||||
case t.Implements(textType) || reflect.PointerTo(t).Implements(textType):
|
||||
return obj("type", "string")
|
||||
case t.Implements(marshalerType) || reflect.PointerTo(t).Implements(marshalerType):
|
||||
return obj("description", "Encoded by the type's own JSON encoding.")
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Bool:
|
||||
return obj("type", "boolean")
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Uint8, reflect.Uint16, reflect.Uint32:
|
||||
return obj("type", "integer")
|
||||
case reflect.Int64, reflect.Uint, reflect.Uint64:
|
||||
return obj("type", "integer", "format", "int64")
|
||||
case reflect.Float32, reflect.Float64:
|
||||
return obj("type", "number")
|
||||
case reflect.String:
|
||||
return obj("type", "string")
|
||||
case reflect.Pointer:
|
||||
return g.schema(t.Elem())
|
||||
case reflect.Slice, reflect.Array:
|
||||
if t.Elem().Kind() == reflect.Uint8 {
|
||||
return obj("type", "string", "contentEncoding", "base64")
|
||||
}
|
||||
return obj("type", "array", "items", g.schema(t.Elem()))
|
||||
case reflect.Map:
|
||||
return obj("type", "object", "additionalProperties", g.schema(t.Elem()))
|
||||
case reflect.Interface:
|
||||
return obj()
|
||||
case reflect.Struct:
|
||||
if t.Name() == "" {
|
||||
return g.structSchema(t, "")
|
||||
}
|
||||
name := g.names[t]
|
||||
if name == "" {
|
||||
// Reached only through a body collect did not walk, which would be
|
||||
// a bug here; named anyway rather than inlined so it shows.
|
||||
name = t.Name()
|
||||
g.names[t] = name
|
||||
}
|
||||
if _, ok := g.schemas[name]; !ok {
|
||||
g.schemas[name] = nil // placeholder: a type that contains itself
|
||||
g.schemas[name] = g.structSchema(t, g.typeDocOf(t).doc)
|
||||
}
|
||||
return ref(name)
|
||||
}
|
||||
return obj()
|
||||
}
|
||||
|
||||
func (g *generator) structSchema(t reflect.Type, doc string) object {
|
||||
s := obj("type", "object")
|
||||
if doc != "" {
|
||||
s.set("description", doc)
|
||||
}
|
||||
var props object
|
||||
for _, f := range fields(t) {
|
||||
var p any
|
||||
if f.asString {
|
||||
p = obj("type", "string")
|
||||
} else {
|
||||
p = g.schema(f.typ)
|
||||
}
|
||||
if d := g.typeDocOf(f.owner).fields[f.goName]; d != "" {
|
||||
p = withDescription(p, d)
|
||||
}
|
||||
props.set(f.name, p)
|
||||
}
|
||||
s.set("properties", props)
|
||||
return s
|
||||
}
|
||||
|
||||
// withDescription puts a field's comment on its schema. A reference keeps
|
||||
// its own description where it is declared; in 3.1 a sibling description
|
||||
// beside $ref is allowed and is the field's.
|
||||
func withDescription(p any, d string) any {
|
||||
o, ok := p.(object)
|
||||
if !ok {
|
||||
return p
|
||||
}
|
||||
if prev, ok := o.get("description"); ok {
|
||||
d = d + "\n\n" + prev.(string)
|
||||
}
|
||||
o = slices.Clone(o)
|
||||
o.set("description", d)
|
||||
return o
|
||||
}
|
||||
|
||||
// field is one key of a struct's JSON.
|
||||
type field struct {
|
||||
name string
|
||||
goName string
|
||||
owner reflect.Type // the struct it is declared in, for its comment
|
||||
typ reflect.Type
|
||||
asString bool
|
||||
depth int
|
||||
}
|
||||
|
||||
// fields lists a struct's JSON keys as encoding/json writes them: exported
|
||||
// fields, by tag name, "-" left out, embedded structs without a name of
|
||||
// their own flattened, and a key at a shallower depth winning over the same
|
||||
// key from deeper in.
|
||||
func fields(t reflect.Type) []field {
|
||||
var out []field
|
||||
var walk func(t reflect.Type, depth int)
|
||||
walk = func(t reflect.Type, depth int) {
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
sf := t.Field(i)
|
||||
tag := sf.Tag.Get("json")
|
||||
if tag == "-" {
|
||||
continue
|
||||
}
|
||||
name, opts, _ := strings.Cut(tag, ",")
|
||||
ft := sf.Type
|
||||
if sf.Anonymous && name == "" {
|
||||
et := ft
|
||||
if et.Kind() == reflect.Pointer {
|
||||
et = et.Elem()
|
||||
}
|
||||
if et.Kind() == reflect.Struct && !special(et) {
|
||||
walk(et, depth+1)
|
||||
continue
|
||||
}
|
||||
}
|
||||
if !sf.IsExported() {
|
||||
continue
|
||||
}
|
||||
if name == "" {
|
||||
name = sf.Name
|
||||
}
|
||||
out = append(out, field{name: name, goName: sf.Name, owner: t, typ: ft, depth: depth,
|
||||
asString: slices.Contains(strings.Split(opts, ","), "string")})
|
||||
}
|
||||
}
|
||||
walk(t, 0)
|
||||
// The shallowest of each name, in the order they were met.
|
||||
best := map[string]int{}
|
||||
for _, f := range out {
|
||||
if d, ok := best[f.name]; !ok || f.depth < d {
|
||||
best[f.name] = f.depth
|
||||
}
|
||||
}
|
||||
var kept []field
|
||||
taken := map[string]bool{}
|
||||
for _, f := range out {
|
||||
if f.depth == best[f.name] && !taken[f.name] {
|
||||
taken[f.name] = true
|
||||
kept = append(kept, f)
|
||||
}
|
||||
}
|
||||
return kept
|
||||
}
|
||||
|
||||
// typeDocOf reads a type's comment and its fields' from its package's source.
|
||||
func (g *generator) typeDocOf(t reflect.Type) typeDoc {
|
||||
pkg := t.PkgPath()
|
||||
if pkg != Module && !strings.HasPrefix(pkg, Module+"/") {
|
||||
return typeDoc{}
|
||||
}
|
||||
docs, ok := g.docs[pkg]
|
||||
if !ok {
|
||||
docs = parseDocs(filepath.Join(g.root, filepath.FromSlash(strings.TrimPrefix(strings.TrimPrefix(pkg, Module), "/"))))
|
||||
g.docs[pkg] = docs
|
||||
}
|
||||
return docs[t.Name()]
|
||||
}
|
||||
|
||||
// parseDocs reads the struct types declared in a directory and their
|
||||
// comments. A directory it cannot read gives no comments rather than an
|
||||
// error: the document is still right, only quieter.
|
||||
func parseDocs(dir string) map[string]typeDoc {
|
||||
out := map[string]typeDoc{}
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
fset := token.NewFileSet()
|
||||
for _, e := range entries {
|
||||
n := e.Name()
|
||||
if e.IsDir() || !strings.HasSuffix(n, ".go") || strings.HasSuffix(n, "_test.go") {
|
||||
continue
|
||||
}
|
||||
f, err := parser.ParseFile(fset, filepath.Join(dir, n), nil, parser.ParseComments)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for _, d := range f.Decls {
|
||||
gd, ok := d.(*ast.GenDecl)
|
||||
if !ok || gd.Tok != token.TYPE {
|
||||
continue
|
||||
}
|
||||
for _, sp := range gd.Specs {
|
||||
ts := sp.(*ast.TypeSpec)
|
||||
doc := ts.Doc
|
||||
if doc == nil && len(gd.Specs) == 1 {
|
||||
doc = gd.Doc
|
||||
}
|
||||
td := typeDoc{doc: text(doc), fields: map[string]string{}}
|
||||
if st, ok := ts.Type.(*ast.StructType); ok {
|
||||
for _, fl := range st.Fields.List {
|
||||
c := fl.Doc
|
||||
if c == nil {
|
||||
c = fl.Comment
|
||||
}
|
||||
for _, name := range fieldNames(fl) {
|
||||
td.fields[name] = text(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
out[ts.Name.Name] = td
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func fieldNames(fl *ast.Field) []string {
|
||||
if len(fl.Names) > 0 {
|
||||
var out []string
|
||||
for _, n := range fl.Names {
|
||||
out = append(out, n.Name)
|
||||
}
|
||||
return out
|
||||
}
|
||||
// Embedded: named by its type.
|
||||
t := fl.Type
|
||||
if s, ok := t.(*ast.StarExpr); ok {
|
||||
t = s.X
|
||||
}
|
||||
switch x := t.(type) {
|
||||
case *ast.Ident:
|
||||
return []string{x.Name}
|
||||
case *ast.SelectorExpr:
|
||||
return []string{x.Sel.Name}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// text is a comment as prose: the comment markers gone, lines within a
|
||||
// paragraph joined, paragraphs kept apart.
|
||||
func text(g *ast.CommentGroup) string {
|
||||
if g == nil {
|
||||
return ""
|
||||
}
|
||||
var paras []string
|
||||
var cur []string
|
||||
for _, line := range strings.Split(g.Text(), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" {
|
||||
if len(cur) > 0 {
|
||||
paras = append(paras, strings.Join(cur, " "))
|
||||
cur = nil
|
||||
}
|
||||
continue
|
||||
}
|
||||
cur = append(cur, line)
|
||||
}
|
||||
if len(cur) > 0 {
|
||||
paras = append(paras, strings.Join(cur, " "))
|
||||
}
|
||||
return strings.Join(paras, "\n\n")
|
||||
}
|
||||
|
||||
// --- an object that keeps its keys in order ---
|
||||
|
||||
type kv struct {
|
||||
k string
|
||||
v any
|
||||
}
|
||||
|
||||
// object is a JSON object written in the order its keys were set, so the
|
||||
// document reads in the order it was meant to — openapi, info, paths — and
|
||||
// a struct's properties in the order its fields are declared.
|
||||
type object []kv
|
||||
|
||||
func obj(pairs ...any) object {
|
||||
var o object
|
||||
for i := 0; i+1 < len(pairs); i += 2 {
|
||||
o.set(pairs[i].(string), pairs[i+1])
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
func (o *object) set(k string, v any) {
|
||||
for i := range *o {
|
||||
if (*o)[i].k == k {
|
||||
(*o)[i].v = v
|
||||
return
|
||||
}
|
||||
}
|
||||
*o = append(*o, kv{k, v})
|
||||
}
|
||||
|
||||
func (o object) get(k string) (any, bool) {
|
||||
for _, p := range o {
|
||||
if p.k == k {
|
||||
return p.v, true
|
||||
}
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// MarshalJSON writes the keys in order.
|
||||
func (o object) MarshalJSON() ([]byte, error) {
|
||||
var b bytes.Buffer
|
||||
b.WriteByte('{')
|
||||
for i, p := range o {
|
||||
if i > 0 {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
k, err := marshal(p.k)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.Write(k)
|
||||
b.WriteByte(':')
|
||||
v, err := marshal(p.v)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.Write(v)
|
||||
}
|
||||
b.WriteByte('}')
|
||||
return b.Bytes(), nil
|
||||
}
|
||||
|
||||
// marshal is json.Marshal without escaping <, > and &, which are common in
|
||||
// the descriptions and harmless in a document nobody puts in a page.
|
||||
func marshal(v any) ([]byte, error) {
|
||||
var b bytes.Buffer
|
||||
enc := json.NewEncoder(&b)
|
||||
enc.SetEscapeHTML(false)
|
||||
if err := enc.Encode(v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return bytes.TrimRight(b.Bytes(), "\n"), nil
|
||||
}
|
||||
|
||||
func sortedKeys(m map[string]bool) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package openapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
|
||||
)
|
||||
|
||||
func repoRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("cannot locate this test")
|
||||
}
|
||||
return filepath.Join(filepath.Dir(file), "..", "..", "..")
|
||||
}
|
||||
|
||||
func generated(t *testing.T) map[string]any {
|
||||
t.Helper()
|
||||
b, err := Generate(repoRoot(t), adminapi.Routes, adminapi.PathParams)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var doc map[string]any
|
||||
if err := json.Unmarshal(b, &doc); err != nil {
|
||||
t.Fatalf("the document is not JSON: %v", err)
|
||||
}
|
||||
return doc
|
||||
}
|
||||
|
||||
// at walks a decoded document by keys.
|
||||
func at(t *testing.T, v any, keys ...string) any {
|
||||
t.Helper()
|
||||
for _, k := range keys {
|
||||
m, ok := v.(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("%s: not an object", strings.Join(keys, "."))
|
||||
}
|
||||
v, ok = m[k]
|
||||
if !ok {
|
||||
t.Fatalf("no %s (of %s)", k, strings.Join(keys, "."))
|
||||
}
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func TestTheAdminAPIDocumentIsNotStale(t *testing.T) {
|
||||
// Like the configuration reference: a description of the API that is not
|
||||
// regenerated describes some earlier week's API, and the server serves the
|
||||
// committed copy to every client generated from it.
|
||||
want, err := Generate(repoRoot(t), adminapi.Routes, adminapi.PathParams)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := os.ReadFile(filepath.Join(repoRoot(t), "docs", "admin-api.openapi.json"))
|
||||
if err != nil {
|
||||
t.Fatalf("docs/admin-api.openapi.json is missing; run go generate ./internal/docgen: %v", err)
|
||||
}
|
||||
if string(got) != string(want) {
|
||||
t.Error("docs/admin-api.openapi.json no longer matches the admin API's routes and types.\n" +
|
||||
"Run: go generate ./internal/docgen")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryReferenceInTheDocumentResolves(t *testing.T) {
|
||||
doc := generated(t)
|
||||
var walk func(v any)
|
||||
n := 0
|
||||
walk = func(v any) {
|
||||
switch x := v.(type) {
|
||||
case map[string]any:
|
||||
if r, ok := x["$ref"].(string); ok {
|
||||
n++
|
||||
parts := strings.Split(strings.TrimPrefix(r, "#/"), "/")
|
||||
at(t, doc, parts...)
|
||||
}
|
||||
for _, e := range x {
|
||||
walk(e)
|
||||
}
|
||||
case []any:
|
||||
for _, e := range x {
|
||||
walk(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
walk(doc)
|
||||
if n < 50 {
|
||||
t.Errorf("only %d references; the schemas are not being shared", n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestABodyIsDescribedByItsGoType(t *testing.T) {
|
||||
doc := generated(t)
|
||||
scale := at(t, doc, "paths", "/api/v1/models/{name}/scale", "post")
|
||||
if r := at(t, scale, "requestBody", "content", "application/json", "schema", "$ref"); r != "#/components/schemas/ScaleRequest" {
|
||||
t.Errorf("scale's body is %v", r)
|
||||
}
|
||||
props := at(t, doc, "components", "schemas", "ScaleRequest", "properties").(map[string]any)
|
||||
// The tag's name, not the field's.
|
||||
for _, k := range []string{"min", "max", "idle_timeout"} {
|
||||
if _, ok := props[k]; !ok {
|
||||
t.Errorf("ScaleRequest has no %s: %v", k, props)
|
||||
}
|
||||
}
|
||||
// The field's comment is its description, and a duration says its unit.
|
||||
if d := at(t, props, "max", "description").(string); !strings.Contains(d, "follows Min") {
|
||||
t.Errorf("max lost its comment: %q", d)
|
||||
}
|
||||
if d := at(t, props, "idle_timeout", "description").(string); !strings.Contains(d, "nanoseconds") {
|
||||
t.Errorf("idle_timeout does not say it is nanoseconds: %q", d)
|
||||
}
|
||||
// A type from another package is named with its package.
|
||||
if r := at(t, doc, "paths", "/api/v1/models", "post", "requestBody", "content", "application/json", "schema", "$ref"); r != "#/components/schemas/catalog.Model" {
|
||||
t.Errorf("adding a model takes %v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnEmbeddedStructsFieldsAreItsOwn(t *testing.T) {
|
||||
// CapacityView embeds bench.Capacity, and encoding/json writes the
|
||||
// embedded fields at the top level: so must the schema.
|
||||
doc := generated(t)
|
||||
props := at(t, doc, "components", "schemas", "CapacityView", "properties").(map[string]any)
|
||||
for _, k := range []string{"measured_by", "model", "profile"} {
|
||||
if _, ok := props[k]; !ok {
|
||||
t.Errorf("CapacityView has no %s: %v", k, props)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryChangeNeedsTheTokenAndNamesItsActor(t *testing.T) {
|
||||
doc := generated(t)
|
||||
if s := at(t, doc, "components", "securitySchemes", "adminToken", "scheme"); s != "bearer" {
|
||||
t.Errorf("scheme %v", s)
|
||||
}
|
||||
if len(at(t, doc, "security").([]any)) != 1 {
|
||||
t.Error("the token is not required document-wide")
|
||||
}
|
||||
del := at(t, doc, "paths", "/api/v1/models/{name}", "delete")
|
||||
found := false
|
||||
for _, p := range at(t, del, "parameters").([]any) {
|
||||
if p.(map[string]any)["$ref"] == "#/components/parameters/Actor" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Error("removing a model does not take X-Llmgw-Actor")
|
||||
}
|
||||
if _, ok := at(t, del, "responses").(map[string]any)["401"]; !ok {
|
||||
t.Error("removing a model does not say it can be refused for the token")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPathParameterWithNoDescriptionIsAnError(t *testing.T) {
|
||||
routes := []adminapi.Route{{Method: "GET", Path: "/api/v1/things/{id}", Summary: "x", Response: adminapi.Result{}}}
|
||||
if _, err := Generate(repoRoot(t), routes, adminapi.PathParams); err == nil {
|
||||
t.Error("a parameter nobody described was let through")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
# C19: OpenAPI for the admin API, and a Terraform provider
|
||||
|
||||
## CHANGELOG
|
||||
|
||||
### The admin API described, and a Terraform provider
|
||||
|
||||
- **The admin API in OpenAPI 3.1.** Every endpoint under `/api/v1`, its path
|
||||
and query parameters, and the schema of each request and answer with the
|
||||
meaning of each field, in
|
||||
[docs/admin-api.openapi.json](docs/admin-api.openapi.json) and served by
|
||||
every control plane at `GET /api/v1/openapi.json` (with the token, like the
|
||||
rest). The bearer token, the `X-Llmgw-Actor` header and the error shape are
|
||||
in it. Generated by `go generate ./internal/docgen` from a route table in
|
||||
`internal/control/adminapi/routes.go` and the Go types the handlers use, with
|
||||
the fields' comments as descriptions; a test fails when an endpoint is
|
||||
served and not in the table, another when the committed document is stale.
|
||||
Checked with openapi-spec-validator and openapi-generator-cli's validate.
|
||||
- **[docs/admin-api.md](docs/admin-api.md)**: how to reach the API (loopback,
|
||||
an SSH tunnel, `admin_remote`), the token at `/var/lib/llmgw/admin-token`,
|
||||
what a request must carry — an address in `Host`, `application/json` on
|
||||
every change, even a `DELETE` — and some calls with `curl`.
|
||||
- **`GET /api/v1/models/{name}`**: one model's whole definition as it is in
|
||||
force, with the defaults the catalog filled in. The listing has only its
|
||||
state and its profiles' names.
|
||||
- **A Terraform provider**, in `terraform-provider-llmgw/`, a module of its
|
||||
own so the gateway does not carry its dependencies. `llmgw_model` (a model
|
||||
added at runtime: scaling, class, owner and adapters change in place, the
|
||||
rest replaces it), `llmgw_quota`, `llmgw_guard_attachment`, and the
|
||||
`llmgw_models` and `llmgw_nodes` data sources. Its client is checked
|
||||
against the OpenAPI document, endpoint by endpoint and field by field, so a
|
||||
change to the admin API that breaks it fails its tests; CI runs them.
|
||||
Built and installed by hand for now: see its
|
||||
[README](terraform-provider-llmgw/README.md).
|
||||
|
||||
## BACKLOG
|
||||
|
||||
**Done (branch `features-c2`):** `adminapi.Routes` and
|
||||
`internal/docgen/openapi` generating `docs/admin-api.openapi.json` (3.1,
|
||||
bearer security, every route, query parameters, schemas from the Go types
|
||||
with their comments), served embedded at `GET /api/v1/openapi.json`, kept true
|
||||
by a route-table test and a staleness test; validated with
|
||||
openapi-spec-validator and openapi-generator-cli. `GET /api/v1/models/{name}`.
|
||||
`docs/admin-api.md`. `terraform-provider-llmgw/` (terraform-plugin-framework):
|
||||
`llmgw_model`, `llmgw_quota`, `llmgw_guard_attachment`, data sources
|
||||
`llmgw_models` and `llmgw_nodes`; client tests against a fake admin API and
|
||||
the OpenAPI document; acceptance tests with Terraform and OpenTofu against the
|
||||
fake; one apply, empty plan, in-place change and destroy against a real
|
||||
`llmgw server` run by hand. Left: organizations, teams, projects, people and
|
||||
keys are not in the admin API (they are `llmgw org`/`users`/`keys` against the
|
||||
store), so not in the provider; a model's definition beyond scaling, class,
|
||||
owner and adapters is changed by replacing it (the API has no edit, and
|
||||
`resources` is not used yet); `canary`, `slo`, `scaling.schedule` and
|
||||
`prewarm`, `guard` role and `origin` are not in `llmgw_model`; models cannot be
|
||||
imported; the provider is not published to a registry and has no release
|
||||
build; the acceptance tests are not run in CI (they need Terraform); the
|
||||
document's schemas mark nothing required and give no enums; `catalog.SLO`'s
|
||||
fields have no JSON tags, so they are `TTFTp99` and `TPOTp99` on the wire.
|
||||
@@ -0,0 +1,135 @@
|
||||
# terraform-provider-llmgw
|
||||
|
||||
A Terraform (and OpenTofu) provider for llmgw: the models in a cluster's
|
||||
catalog, capacity quotas on its teams and projects, and where its guardrail
|
||||
policies apply, kept in configuration and reviewed like the rest of it.
|
||||
|
||||
It talks to the control plane's admin API with the admin token, the same API
|
||||
`llmgw ctl` uses; see [docs/admin-api.md](../docs/admin-api.md) for how that
|
||||
API is reached and what it asks of a request.
|
||||
|
||||
## What it manages
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| `llmgw_model` | A model added to the catalog at runtime, as `llmgw ctl model add` adds one. Scaling, class, owner and adapters change in place; any other change to the definition replaces the model, which stops its replicas while it is added again. |
|
||||
| `llmgw_quota` | A capacity quota on an organization, team or project: a guarantee, a limit, or both, in GPUs, CPU cores and memory. |
|
||||
| `llmgw_guard_attachment` | A guardrail policy attached to an organization, team, project, key or person. The policies are defined in `server.yaml`'s `guard:` section; this says where they apply. |
|
||||
| `data.llmgw_models` | Every model in the catalog, with its replicas now. |
|
||||
| `data.llmgw_nodes` | Every node, with its cards and state now. |
|
||||
|
||||
Not here, because the admin API does not serve them: organizations, teams,
|
||||
projects, people and API keys. Those live in the store and are managed with
|
||||
`llmgw org`, `llmgw users` and `llmgw keys` on the control plane. A quota or
|
||||
an attachment names a team that already exists, by its path or its ID.
|
||||
|
||||
Models declared in the control plane's `server.yaml` are not managed here
|
||||
either: the cluster refuses to remove or change them through the API, since
|
||||
the configuration file would put them back on the next restart.
|
||||
|
||||
## Configuring it
|
||||
|
||||
```hcl
|
||||
terraform {
|
||||
required_providers {
|
||||
llmgw = { source = "gitea.mynodes.xyz/ilker/llmgw" }
|
||||
}
|
||||
}
|
||||
|
||||
provider "llmgw" {
|
||||
endpoint = "http://127.0.0.1:9090" # or LLMGW_ADMIN_URL
|
||||
# token = "..." # or LLMGW_ADMIN_TOKEN
|
||||
token_file = "/var/lib/llmgw/admin-token" # read when no token is given
|
||||
actor = "terraform" # who the audit log says made the change
|
||||
}
|
||||
```
|
||||
|
||||
All four are optional and default to what is shown. The admin listener is on
|
||||
loopback, so run Terraform on the control plane — as a user that can read the
|
||||
token file, which is root's — or from elsewhere through a tunnel:
|
||||
|
||||
```sh
|
||||
ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
|
||||
export LLMGW_ADMIN_TOKEN=$(ssh admin@control-plane sudo cat /var/lib/llmgw/admin-token)
|
||||
terraform apply
|
||||
```
|
||||
|
||||
The endpoint must be an address, not a name: the admin listener refuses a
|
||||
`Host` that is a DNS name.
|
||||
|
||||
[examples/main.tf](examples/main.tf) has a model on CPUs, one on GPUs, a team's
|
||||
quota, a policy attachment and the two data sources.
|
||||
|
||||
## Installing it
|
||||
|
||||
It is not in a public registry. Build it and point Terraform at it, either as
|
||||
a development override:
|
||||
|
||||
```sh
|
||||
cd terraform-provider-llmgw
|
||||
go build -o ~/.terraform.d/llmgw/ .
|
||||
cat >> ~/.terraformrc <<'EOF'
|
||||
provider_installation {
|
||||
dev_overrides {
|
||||
"gitea.mynodes.xyz/ilker/llmgw" = "/home/you/.terraform.d/llmgw"
|
||||
}
|
||||
direct {}
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
or in a filesystem mirror, which `terraform init` installs from like a
|
||||
registry and records in the lock file:
|
||||
|
||||
```sh
|
||||
v=0.1.0
|
||||
d=~/.terraform.d/plugins/gitea.mynodes.xyz/ilker/llmgw/$v/$(go env GOOS)_$(go env GOARCH)
|
||||
mkdir -p $d
|
||||
go build -ldflags "-X main.version=$v" -o $d/terraform-provider-llmgw_v$v .
|
||||
```
|
||||
|
||||
with `version = "0.1.0"` beside `source` in `required_providers`.
|
||||
|
||||
## How the state behaves
|
||||
|
||||
- **A model's definition is kept as written.** The catalog fills in what a
|
||||
definition leaves out — `tp = 1`, `engine = "vllm"`, a 20-minute start-up
|
||||
timeout — and the state does not take those values, so a plan does not show
|
||||
them as differences. What can change in place is read back from the
|
||||
cluster: a model scaled or reclassified from a terminal shows as a change to
|
||||
put right on the next plan, and one removed from a terminal is added again.
|
||||
- **`scaling.max` left out follows `min`**, as in the configuration file, and
|
||||
is read back. Leaving the whole `scaling` block out means no replicas: the
|
||||
model is in the catalog and does not run.
|
||||
- **Durations are written as Go writes them**, `"30m"` or `"1h30m"`.
|
||||
- **An owner or a quota's scope is kept as written** (`team:Acme/ml`) beside
|
||||
the ID the cluster resolved it to (`owner_id`, and the quota's `id`), and is
|
||||
found again by the ID, so renaming the team in the store does not lose it.
|
||||
- **Import** works for quotas, by `team:ID` as `GET /api/v1/quotas` lists it
|
||||
(the `scope` is then that ID; write it so in the configuration, or the next
|
||||
plan replaces the quota with itself), and for attachments, by
|
||||
`scope:SCOPE-ID:policy` as their `id` shows. Models cannot be imported yet.
|
||||
|
||||
## Developing it
|
||||
|
||||
A module of its own, so the gateway's binary does not carry Terraform's
|
||||
dependencies; `go test ./...` at the repository's root does not include it.
|
||||
|
||||
```sh
|
||||
cd terraform-provider-llmgw
|
||||
go vet ./... && go test ./... # the client, against an admin API in memory and the OpenAPI document
|
||||
TF_ACC=1 go test ./internal/provider/ # Terraform itself, plan and apply, against the same fake
|
||||
```
|
||||
|
||||
- `internal/client` is the HTTP client. Its `Endpoints` lists every endpoint it
|
||||
calls and it refuses any other; a test checks each of them, and every field
|
||||
of every body it sends and reads, against the gateway's generated
|
||||
[OpenAPI document](../docs/admin-api.openapi.json). A change to the admin API
|
||||
that breaks the provider fails here.
|
||||
- `internal/fakeapi` is an admin API in memory: the token, the content type,
|
||||
the catalog's defaults and the tenancy tree's names, enough to run the
|
||||
provider without a cluster.
|
||||
- The acceptance tests need `TF_ACC=1` and a `terraform` on `PATH` (or
|
||||
`TF_ACC_TERRAFORM_PATH`); without one, terraform-plugin-testing downloads the
|
||||
latest. They pass with OpenTofu too:
|
||||
`TF_ACC_TERRAFORM_PATH=$(which tofu) TF_ACC_PROVIDER_HOST=registry.opentofu.org`.
|
||||
@@ -0,0 +1,94 @@
|
||||
# A model, a team's quota and a guardrail policy, managed through the control
|
||||
# plane's admin API. Run where the admin listener is reachable: on the control
|
||||
# plane itself, or through `ssh -N -L 9090:127.0.0.1:9090 control-plane`.
|
||||
|
||||
terraform {
|
||||
required_providers {
|
||||
llmgw = {
|
||||
source = "gitea.mynodes.xyz/ilker/llmgw"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
provider "llmgw" {
|
||||
# The defaults, written out. The token comes from LLMGW_ADMIN_TOKEN when it
|
||||
# is set, and otherwise from token_file, which only root can read.
|
||||
endpoint = "http://127.0.0.1:9090"
|
||||
token_file = "/var/lib/llmgw/admin-token"
|
||||
actor = "terraform"
|
||||
}
|
||||
|
||||
# A small model on CPUs, loaded when asked for and stopped after half an hour
|
||||
# unused.
|
||||
resource "llmgw_model" "qwen_small" {
|
||||
name = "qwen-small"
|
||||
description = "Qwen 2.5 1.5B instruct, for quick questions"
|
||||
source = "/srv/models/qwen2.5-1.5b-instruct-q4_k_m.gguf"
|
||||
aliases = ["qwen"]
|
||||
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 4
|
||||
cpu_cores = 4
|
||||
}]
|
||||
|
||||
scaling = {
|
||||
min = 0
|
||||
max = 2
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
|
||||
class = "opportunistic"
|
||||
owner = "team:Acme/ml"
|
||||
}
|
||||
|
||||
# A larger one on GPUs, always running.
|
||||
resource "llmgw_model" "llama_70b" {
|
||||
name = "llama-3.3-70b"
|
||||
source = "meta-llama/Llama-3.3-70B-Instruct"
|
||||
|
||||
profiles = [{
|
||||
name = "h100"
|
||||
engine = "vllm"
|
||||
gpu_class = "h100-80g"
|
||||
tp = 4
|
||||
}]
|
||||
|
||||
scaling = {
|
||||
min = 1
|
||||
max = 3
|
||||
}
|
||||
|
||||
capabilities = {
|
||||
tools = true
|
||||
}
|
||||
fallbacks = [llmgw_model.qwen_small.name]
|
||||
}
|
||||
|
||||
# The ml team may hold at most eight GPUs, and is always given two.
|
||||
resource "llmgw_quota" "ml" {
|
||||
scope = "team:Acme/ml"
|
||||
guarantee = { gpus = 2 }
|
||||
limit = { gpus = 8 }
|
||||
}
|
||||
|
||||
# The finance policy, from server.yaml's guard.policies, on the ml team.
|
||||
resource "llmgw_guard_attachment" "ml_finance" {
|
||||
target = "team:Acme/ml"
|
||||
policy = "finance"
|
||||
}
|
||||
|
||||
data "llmgw_nodes" "all" {}
|
||||
|
||||
data "llmgw_models" "all" {
|
||||
depends_on = [llmgw_model.qwen_small, llmgw_model.llama_70b]
|
||||
}
|
||||
|
||||
output "free_gpus" {
|
||||
value = sum([for n in data.llmgw_nodes.all.nodes : n.gpu_free if n.state == "ready"])
|
||||
}
|
||||
|
||||
output "models_not_ready" {
|
||||
value = [for m in data.llmgw_models.all.models : m.name if m.ready_replicas < m.min_replicas]
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
module gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw
|
||||
|
||||
go 1.26.6
|
||||
|
||||
require (
|
||||
github.com/hashicorp/terraform-plugin-framework v1.19.0
|
||||
github.com/hashicorp/terraform-plugin-go v0.31.0
|
||||
github.com/hashicorp/terraform-plugin-testing v1.16.0
|
||||
)
|
||||
|
||||
require (
|
||||
github.com/ProtonMail/go-crypto v1.4.1 // indirect
|
||||
github.com/agext/levenshtein v1.2.3 // indirect
|
||||
github.com/apparentlymart/go-textseg/v15 v15.0.0 // indirect
|
||||
github.com/cloudflare/circl v1.6.3 // indirect
|
||||
github.com/fatih/color v1.18.0 // indirect
|
||||
github.com/golang/protobuf v1.5.4 // indirect
|
||||
github.com/google/go-cmp v0.7.0 // indirect
|
||||
github.com/hashicorp/errwrap v1.1.0 // indirect
|
||||
github.com/hashicorp/go-checkpoint v0.5.0 // indirect
|
||||
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
|
||||
github.com/hashicorp/go-cty v1.5.0 // indirect
|
||||
github.com/hashicorp/go-hclog v1.6.3 // indirect
|
||||
github.com/hashicorp/go-multierror v1.1.1 // indirect
|
||||
github.com/hashicorp/go-plugin v1.7.0 // indirect
|
||||
github.com/hashicorp/go-retryablehttp v0.7.8 // indirect
|
||||
github.com/hashicorp/go-uuid v1.0.3 // indirect
|
||||
github.com/hashicorp/go-version v1.9.0 // indirect
|
||||
github.com/hashicorp/hc-install v0.9.4 // indirect
|
||||
github.com/hashicorp/hcl/v2 v2.24.0 // indirect
|
||||
github.com/hashicorp/logutils v1.0.0 // indirect
|
||||
github.com/hashicorp/terraform-exec v0.25.1 // indirect
|
||||
github.com/hashicorp/terraform-json v0.27.2 // indirect
|
||||
github.com/hashicorp/terraform-plugin-log v0.10.0 // indirect
|
||||
github.com/hashicorp/terraform-plugin-sdk/v2 v2.40.0 // indirect
|
||||
github.com/hashicorp/terraform-registry-address v0.4.0 // indirect
|
||||
github.com/hashicorp/terraform-svchost v0.2.1 // indirect
|
||||
github.com/hashicorp/yamux v0.1.2 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mitchellh/copystructure v1.2.0 // indirect
|
||||
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
|
||||
github.com/mitchellh/go-wordwrap v1.0.1 // indirect
|
||||
github.com/mitchellh/mapstructure v1.5.0 // indirect
|
||||
github.com/mitchellh/reflectwalk v1.0.2 // indirect
|
||||
github.com/oklog/run v1.2.0 // indirect
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible // indirect
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect
|
||||
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
|
||||
github.com/zclconf/go-cty v1.18.1 // indirect
|
||||
golang.org/x/crypto v0.50.0 // indirect
|
||||
golang.org/x/mod v0.35.0 // indirect
|
||||
golang.org/x/net v0.52.0 // indirect
|
||||
golang.org/x/sync v0.20.0 // indirect
|
||||
golang.org/x/sys v0.43.0 // indirect
|
||||
golang.org/x/text v0.36.0 // indirect
|
||||
golang.org/x/tools v0.43.0 // indirect
|
||||
google.golang.org/appengine v1.6.8 // indirect
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 // indirect
|
||||
google.golang.org/grpc v1.79.3 // indirect
|
||||
google.golang.org/protobuf v1.36.11 // indirect
|
||||
)
|
||||
@@ -0,0 +1,242 @@
|
||||
dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
|
||||
dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
|
||||
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
|
||||
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
|
||||
github.com/ProtonMail/go-crypto v1.4.1 h1:9RfcZHqEQUvP8RzecWEUafnZVtEvrBVL9BiF67IQOfM=
|
||||
github.com/ProtonMail/go-crypto v1.4.1/go.mod h1:e1OaTyu5SYVrO9gKOEhTc+5UcXtTUa+P3uLudwcgPqo=
|
||||
github.com/agext/levenshtein v1.2.3 h1:YB2fHEn0UJagG8T1rrWknE3ZQzWM06O8AMAatNn7lmo=
|
||||
github.com/agext/levenshtein v1.2.3/go.mod h1:JEDfjyjHDjOF/1e4FlBE/PkbqA9OfWu2ki2W0IB5558=
|
||||
github.com/apparentlymart/go-textseg/v12 v12.0.0/go.mod h1:S/4uRK2UtaQttw1GenVJEynmyUenKwP++x/+DdGV/Ec=
|
||||
github.com/apparentlymart/go-textseg/v15 v15.0.0 h1:uYvfpb3DyLSCGWnctWKGj857c6ew1u1fNQOlOtuGxQY=
|
||||
github.com/apparentlymart/go-textseg/v15 v15.0.0/go.mod h1:K8XmNZdhEBkdlyDdvbmmsvpAG721bKi0joRfFdHIWJ4=
|
||||
github.com/bufbuild/protocompile v0.14.1 h1:iA73zAf/fyljNjQKwYzUHD6AD4R8KMasmwa/FBatYVw=
|
||||
github.com/bufbuild/protocompile v0.14.1/go.mod h1:ppVdAIhbr2H8asPk6k4pY7t9zB1OU5DoEw9xY/FUi1c=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/cloudflare/circl v1.6.3 h1:9GPOhQGF9MCYUeXyMYlqTR6a5gTrgR/fBLXvUgtVcg8=
|
||||
github.com/cloudflare/circl v1.6.3/go.mod h1:2eXP6Qfat4O/Yhh8BznvKnJ+uzEoTQ6jVKJRn81BiS4=
|
||||
github.com/cyphar/filepath-securejoin v0.4.1 h1:JyxxyPEaktOD+GAnqIqTf9A8tHyAG22rowi7HkoSU1s=
|
||||
github.com/cyphar/filepath-securejoin v0.4.1/go.mod h1:Sdj7gXlvMcPZsbhwhQ33GguGLDGQL7h7bg04C/+u9jI=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
|
||||
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=
|
||||
github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ=
|
||||
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
|
||||
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
|
||||
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376 h1:+zs/tPmkDkHx3U66DAb0lQFJrpS6731Oaa12ikc+DiI=
|
||||
github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376/go.mod h1:an3vInlBmSxCcxctByoQdvwPiA7DTK7jaaFDBTtu0ic=
|
||||
github.com/go-git/go-billy/v5 v5.8.0 h1:I8hjc3LbBlXTtVuFNJuwYuMiHvQJDq1AT6u4DwDzZG0=
|
||||
github.com/go-git/go-billy/v5 v5.8.0/go.mod h1:RpvI/rw4Vr5QA+Z60c6d6LXH0rYJo0uD5SqfmrrheCY=
|
||||
github.com/go-git/go-git/v5 v5.18.0 h1:O831KI+0PR51hM2kep6T8k+w0/LIAD490gvqMCvL5hM=
|
||||
github.com/go-git/go-git/v5 v5.18.0/go.mod h1:pW/VmeqkanRFqR6AljLcs7EA7FbZaN5MQqO7oZADXpo=
|
||||
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
|
||||
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-test/deep v1.0.3 h1:ZrJSEWsXzPOxaZnFteGEfooLba+ju3FYIbOrS+rQd68=
|
||||
github.com/go-test/deep v1.0.3/go.mod h1:wGDj63lr65AM2AQyKZd/NYHGb0R+1RLqB8NKt3aSFNA=
|
||||
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8 h1:f+oWsMOmNPc8JmEHVZIycC7hBoQxHH9pNKQORJNozsQ=
|
||||
github.com/golang/groupcache v0.0.0-20241129210726-2c02b8208cf8/go.mod h1:wcDNUvekVysuuOpQKo3191zZyTpiI6se1N1ULghS0sw=
|
||||
github.com/golang/protobuf v1.1.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
|
||||
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk=
|
||||
github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY=
|
||||
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
|
||||
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
|
||||
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
|
||||
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
|
||||
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
|
||||
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/go-checkpoint v0.5.0 h1:MFYpPZCnQqQTE18jFwSII6eUQrD/oxMFp3mlgcqk5mU=
|
||||
github.com/hashicorp/go-checkpoint v0.5.0/go.mod h1:7nfLNL10NsxqO4iWuW6tWW0HjZuDrwkBuEQsVcpCOgg=
|
||||
github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80=
|
||||
github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
|
||||
github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
|
||||
github.com/hashicorp/go-cty v1.5.0 h1:EkQ/v+dDNUqnuVpmS5fPqyY71NXVgT5gf32+57xY8g0=
|
||||
github.com/hashicorp/go-cty v1.5.0/go.mod h1:lFUCG5kd8exDobgSfyj4ONE/dc822kiYMguVKdHGMLM=
|
||||
github.com/hashicorp/go-hclog v1.6.3 h1:Qr2kF+eVWjTiYmU7Y31tYlP1h0q/X3Nl3tPGdaB11/k=
|
||||
github.com/hashicorp/go-hclog v1.6.3/go.mod h1:W4Qnvbt70Wk/zYJryRzDRU/4r0kIg0PVHBcfoyhpF5M=
|
||||
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
|
||||
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
|
||||
github.com/hashicorp/go-plugin v1.7.0 h1:YghfQH/0QmPNc/AZMTFE3ac8fipZyZECHdDPshfk+mA=
|
||||
github.com/hashicorp/go-plugin v1.7.0/go.mod h1:BExt6KEaIYx804z8k4gRzRLEvxKVb+kn0NMcihqOqb8=
|
||||
github.com/hashicorp/go-retryablehttp v0.7.8 h1:ylXZWnqa7Lhqpk0L1P1LzDtGcCR0rPVUrx/c8Unxc48=
|
||||
github.com/hashicorp/go-retryablehttp v0.7.8/go.mod h1:rjiScheydd+CxvumBsIrFKlx3iS0jrZ7LvzFGFmuKbw=
|
||||
github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
|
||||
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
|
||||
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
|
||||
github.com/hashicorp/go-version v1.9.0 h1:CeOIz6k+LoN3qX9Z0tyQrPtiB1DFYRPfCIBtaXPSCnA=
|
||||
github.com/hashicorp/go-version v1.9.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
|
||||
github.com/hashicorp/hc-install v0.9.4 h1:KKWOpUG0EqIV63Qk2GGFrZ0s275NVs5lKf9N5vjBNoc=
|
||||
github.com/hashicorp/hc-install v0.9.4/go.mod h1:4LRYeEN2bMIFfIv57ldMWt9awfuZhvpbRt0vWmv51WU=
|
||||
github.com/hashicorp/hcl/v2 v2.24.0 h1:2QJdZ454DSsYGoaE6QheQZjtKZSUs9Nh2izTWiwQxvE=
|
||||
github.com/hashicorp/hcl/v2 v2.24.0/go.mod h1:oGoO1FIQYfn/AgyOhlg9qLC6/nOJPX3qGbkZpYAcqfM=
|
||||
github.com/hashicorp/logutils v1.0.0 h1:dLEQVugN8vlakKOUE3ihGLTZJRB4j+M2cdTm/ORI65Y=
|
||||
github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64=
|
||||
github.com/hashicorp/terraform-exec v0.25.1 h1:PRutYRGM8pixV3B8812NYoBK5O+yuf3qcB/70KFKGiU=
|
||||
github.com/hashicorp/terraform-exec v0.25.1/go.mod h1:+izOYrs9sKMQK4OYvGDnrSSJHY/pm4e4eXFqSL2Q5mA=
|
||||
github.com/hashicorp/terraform-json v0.27.2 h1:BwGuzM6iUPqf9JYM/Z4AF1OJ5VVJEEzoKST/tRDBJKU=
|
||||
github.com/hashicorp/terraform-json v0.27.2/go.mod h1:GzPLJ1PLdUG5xL6xn1OXWIjteQRT2CNT9o/6A9mi9hE=
|
||||
github.com/hashicorp/terraform-plugin-framework v1.19.0 h1:q0bwyhxAOR3vfdgbk9iplv3MlTv/dhBHTXjQOtQDoBA=
|
||||
github.com/hashicorp/terraform-plugin-framework v1.19.0/go.mod h1:YRXOBu0jvs7xp4AThBbX4mAzYaMJ1JgtFH//oGKxwLc=
|
||||
github.com/hashicorp/terraform-plugin-go v0.31.0 h1:0Fz2r9DQ+kNNl6bx8HRxFd1TfMKUvnrOtvJPmp3Z0q8=
|
||||
github.com/hashicorp/terraform-plugin-go v0.31.0/go.mod h1:A88bDhd/cW7FnwqxQRz3slT+QY6yzbHKc6AOTtmdeS8=
|
||||
github.com/hashicorp/terraform-plugin-log v0.10.0 h1:eu2kW6/QBVdN4P3Ju2WiB2W3ObjkAsyfBsL3Wh1fj3g=
|
||||
github.com/hashicorp/terraform-plugin-log v0.10.0/go.mod h1:/9RR5Cv2aAbrqcTSdNmY1NRHP4E3ekrXRGjqORpXyB0=
|
||||
github.com/hashicorp/terraform-plugin-sdk/v2 v2.40.0 h1:MKS/2URqeJRwJdbOfcbdsZCq/IRrNkqJNN0GtVIsuGs=
|
||||
github.com/hashicorp/terraform-plugin-sdk/v2 v2.40.0/go.mod h1:PuG4P97Ju3QXW6c6vRkRadWJbvnEu2Xh+oOuqcYOqX4=
|
||||
github.com/hashicorp/terraform-plugin-testing v1.16.0 h1:GB97nGnJ1hESpDrCjqZig38RodSF0gdRzxlDupLXP38=
|
||||
github.com/hashicorp/terraform-plugin-testing v1.16.0/go.mod h1:eQPYAy9xFMV7xtIFX8Y+wJGtUB++HBl329zCF6PBMZk=
|
||||
github.com/hashicorp/terraform-registry-address v0.4.0 h1:S1yCGomj30Sao4l5BMPjTGZmCNzuv7/GDTDX99E9gTk=
|
||||
github.com/hashicorp/terraform-registry-address v0.4.0/go.mod h1:LRS1Ay0+mAiRkUyltGT+UHWkIqTFvigGn/LbMshfflE=
|
||||
github.com/hashicorp/terraform-svchost v0.2.1 h1:ubvrTFw3Q7CsoEaX7V06PtCTKG3wu7GyyobAoN4eF3Q=
|
||||
github.com/hashicorp/terraform-svchost v0.2.1/go.mod h1:zDMheBLvNzu7Q6o9TBvPqiZToJcSuCLXjAXxBslSky4=
|
||||
github.com/hashicorp/yamux v0.1.2 h1:XtB8kyFOyHXYVFnwT5C3+Bdo8gArse7j2AQ0DA0Uey8=
|
||||
github.com/hashicorp/yamux v0.1.2/go.mod h1:C+zze2n6e/7wshOZep2A70/aQU6QBRWJO/G6FT1wIns=
|
||||
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 h1:BQSFePA1RWJOlocH6Fxy8MmwDt+yVQYULKfN0RoTN8A=
|
||||
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99/go.mod h1:1lJo3i6rXxKeerYnT8Nvf0QmHCRC1n8sfWVwXF2Frvo=
|
||||
github.com/jhump/protoreflect v1.17.0 h1:qOEr613fac2lOuTgWN4tPAtLL7fUSbuJL5X5XumQh94=
|
||||
github.com/jhump/protoreflect v1.17.0/go.mod h1:h9+vUUL38jiBzck8ck+6G/aeMX8Z4QUY/NiJPwPNi+8=
|
||||
github.com/kevinburke/ssh_config v1.2.0 h1:x584FjTGwHzMwvHx18PXxbBVzfnxogHaAReU4gf13a4=
|
||||
github.com/kevinburke/ssh_config v1.2.0/go.mod h1:CT57kijsi8u/K/BOFA39wgDQJ9CxiF4nAY/ojJ6r6mM=
|
||||
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
|
||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
||||
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
|
||||
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||
github.com/mattn/go-colorable v0.1.9/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
|
||||
github.com/mattn/go-colorable v0.1.12/go.mod h1:u5H1YNBxpqRaxsYJYSkiCWKzEfiAb1Gb520KVy5xxl4=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
|
||||
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
|
||||
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/mitchellh/copystructure v1.2.0 h1:vpKXTN4ewci03Vljg/q9QvCGUDttBOGBIa15WveJJGw=
|
||||
github.com/mitchellh/copystructure v1.2.0/go.mod h1:qLl+cE2AmVv+CoeAwDPye/v+N2HKCj9FbZEVFJRxO9s=
|
||||
github.com/mitchellh/go-testing-interface v1.14.1 h1:jrgshOhYAUVNMAJiKbEu7EqAwgJJ2JqpQmpLJOu07cU=
|
||||
github.com/mitchellh/go-testing-interface v1.14.1/go.mod h1:gfgS7OtZj6MA4U1UrDRp04twqAjfvlZyCfX3sDjEym8=
|
||||
github.com/mitchellh/go-wordwrap v1.0.1 h1:TLuKupo69TCn6TQSyGxwI1EblZZEsQ0vMlAFQflz0v0=
|
||||
github.com/mitchellh/go-wordwrap v1.0.1/go.mod h1:R62XHJLzvMFRBbcrT7m7WgmE1eOyTSsCt+hzestvNj0=
|
||||
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
|
||||
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
|
||||
github.com/mitchellh/reflectwalk v1.0.2 h1:G2LzWKi524PWgd3mLHV8Y5k7s6XUvT0Gef6zxSIeXaQ=
|
||||
github.com/mitchellh/reflectwalk v1.0.2/go.mod h1:mSTlrgnPZtwu0c4WaC2kGObEpuNDbx0jmZXqmk4esnw=
|
||||
github.com/oklog/run v1.2.0 h1:O8x3yXwah4A73hJdlrwo/2X6J62gE5qTMusH0dvz60E=
|
||||
github.com/oklog/run v1.2.0/go.mod h1:mgDbKRSwPhJfesJ4PntqFUbKQRZ50NgmZTSPlFA0YFk=
|
||||
github.com/pjbgf/sha1cd v0.3.2 h1:a9wb0bp1oC2TGwStyn0Umc/IGKQnEgF0vVaZ8QF8eo4=
|
||||
github.com/pjbgf/sha1cd v0.3.2/go.mod h1:zQWigSxVmsHEZow5qaLtPYxpcKMMQpa09ixqBxuCS6A=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
|
||||
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
|
||||
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
|
||||
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3 h1:n661drycOFuPLCN3Uc8sB6B/s6Z4t2xvBgU1htSHuq8=
|
||||
github.com/sergi/go-diff v1.3.2-0.20230802210424-5b0b94c5c0d3/go.mod h1:A0bzQcvG0E7Rwjx0REVgAGH58e96+X0MeOfepqsbeW4=
|
||||
github.com/skeema/knownhosts v1.3.1 h1:X2osQ+RAjK76shCbvhHHHVl3ZlgDm8apHEHFqRjnBY8=
|
||||
github.com/skeema/knownhosts v1.3.1/go.mod h1:r7KTdC8l4uxWRyK2TpQZ/1o5HaSzh06ePQNxPwTcfiY=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.7.2/go.mod h1:R6va5+xMeoiuVRoj+gSkQ7d3FALtqAAGI1FQKckRals=
|
||||
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
|
||||
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/vmihailenco/msgpack v3.3.3+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible h1:dSLoQfGFAo3F6OoNhwUmLwVgaUXK79GlxNBwueZn0xI=
|
||||
github.com/vmihailenco/msgpack v4.0.4+incompatible/go.mod h1:fy3FlTQTDXWkZ7Bh6AcGMlsjHatGryHQYUTf1ShIgkk=
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1 h1:cQriyiUvjTwOHg8QZaPihLWeRAAVoCpE00IUPn0Bjt8=
|
||||
github.com/vmihailenco/msgpack/v5 v5.4.1/go.mod h1:GaZTsDaehaPpQVyxrf5mtQlH+pc21PIudVV/E3rRQok=
|
||||
github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g=
|
||||
github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds=
|
||||
github.com/xanzy/ssh-agent v0.3.3 h1:+/15pJfg/RsTxqYcX6fHqOXZwwMP+2VyYWJeWM2qQFM=
|
||||
github.com/xanzy/ssh-agent v0.3.3/go.mod h1:6dzNDKs0J9rVPHPhaGCukekBHKqfl+L3KghI1Bc68Uw=
|
||||
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
|
||||
github.com/zclconf/go-cty v1.18.1 h1:yEGE8M4iIZlyKQURZNb2SnEyZlZHUcBCnx6KF81KuwM=
|
||||
github.com/zclconf/go-cty v1.18.1/go.mod h1:qpnV6EDNgC1sns/AleL1fvatHw72j+S+nS+MJ+T2CSg=
|
||||
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940 h1:4r45xpDWB6ZMSMNJFMOjqrGHynW3DIBuR2H9j0ug+Mo=
|
||||
github.com/zclconf/go-cty-debug v0.0.0-20240509010212-0d6042c53940/go.mod h1:CmBdvvj3nqzfzJ6nTCIwDTPZ56aVGvDrmztiO5g3qrM=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
|
||||
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
|
||||
go.opentelemetry.io/otel v1.39.0 h1:8yPrr/S0ND9QEfTfdP9V+SiwT4E0G7Y5MO7p85nis48=
|
||||
go.opentelemetry.io/otel v1.39.0/go.mod h1:kLlFTywNWrFyEdH0oj2xK0bFYZtHRYUdv1NklR/tgc8=
|
||||
go.opentelemetry.io/otel/metric v1.39.0 h1:d1UzonvEZriVfpNKEVmHXbdf909uGTOQjA0HF0Ls5Q0=
|
||||
go.opentelemetry.io/otel/metric v1.39.0/go.mod h1:jrZSWL33sD7bBxg1xjrqyDjnuzTUB0x1nBERXd7Ftcs=
|
||||
go.opentelemetry.io/otel/sdk v1.39.0 h1:nMLYcjVsvdui1B/4FRkwjzoRVsMK8uL/cj0OyhKzt18=
|
||||
go.opentelemetry.io/otel/sdk v1.39.0/go.mod h1:vDojkC4/jsTJsE+kh+LXYQlbL8CgrEcwmt1ENZszdJE=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.39.0 h1:cXMVVFVgsIf2YL6QkRF4Urbr/aMInf+2WKg+sEJTtB8=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.39.0/go.mod h1:xq9HEVH7qeX69/JnwEfp6fVq5wosJsY1mt4lLfYdVew=
|
||||
go.opentelemetry.io/otel/trace v1.39.0 h1:2d2vfpEDmCJ5zVYz7ijaJdOF59xLomrvj7bjt6/qCJI=
|
||||
go.opentelemetry.io/otel/trace v1.39.0/go.mod h1:88w4/PnZSazkGzz/w84VHpQafiU4EtqqlVdxWy+rNOA=
|
||||
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
|
||||
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
|
||||
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
|
||||
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
|
||||
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
|
||||
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
|
||||
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
|
||||
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
|
||||
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
|
||||
golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg=
|
||||
golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
|
||||
golang.org/x/net v0.52.0 h1:He/TN1l0e4mmR3QqHMT2Xab3Aj3L9qjbhRm78/6jrW0=
|
||||
golang.org/x/net v0.52.0/go.mod h1:R1MAz7uMZxVMualyPXb+VaqGSa3LIaUqk0eEt3w36Sw=
|
||||
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
|
||||
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
|
||||
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
|
||||
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
|
||||
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
|
||||
golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20210630005230-0f9fa26af87c/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20210927094055-39ccf1dd6fa6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220503163025-988cb79eb6c6/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
|
||||
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
|
||||
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
|
||||
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
|
||||
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
|
||||
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
|
||||
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
|
||||
golang.org/x/text v0.3.8/go.mod h1:E6s5w1FMmriuDzIBO73fBruAKo1PCIq6d2Q6DHfQ8WQ=
|
||||
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
|
||||
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
|
||||
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
|
||||
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
|
||||
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=
|
||||
golang.org/x/tools v0.43.0 h1:12BdW9CeB3Z+J/I/wj34VMl8X+fEXBxVR90JeMX5E7s=
|
||||
golang.org/x/tools v0.43.0/go.mod h1:uHkMso649BX2cZK6+RpuIPXS3ho2hZo4FVwfoy1vIk0=
|
||||
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
gonum.org/v1/gonum v0.16.0 h1:5+ul4Swaf3ESvrOnidPp4GZbzf0mxVQpDCYUQE7OJfk=
|
||||
gonum.org/v1/gonum v0.16.0/go.mod h1:fef3am4MQ93R2HHpKnLk4/Tbh/s0+wqD5nfa6Pnwy4E=
|
||||
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
|
||||
google.golang.org/appengine v1.6.8 h1:IhEN5q69dyKagZPYMSdIjS2HqprW324FRQZJcGqPAsM=
|
||||
google.golang.org/appengine v1.6.8/go.mod h1:1jJ3jBArFh5pcgW8gCtRJnepW8FzD1V44FJffLiz/Ds=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217 h1:gRkg/vSppuSQoDjxyiGfN4Upv/h/DQmIR10ZU8dh4Ww=
|
||||
google.golang.org/genproto/googleapis/rpc v0.0.0-20251202230838-ff82c1b0f217/go.mod h1:7i2o+ce6H/6BluujYR+kqX3GKH+dChPTQU19wjRPiGk=
|
||||
google.golang.org/grpc v1.79.3 h1:sybAEdRIEtvcD68Gx7dmnwjZKlyfuc61Dyo9pGXXkKE=
|
||||
google.golang.org/grpc v1.79.3/go.mod h1:KmT0Kjez+0dde/v2j9vzwoAScgEPx/Bw1CYChhHLrHQ=
|
||||
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
|
||||
google.golang.org/protobuf v1.26.0/go.mod h1:9q0QmTI4eRPtz6boOQmLYwt+qCgq0jsYwAQnmE0givc=
|
||||
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
|
||||
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
|
||||
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
|
||||
gopkg.in/warnings.v0 v0.1.2 h1:wFXVbFY8DY5/xOe1ECiWdKCzZlxgshcYVNkBHstARME=
|
||||
gopkg.in/warnings.v0 v0.1.2/go.mod h1:jksf8JmL6Qr/oQM2OXTHunEvvTAsrWBLb6OOjuVWRNI=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
@@ -0,0 +1,438 @@
|
||||
// Package client talks to llmgw's admin API over HTTP with the admin token.
|
||||
//
|
||||
// Its own types rather than the gateway's: the provider is a separate module
|
||||
// so the gateway does not carry Terraform's dependencies, and the gateway's
|
||||
// packages are internal to it. What keeps the two in step is the OpenAPI
|
||||
// document the gateway generates, docs/admin-api.openapi.json, which a test
|
||||
// here checks every endpoint and field this client uses against.
|
||||
package client
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"slices"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Endpoints are the admin API's endpoints this client calls, as the OpenAPI
|
||||
// document names them. A request for any other is refused before it is
|
||||
// sent, so this list is the whole of what the contract test has to check.
|
||||
var Endpoints = []string{
|
||||
"GET /api/v1/models",
|
||||
"POST /api/v1/models",
|
||||
"GET /api/v1/models/{name}",
|
||||
"DELETE /api/v1/models/{name}",
|
||||
"POST /api/v1/models/{name}/scale",
|
||||
"POST /api/v1/models/{name}/class",
|
||||
"POST /api/v1/models/{name}/owner",
|
||||
"POST /api/v1/models/{name}/adapters",
|
||||
"GET /api/v1/nodes",
|
||||
"GET /api/v1/quotas",
|
||||
"PUT /api/v1/quotas",
|
||||
"DELETE /api/v1/quotas/{scope}",
|
||||
"GET /api/v1/guard",
|
||||
"POST /api/v1/guard/attachments",
|
||||
"DELETE /api/v1/guard/attachments",
|
||||
}
|
||||
|
||||
// Client is a connection to one control plane's admin API.
|
||||
type Client struct {
|
||||
base *url.URL
|
||||
token string
|
||||
actor string
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
// New builds a client for the admin listener at endpoint, such as
|
||||
// http://127.0.0.1:9090. Actor is who changes are recorded as made by.
|
||||
func New(endpoint, token, actor string) (*Client, error) {
|
||||
u, err := url.Parse(strings.TrimRight(endpoint, "/"))
|
||||
if err != nil || (u.Scheme != "http" && u.Scheme != "https") || u.Host == "" {
|
||||
return nil, fmt.Errorf("endpoint %q is not an http or https address", endpoint)
|
||||
}
|
||||
if token == "" {
|
||||
return nil, errors.New("no admin token: set token, token_file or LLMGW_ADMIN_TOKEN")
|
||||
}
|
||||
return &Client{base: u, token: token, actor: actor, http: &http.Client{Timeout: 2 * time.Minute}}, nil
|
||||
}
|
||||
|
||||
// Error is the API refusing a request, with what it said.
|
||||
type Error struct {
|
||||
Method, Path string
|
||||
Status int
|
||||
Message string
|
||||
}
|
||||
|
||||
func (e *Error) Error() string {
|
||||
return fmt.Sprintf("%s %s: %d %s", e.Method, e.Path, e.Status, e.Message)
|
||||
}
|
||||
|
||||
// IsNotFound reports whether err is the API saying there is no such thing.
|
||||
func IsNotFound(err error) bool {
|
||||
var e *Error
|
||||
return errors.As(err, &e) && e.Status == http.StatusNotFound
|
||||
}
|
||||
|
||||
// do sends one request to an endpoint named as in Endpoints, with args put
|
||||
// in its path parameters in order.
|
||||
func (c *Client) do(ctx context.Context, endpoint string, args []string, in, out any) error {
|
||||
if !slices.Contains(Endpoints, endpoint) {
|
||||
return fmt.Errorf("the client does not call %s: add it to Endpoints", endpoint)
|
||||
}
|
||||
method, tmpl, _ := strings.Cut(endpoint, " ")
|
||||
p := tmpl
|
||||
for _, a := range args {
|
||||
i, j := strings.Index(p, "{"), strings.Index(p, "}")
|
||||
if i < 0 || j < i {
|
||||
return fmt.Errorf("%s takes fewer parameters than given", endpoint)
|
||||
}
|
||||
p = p[:i] + url.PathEscape(a) + p[j+1:]
|
||||
}
|
||||
if strings.Contains(p, "{") {
|
||||
return fmt.Errorf("%s takes more parameters than given", endpoint)
|
||||
}
|
||||
|
||||
var body io.Reader
|
||||
if in != nil {
|
||||
b, err := json.Marshal(in)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body = bytes.NewReader(b)
|
||||
}
|
||||
plain, err := url.PathUnescape(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
u := *c.base
|
||||
u.Path = strings.TrimRight(u.Path, "/") + plain
|
||||
// RawPath keeps an escaped slash in a name escaped: a model or a scope
|
||||
// called Acme/ml is one segment, not two.
|
||||
u.RawPath = strings.TrimRight(c.base.EscapedPath(), "/") + p
|
||||
req, err := http.NewRequestWithContext(ctx, method, u.String(), body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
req.Header.Set("Accept", "application/json")
|
||||
// Every change must say it is JSON, even one without a body: the admin
|
||||
// listener refuses any other, as a page's form post cannot set it.
|
||||
if method != http.MethodGet {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
if c.actor != "" {
|
||||
req.Header.Set("X-Llmgw-Actor", c.actor)
|
||||
}
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
b, err := io.ReadAll(io.LimitReader(resp.Body, 32<<20))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if resp.StatusCode >= 300 {
|
||||
var e struct {
|
||||
Error string `json:"error"`
|
||||
}
|
||||
msg := strings.TrimSpace(string(b))
|
||||
if json.Unmarshal(b, &e) == nil && e.Error != "" {
|
||||
msg = e.Error
|
||||
}
|
||||
return &Error{Method: method, Path: tmpl, Status: resp.StatusCode, Message: msg}
|
||||
}
|
||||
if out != nil {
|
||||
if err := json.Unmarshal(b, out); err != nil {
|
||||
return fmt.Errorf("%s %s: the answer is not what was expected: %w", method, tmpl, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- models ---
|
||||
|
||||
// Model is a model's definition, as the catalog holds it.
|
||||
type Model struct {
|
||||
Name string `json:"name,omitempty"`
|
||||
Aliases []string `json:"aliases,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Source string `json:"source,omitempty"`
|
||||
Profiles []Profile `json:"profiles,omitempty"`
|
||||
Scaling Scaling `json:"scaling,omitzero"`
|
||||
Class string `json:"class,omitempty"`
|
||||
Owner string `json:"owner,omitempty"`
|
||||
External *External `json:"external,omitempty"`
|
||||
Adapters []Adapter `json:"adapters,omitempty"`
|
||||
Fallbacks []string `json:"fallbacks,omitempty"`
|
||||
RoutingProfile string `json:"routing_profile,omitempty"`
|
||||
ContextLength int64 `json:"context_length,omitempty"`
|
||||
MaxOutput int64 `json:"max_output,omitempty"`
|
||||
Capabilities *Capabilities `json:"capabilities,omitempty"`
|
||||
}
|
||||
|
||||
// Profile is how a model runs on one kind of node.
|
||||
type Profile struct {
|
||||
Name string `json:"name,omitempty"`
|
||||
Engine string `json:"engine,omitempty"`
|
||||
GPUClass string `json:"gpu_class,omitempty"`
|
||||
TP int64 `json:"tp,omitempty"`
|
||||
PP int64 `json:"pp,omitempty"`
|
||||
GPUsPerNode int64 `json:"gpus_per_node,omitempty"`
|
||||
Nodes int64 `json:"nodes,omitempty"`
|
||||
Args []string `json:"args,omitempty"`
|
||||
Env map[string]string `json:"env,omitempty"`
|
||||
MinMemoryGB int64 `json:"min_memory_gb,omitempty"`
|
||||
CPUCores int64 `json:"cpu_cores,omitempty"`
|
||||
MinDiskGB int64 `json:"min_disk_gb,omitempty"`
|
||||
NodeSelector map[string]string `json:"node_selector,omitempty"`
|
||||
Weight float64 `json:"weight,omitempty"`
|
||||
Priority int64 `json:"priority,omitempty"`
|
||||
StartupTimeout time.Duration `json:"startup_timeout,omitempty"`
|
||||
}
|
||||
|
||||
// Scaling is how many replicas a model runs.
|
||||
type Scaling struct {
|
||||
Min int64 `json:"min,omitempty"`
|
||||
Max int64 `json:"max,omitempty"`
|
||||
IdleTimeout time.Duration `json:"idle_timeout,omitempty"`
|
||||
}
|
||||
|
||||
// External is a model served by a provider outside the cluster.
|
||||
type External struct {
|
||||
URL string `json:"url"`
|
||||
Model string `json:"model"`
|
||||
APIKeyFile string `json:"api_key_file,omitempty"`
|
||||
}
|
||||
|
||||
// Adapter is a LoRA adapter a model serves.
|
||||
type Adapter struct {
|
||||
Name string `json:"name"`
|
||||
Source string `json:"source"`
|
||||
Scale float64 `json:"scale,omitempty"`
|
||||
}
|
||||
|
||||
// Capabilities are what a model can do beyond chat.
|
||||
type Capabilities struct {
|
||||
Tools bool `json:"tools,omitempty"`
|
||||
Vision bool `json:"vision,omitempty"`
|
||||
Embeddings bool `json:"embeddings,omitempty"`
|
||||
}
|
||||
|
||||
// ModelView is a model as the listing shows it.
|
||||
type ModelView struct {
|
||||
Name string `json:"name"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Aliases []string `json:"aliases,omitempty"`
|
||||
Source string `json:"source"`
|
||||
Min int64 `json:"min_replicas"`
|
||||
Max int64 `json:"max_replicas"`
|
||||
Ready int64 `json:"ready_replicas"`
|
||||
Total int64 `json:"total_replicas"`
|
||||
Desired int64 `json:"desired_replicas"`
|
||||
Parked bool `json:"parked,omitempty"`
|
||||
Reason string `json:"reason,omitempty"`
|
||||
Profiles []string `json:"profiles"`
|
||||
Class string `json:"class"`
|
||||
Owner string `json:"owner,omitempty"`
|
||||
External string `json:"external,omitempty"`
|
||||
}
|
||||
|
||||
// ScaleRequest sets a model's replicas.
|
||||
type ScaleRequest struct {
|
||||
Min int64 `json:"min"`
|
||||
Max int64 `json:"max,omitempty"`
|
||||
IdleTimeout time.Duration `json:"idle_timeout,omitempty"`
|
||||
}
|
||||
|
||||
// ClassRequest sets a model's capacity class.
|
||||
type ClassRequest struct {
|
||||
Class string `json:"class"`
|
||||
}
|
||||
|
||||
// OwnerRequest sets a model's owner.
|
||||
type OwnerRequest struct {
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
// OwnerView is a model's owner, as scope:ID.
|
||||
type OwnerView struct {
|
||||
Model string `json:"model"`
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
// AdaptersRequest replaces a model's adapters.
|
||||
type AdaptersRequest struct {
|
||||
Adapters []Adapter `json:"adapters"`
|
||||
}
|
||||
|
||||
// Models lists the catalog.
|
||||
func (c *Client) Models(ctx context.Context) ([]ModelView, error) {
|
||||
var out []ModelView
|
||||
return out, c.do(ctx, "GET /api/v1/models", nil, nil, &out)
|
||||
}
|
||||
|
||||
// AddModel adds a model to the catalog.
|
||||
func (c *Client) AddModel(ctx context.Context, m Model) error {
|
||||
return c.do(ctx, "POST /api/v1/models", nil, m, nil)
|
||||
}
|
||||
|
||||
// Model reads one model's definition, as it is in force.
|
||||
func (c *Client) Model(ctx context.Context, name string) (Model, error) {
|
||||
var out Model
|
||||
return out, c.do(ctx, "GET /api/v1/models/{name}", []string{name}, nil, &out)
|
||||
}
|
||||
|
||||
// RemoveModel removes a model added at runtime.
|
||||
func (c *Client) RemoveModel(ctx context.Context, name string) error {
|
||||
return c.do(ctx, "DELETE /api/v1/models/{name}", []string{name}, nil, nil)
|
||||
}
|
||||
|
||||
// Scale sets how many replicas a model runs.
|
||||
func (c *Client) Scale(ctx context.Context, name string, s ScaleRequest) error {
|
||||
return c.do(ctx, "POST /api/v1/models/{name}/scale", []string{name}, s, nil)
|
||||
}
|
||||
|
||||
// SetClass sets a model's capacity class; empty goes back to its default.
|
||||
func (c *Client) SetClass(ctx context.Context, name, class string) error {
|
||||
return c.do(ctx, "POST /api/v1/models/{name}/class", []string{name}, ClassRequest{Class: class}, nil)
|
||||
}
|
||||
|
||||
// SetOwner sets a model's owner, by path or ID; empty makes it shared. It
|
||||
// answers with the owner as scope:ID.
|
||||
func (c *Client) SetOwner(ctx context.Context, name, owner string) (string, error) {
|
||||
var out OwnerView
|
||||
err := c.do(ctx, "POST /api/v1/models/{name}/owner", []string{name}, OwnerRequest{Owner: owner}, &out)
|
||||
return out.Owner, err
|
||||
}
|
||||
|
||||
// SetAdapters replaces a model's LoRA adapters.
|
||||
func (c *Client) SetAdapters(ctx context.Context, name string, adapters []Adapter) error {
|
||||
if adapters == nil {
|
||||
adapters = []Adapter{}
|
||||
}
|
||||
return c.do(ctx, "POST /api/v1/models/{name}/adapters", []string{name}, AdaptersRequest{Adapters: adapters}, nil)
|
||||
}
|
||||
|
||||
// --- nodes ---
|
||||
|
||||
// NodeView is a node as the listing shows it.
|
||||
type NodeView struct {
|
||||
ID string `json:"id"`
|
||||
Hostname string `json:"hostname,omitempty"`
|
||||
MgmtAddr string `json:"mgmt_addr,omitempty"`
|
||||
State string `json:"state"`
|
||||
Labels map[string]string `json:"labels,omitempty"`
|
||||
GPUTotal int64 `json:"gpu_total"`
|
||||
GPUFree int64 `json:"gpu_free"`
|
||||
MemoryGB int64 `json:"memory_gb,omitempty"`
|
||||
CPUCount int64 `json:"cpu_count,omitempty"`
|
||||
Engines []string `json:"engines,omitempty"`
|
||||
OS string `json:"os,omitempty"`
|
||||
Classes []string `json:"gpu_classes,omitempty"`
|
||||
Fabrics []string `json:"fabrics,omitempty"`
|
||||
AgentVersion string `json:"agent_version,omitempty"`
|
||||
Replicas int64 `json:"replicas"`
|
||||
}
|
||||
|
||||
// Nodes lists the fleet.
|
||||
func (c *Client) Nodes(ctx context.Context) ([]NodeView, error) {
|
||||
var out []NodeView
|
||||
return out, c.do(ctx, "GET /api/v1/nodes", nil, nil, &out)
|
||||
}
|
||||
|
||||
// --- quotas ---
|
||||
|
||||
// Resources is an amount of the cluster; nil is not governed.
|
||||
type Resources struct {
|
||||
GPUs *int64 `json:"gpus,omitempty"`
|
||||
CPUCores *int64 `json:"cpu_cores,omitempty"`
|
||||
MemoryGB *int64 `json:"memory_gb,omitempty"`
|
||||
}
|
||||
|
||||
// QuotaView is one scope's quota.
|
||||
type QuotaView struct {
|
||||
Scope string `json:"scope"`
|
||||
Name string `json:"name"`
|
||||
Guarantee Resources `json:"guarantee"`
|
||||
Limit Resources `json:"limit"`
|
||||
}
|
||||
|
||||
// QuotaRequest sets a scope's quota, the scope by path or ID.
|
||||
type QuotaRequest struct {
|
||||
Scope string `json:"scope"`
|
||||
Guarantee Resources `json:"guarantee"`
|
||||
Limit Resources `json:"limit"`
|
||||
}
|
||||
|
||||
// QuotaSet is the scope a quota was set on, as scope:ID.
|
||||
type QuotaSet struct {
|
||||
Scope string `json:"scope"`
|
||||
}
|
||||
|
||||
// Quotas lists every capacity quota.
|
||||
func (c *Client) Quotas(ctx context.Context) ([]QuotaView, error) {
|
||||
var out []QuotaView
|
||||
return out, c.do(ctx, "GET /api/v1/quotas", nil, nil, &out)
|
||||
}
|
||||
|
||||
// SetQuota sets a scope's quota whole, and answers with the scope as
|
||||
// scope:ID.
|
||||
func (c *Client) SetQuota(ctx context.Context, q QuotaRequest) (string, error) {
|
||||
var out QuotaSet
|
||||
err := c.do(ctx, "PUT /api/v1/quotas", nil, q, &out)
|
||||
return out.Scope, err
|
||||
}
|
||||
|
||||
// ClearQuota removes a scope's quota.
|
||||
func (c *Client) ClearQuota(ctx context.Context, scope string) error {
|
||||
return c.do(ctx, "DELETE /api/v1/quotas/{scope}", []string{scope}, nil, nil)
|
||||
}
|
||||
|
||||
// --- guardrails ---
|
||||
|
||||
// Attachment is a policy attached to a scope. The API writes its fields by
|
||||
// their Go names.
|
||||
type Attachment struct {
|
||||
Scope string `json:"Scope"`
|
||||
ScopeID string `json:"ScopeID"`
|
||||
Policy string `json:"Policy"`
|
||||
}
|
||||
|
||||
// GuardView is the policy library and its attachments.
|
||||
type GuardView struct {
|
||||
Policies []string `json:"policies"`
|
||||
Default []string `json:"default"`
|
||||
Attachments []Attachment `json:"attachments"`
|
||||
}
|
||||
|
||||
// GuardAttachmentRequest names a policy and where it applies.
|
||||
type GuardAttachmentRequest struct {
|
||||
Target string `json:"target"`
|
||||
Policy string `json:"policy"`
|
||||
}
|
||||
|
||||
// Guard reads the policies and where they are attached.
|
||||
func (c *Client) Guard(ctx context.Context) (GuardView, error) {
|
||||
var out GuardView
|
||||
return out, c.do(ctx, "GET /api/v1/guard", nil, nil, &out)
|
||||
}
|
||||
|
||||
// Attach attaches a policy, answering with where it went by ID.
|
||||
func (c *Client) Attach(ctx context.Context, target, policy string) (Attachment, error) {
|
||||
var out Attachment
|
||||
err := c.do(ctx, "POST /api/v1/guard/attachments", nil, GuardAttachmentRequest{Target: target, Policy: policy}, &out)
|
||||
return out, err
|
||||
}
|
||||
|
||||
// Detach takes a policy off.
|
||||
func (c *Client) Detach(ctx context.Context, target, policy string) error {
|
||||
return c.do(ctx, "DELETE /api/v1/guard/attachments", nil, GuardAttachmentRequest{Target: target, Policy: policy}, nil)
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
package client_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/fakeapi"
|
||||
)
|
||||
|
||||
func serve(t *testing.T) (*fakeapi.API, *client.Client) {
|
||||
t.Helper()
|
||||
api := fakeapi.New("secret", client.NodeView{ID: "gpu01", State: "ready", GPUTotal: 8, GPUFree: 6})
|
||||
srv := httptest.NewServer(api)
|
||||
t.Cleanup(srv.Close)
|
||||
c, err := client.New(srv.URL, "secret", "terraform")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return api, c
|
||||
}
|
||||
|
||||
func TestAModelIsAddedReadScaledAndRemoved(t *testing.T) {
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
err := c.AddModel(ctx, client.Model{Name: "qwen", Source: "/m/qwen.gguf",
|
||||
Profiles: []client.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 4}},
|
||||
Scaling: client.Scaling{Min: 1}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m, err := c.Model(ctx, "qwen")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// As the catalog filled it in.
|
||||
if m.Scaling.Max != 1 || m.Profiles[0].TP != 1 || m.Profiles[0].StartupTimeout != 20*time.Minute {
|
||||
t.Errorf("read back %+v", m)
|
||||
}
|
||||
if err := c.Scale(ctx, "qwen", client.ScaleRequest{Min: 0, Max: 2, IdleTimeout: 30 * time.Minute}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if m, _ := api.Model("qwen"); m.Scaling != (client.Scaling{Min: 0, Max: 2, IdleTimeout: 30 * time.Minute}) {
|
||||
t.Errorf("scaled to %+v", m.Scaling)
|
||||
}
|
||||
if err := c.RemoveModel(ctx, "qwen"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := c.Model(ctx, "qwen"); !client.IsNotFound(err) {
|
||||
t.Errorf("a removed model read as %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryChangeSaysItIsJSONAndWhoMadeIt(t *testing.T) {
|
||||
// The admin listener refuses a change without application/json, a DELETE
|
||||
// with no body included; and the audit log names whoever the change says
|
||||
// it is from.
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
if err := c.AddModel(ctx, client.Model{Name: "m", Source: "/m"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := c.RemoveModel(ctx, "m"); err != nil {
|
||||
t.Fatalf("a DELETE with no body: %v", err)
|
||||
}
|
||||
if got := api.Actors(); !slices.Equal(got, []string{"terraform", "terraform"}) {
|
||||
t.Errorf("actors %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestARefusalCarriesWhatTheAPISaid(t *testing.T) {
|
||||
_, c := serve(t)
|
||||
ctx := context.Background()
|
||||
_, err := c.SetQuota(ctx, client.QuotaRequest{Scope: "team:Nobody/x", Limit: client.Resources{GPUs: ptr(1)}})
|
||||
var e *client.Error
|
||||
if err == nil || !strings.Contains(err.Error(), `there is no team "Nobody/x"`) {
|
||||
t.Fatalf("got %v", err)
|
||||
}
|
||||
if !errors.As(err, &e) || e.Status != http.StatusBadRequest {
|
||||
t.Errorf("not a 400: %#v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWrongTokenIsRefused(t *testing.T) {
|
||||
api := fakeapi.New("secret")
|
||||
srv := httptest.NewServer(api)
|
||||
defer srv.Close()
|
||||
c, err := client.New(srv.URL, "wrong", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var e *client.Error
|
||||
if _, err := c.Models(context.Background()); !errors.As(err, &e) || e.Status != http.StatusUnauthorized {
|
||||
t.Errorf("got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestANameWithASlashIsOneSegment(t *testing.T) {
|
||||
// A quota's scope is named by path, Acme/ml, and must not become two
|
||||
// segments of the URL on its way to DELETE /api/v1/quotas/{scope}.
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
id, err := c.SetQuota(ctx, client.QuotaRequest{Scope: "team:Acme/ml", Guarantee: client.Resources{GPUs: ptr(2)}})
|
||||
if err != nil || id != "team:t1" {
|
||||
t.Fatalf("set %q, %v", id, err)
|
||||
}
|
||||
if err := c.ClearQuota(ctx, "team:Acme/ml"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, ok := api.Quota("team:t1"); ok {
|
||||
t.Error("the quota is still there")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPolicyIsAttachedByPathAndDetachedByID(t *testing.T) {
|
||||
api, c := serve(t)
|
||||
ctx := context.Background()
|
||||
a, err := c.Attach(ctx, "team:Acme/ml", "finance")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if a != (client.Attachment{Scope: "team", ScopeID: "t1", Policy: "finance"}) {
|
||||
t.Errorf("attached %+v", a)
|
||||
}
|
||||
g, err := c.Guard(ctx)
|
||||
if err != nil || len(g.Attachments) != 1 || g.Attachments[0] != a {
|
||||
t.Fatalf("guard %+v, %v", g, err)
|
||||
}
|
||||
if err := c.Detach(ctx, "team:t1", "finance"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(api.Attachments()) != 0 {
|
||||
t.Error("still attached")
|
||||
}
|
||||
if err := c.Detach(ctx, "team:t1", "finance"); !client.IsNotFound(err) {
|
||||
t.Errorf("detaching twice: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheNodesAreListed(t *testing.T) {
|
||||
_, c := serve(t)
|
||||
ns, err := c.Nodes(context.Background())
|
||||
if err != nil || len(ns) != 1 || ns[0].ID != "gpu01" || ns[0].GPUFree != 6 {
|
||||
t.Errorf("nodes %+v, %v", ns, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnEndpointNotListedIsNotCalled(t *testing.T) {
|
||||
// Endpoints is what the contract test checks against the API's document;
|
||||
// a call outside it would be one nothing checks.
|
||||
if slices.Contains(client.Endpoints, "POST /api/v1/restart") {
|
||||
t.Skip("restart is listed now")
|
||||
}
|
||||
if !strings.Contains(client.CallUnlisted(), "add it to Endpoints") {
|
||||
t.Error("an unlisted endpoint was called")
|
||||
}
|
||||
}
|
||||
|
||||
func TestABadEndpointOrNoTokenIsRefusedUpFront(t *testing.T) {
|
||||
if _, err := client.New("127.0.0.1:9090", "t", ""); err == nil {
|
||||
t.Error("an address without a scheme was taken")
|
||||
}
|
||||
if _, err := client.New("http://127.0.0.1:9090", "", ""); err == nil {
|
||||
t.Error("no token was taken")
|
||||
}
|
||||
}
|
||||
|
||||
func ptr(n int64) *int64 { return &n }
|
||||
@@ -0,0 +1,201 @@
|
||||
package client
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// The gateway's OpenAPI document is generated from its route table and the
|
||||
// Go types its handlers use, and a test there keeps it current. Checking
|
||||
// this client against it is what keeps the two modules in step without the
|
||||
// provider importing the gateway.
|
||||
|
||||
type document struct {
|
||||
Paths map[string]map[string]operation `json:"paths"`
|
||||
Components struct {
|
||||
Schemas map[string]schemaDoc `json:"schemas"`
|
||||
} `json:"components"`
|
||||
}
|
||||
|
||||
type operation struct {
|
||||
RequestBody *struct {
|
||||
Content map[string]struct {
|
||||
Schema schemaDoc `json:"schema"`
|
||||
} `json:"content"`
|
||||
} `json:"requestBody"`
|
||||
Responses map[string]struct {
|
||||
Content map[string]struct {
|
||||
Schema schemaDoc `json:"schema"`
|
||||
} `json:"content"`
|
||||
} `json:"responses"`
|
||||
}
|
||||
|
||||
type schemaDoc struct {
|
||||
Ref string `json:"$ref"`
|
||||
Type string `json:"type"`
|
||||
Items *schemaDoc `json:"items"`
|
||||
Properties map[string]schemaDoc `json:"properties"`
|
||||
}
|
||||
|
||||
func (s schemaDoc) name() string { return strings.TrimPrefix(s.Ref, "#/components/schemas/") }
|
||||
|
||||
func load(t *testing.T) document {
|
||||
t.Helper()
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("cannot locate this test")
|
||||
}
|
||||
p := filepath.Join(filepath.Dir(file), "..", "..", "..", "docs", "admin-api.openapi.json")
|
||||
b, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
t.Fatalf("the gateway's OpenAPI document: %v", err)
|
||||
}
|
||||
var d document
|
||||
if err := json.Unmarshal(b, &d); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
func (d document) op(t *testing.T, endpoint string) (operation, bool) {
|
||||
method, p, _ := strings.Cut(endpoint, " ")
|
||||
op, ok := d.Paths[p][strings.ToLower(method)]
|
||||
return op, ok
|
||||
}
|
||||
|
||||
func TestEveryEndpointTheClientCallsIsInTheDocument(t *testing.T) {
|
||||
d := load(t)
|
||||
for _, e := range Endpoints {
|
||||
if _, ok := d.op(t, e); !ok {
|
||||
t.Errorf("the client calls %s, which the admin API does not serve", e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// bodies is each endpoint's request and answer: the client's type and the
|
||||
// schema the document names for it. Empty is none, or one the client
|
||||
// ignores.
|
||||
var bodies = []struct {
|
||||
endpoint string
|
||||
request any
|
||||
requestSchema string
|
||||
answer any
|
||||
answerSchema string
|
||||
}{
|
||||
{"GET /api/v1/models", nil, "", ModelView{}, "ModelView"},
|
||||
{"POST /api/v1/models", Model{}, "catalog.Model", nil, ""},
|
||||
{"GET /api/v1/models/{name}", nil, "", Model{}, "catalog.Model"},
|
||||
{"POST /api/v1/models/{name}/scale", ScaleRequest{}, "ScaleRequest", nil, ""},
|
||||
{"POST /api/v1/models/{name}/class", ClassRequest{}, "ClassRequest", nil, ""},
|
||||
{"POST /api/v1/models/{name}/owner", OwnerRequest{}, "OwnerRequest", OwnerView{}, "OwnerView"},
|
||||
{"POST /api/v1/models/{name}/adapters", AdaptersRequest{}, "AdaptersRequest", nil, ""},
|
||||
{"GET /api/v1/nodes", nil, "", NodeView{}, "NodeView"},
|
||||
{"GET /api/v1/quotas", nil, "", QuotaView{}, "QuotaView"},
|
||||
{"PUT /api/v1/quotas", QuotaRequest{}, "QuotaRequest", QuotaSet{}, "QuotaSet"},
|
||||
{"GET /api/v1/guard", nil, "", GuardView{}, "GuardView"},
|
||||
{"POST /api/v1/guard/attachments", GuardAttachmentRequest{}, "GuardAttachmentRequest", Attachment{}, "guard.Attachment"},
|
||||
{"DELETE /api/v1/guard/attachments", GuardAttachmentRequest{}, "GuardAttachmentRequest", nil, ""},
|
||||
}
|
||||
|
||||
// nested are the client's types inside those bodies, and their schemas.
|
||||
var nested = map[reflect.Type]string{
|
||||
reflect.TypeOf(Profile{}): "catalog.Profile",
|
||||
reflect.TypeOf(Scaling{}): "catalog.Scaling",
|
||||
reflect.TypeOf(External{}): "catalog.External",
|
||||
reflect.TypeOf(Adapter{}): "catalog.Adapter",
|
||||
reflect.TypeOf(Capabilities{}): "catalog.Capabilities",
|
||||
reflect.TypeOf(Resources{}): "store.Resources",
|
||||
reflect.TypeOf(Attachment{}): "GuardAttachmentView",
|
||||
}
|
||||
|
||||
func TestEveryBodyTheClientSendsAndReadsMatchesTheDocument(t *testing.T) {
|
||||
d := load(t)
|
||||
for _, b := range bodies {
|
||||
op, ok := d.op(t, b.endpoint)
|
||||
if !ok {
|
||||
t.Errorf("%s is not in the document", b.endpoint)
|
||||
continue
|
||||
}
|
||||
if b.request != nil {
|
||||
if op.RequestBody == nil {
|
||||
t.Errorf("%s takes no body in the document", b.endpoint)
|
||||
} else if got := op.RequestBody.Content["application/json"].Schema.name(); got != b.requestSchema {
|
||||
t.Errorf("%s takes %s, not %s", b.endpoint, got, b.requestSchema)
|
||||
}
|
||||
checkType(t, d, reflect.TypeOf(b.request), b.requestSchema)
|
||||
}
|
||||
if b.answer != nil {
|
||||
var s schemaDoc
|
||||
for code, r := range op.Responses {
|
||||
if strings.HasPrefix(code, "2") {
|
||||
s = r.Content["application/json"].Schema
|
||||
}
|
||||
}
|
||||
if s.Type == "array" && s.Items != nil {
|
||||
s = *s.Items
|
||||
}
|
||||
if s.name() != b.answerSchema {
|
||||
t.Errorf("%s answers with %s, not %s", b.endpoint, s.name(), b.answerSchema)
|
||||
}
|
||||
checkType(t, d, reflect.TypeOf(b.answer), b.answerSchema)
|
||||
}
|
||||
}
|
||||
for typ, name := range nested {
|
||||
checkType(t, d, typ, name)
|
||||
}
|
||||
}
|
||||
|
||||
// checkType fails for each of a struct's JSON keys the schema does not have,
|
||||
// or has as another kind of value.
|
||||
func checkType(t *testing.T, d document, typ reflect.Type, schema string) {
|
||||
t.Helper()
|
||||
s, ok := d.Components.Schemas[schema]
|
||||
if !ok {
|
||||
t.Errorf("no schema %s in the document", schema)
|
||||
return
|
||||
}
|
||||
for i := range typ.NumField() {
|
||||
f := typ.Field(i)
|
||||
name, _, _ := strings.Cut(f.Tag.Get("json"), ",")
|
||||
if name == "" {
|
||||
name = f.Name
|
||||
}
|
||||
p, ok := s.Properties[name]
|
||||
if !ok {
|
||||
t.Errorf("%s.%s is %q, which %s does not have", typ.Name(), f.Name, name, schema)
|
||||
continue
|
||||
}
|
||||
if want := kindOf(f.Type); want != "" && p.Type != "" && p.Type != want {
|
||||
t.Errorf("%s.%s is a %s here and a %s in %s", typ.Name(), f.Name, want, p.Type, schema)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func kindOf(t reflect.Type) string {
|
||||
if t == reflect.TypeOf(time.Duration(0)) {
|
||||
return "integer"
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Pointer:
|
||||
return kindOf(t.Elem())
|
||||
case reflect.String:
|
||||
return "string"
|
||||
case reflect.Bool:
|
||||
return "boolean"
|
||||
case reflect.Int, reflect.Int64:
|
||||
return "integer"
|
||||
case reflect.Float64:
|
||||
return "number"
|
||||
case reflect.Slice:
|
||||
return "array"
|
||||
case reflect.Map:
|
||||
return "object"
|
||||
}
|
||||
return ""
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
package client
|
||||
|
||||
import "context"
|
||||
|
||||
// CallUnlisted calls an endpoint that is not in Endpoints, and says what
|
||||
// came of it.
|
||||
func CallUnlisted() string {
|
||||
c := &Client{}
|
||||
err := c.do(context.Background(), "POST /api/v1/restart", nil, nil, nil)
|
||||
if err == nil {
|
||||
return ""
|
||||
}
|
||||
return err.Error()
|
||||
}
|
||||
@@ -0,0 +1,459 @@
|
||||
// Package fakeapi is an admin API in memory, for testing the client and the
|
||||
// provider without a cluster.
|
||||
//
|
||||
// It does what the real one does at the edges the provider depends on: it
|
||||
// demands the token, refuses a change not sent as JSON, fills in the defaults
|
||||
// the catalog fills in, resolves a scope named by path to its ID, and answers
|
||||
// with the same shapes and statuses. It is not the gateway; the OpenAPI
|
||||
// contract test is what ties the client to that.
|
||||
package fakeapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// API is the fake. Its fields may be read by a test while it serves.
|
||||
type API struct {
|
||||
Token string
|
||||
|
||||
mu sync.Mutex
|
||||
models map[string]client.Model
|
||||
order []string
|
||||
quotas map[string]client.QuotaView
|
||||
attachments []client.Attachment
|
||||
nodes []client.NodeView
|
||||
actors []string
|
||||
// scopes maps scope:ID to the path people know it by.
|
||||
scopes map[string]string
|
||||
}
|
||||
|
||||
// New builds a fake with a small tenancy tree — Acme, Acme/ml,
|
||||
// Acme/ml/training — two guard policies and the nodes given.
|
||||
func New(token string, nodes ...client.NodeView) *API {
|
||||
return &API{
|
||||
Token: token,
|
||||
models: map[string]client.Model{},
|
||||
quotas: map[string]client.QuotaView{},
|
||||
nodes: nodes,
|
||||
scopes: map[string]string{
|
||||
"org:o1": "Acme",
|
||||
"team:t1": "Acme / ml",
|
||||
"project:p1": "Acme/ml/training",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Actors are who each change said it was made by, in order.
|
||||
func (a *API) Actors() []string {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
return slices.Clone(a.actors)
|
||||
}
|
||||
|
||||
// Model is a model as the fake holds it, for a test to look at.
|
||||
func (a *API) Model(name string) (client.Model, bool) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m, ok := a.models[name]
|
||||
return m, ok
|
||||
}
|
||||
|
||||
// Quota is a quota as the fake holds it, by scope:ID.
|
||||
func (a *API) Quota(scope string) (client.QuotaView, bool) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
q, ok := a.quotas[scope]
|
||||
return q, ok
|
||||
}
|
||||
|
||||
// Attachments are the guard attachments as the fake holds them.
|
||||
func (a *API) Attachments() []client.Attachment {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
return slices.Clone(a.attachments)
|
||||
}
|
||||
|
||||
// ScaleBehindTerraform changes a model's scaling as someone at a terminal
|
||||
// would, for a test of drift.
|
||||
func (a *API) ScaleBehindTerraform(name string, minimum, maximum int64) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m := a.models[name]
|
||||
m.Scaling.Min, m.Scaling.Max = minimum, maximum
|
||||
a.models[name] = m
|
||||
}
|
||||
|
||||
// RemoveBehindTerraform removes a model as someone at a terminal would.
|
||||
func (a *API) RemoveBehindTerraform(name string) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
delete(a.models, name)
|
||||
a.order = slices.DeleteFunc(a.order, func(n string) bool { return n == name })
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(code)
|
||||
_ = json.NewEncoder(w).Encode(v)
|
||||
}
|
||||
|
||||
func writeError(w http.ResponseWriter, code int, msg string) {
|
||||
writeJSON(w, code, map[string]string{"error": msg})
|
||||
}
|
||||
|
||||
// ServeHTTP checks what the admin listener checks, then routes.
|
||||
func (a *API) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer "+a.Token {
|
||||
writeError(w, http.StatusUnauthorized, "the admin API needs its token")
|
||||
return
|
||||
}
|
||||
if r.Method != http.MethodGet {
|
||||
if ct, _, _ := strings.Cut(r.Header.Get("Content-Type"), ";"); ct != "application/json" {
|
||||
http.Error(w, "a change to the admin API must be sent as application/json", http.StatusUnsupportedMediaType)
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
a.actors = append(a.actors, r.Header.Get("X-Llmgw-Actor"))
|
||||
a.mu.Unlock()
|
||||
}
|
||||
a.mux().ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
func (a *API) mux() *http.ServeMux {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("GET /api/v1/models", a.listModels)
|
||||
mux.HandleFunc("POST /api/v1/models", a.addModel)
|
||||
mux.HandleFunc("GET /api/v1/models/{name}", a.getModel)
|
||||
mux.HandleFunc("DELETE /api/v1/models/{name}", a.removeModel)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/scale", a.scale)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/class", a.class)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/owner", a.owner)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/adapters", a.adapters)
|
||||
mux.HandleFunc("GET /api/v1/nodes", func(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
out := slices.Clone(a.nodes)
|
||||
if out == nil {
|
||||
out = []client.NodeView{}
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
})
|
||||
mux.HandleFunc("GET /api/v1/quotas", a.listQuotas)
|
||||
mux.HandleFunc("PUT /api/v1/quotas", a.putQuota)
|
||||
mux.HandleFunc("DELETE /api/v1/quotas/{scope}", a.deleteQuota)
|
||||
mux.HandleFunc("GET /api/v1/guard", a.guard)
|
||||
mux.HandleFunc("POST /api/v1/guard/attachments", a.attach(true))
|
||||
mux.HandleFunc("DELETE /api/v1/guard/attachments", a.attach(false))
|
||||
return mux
|
||||
}
|
||||
|
||||
// normalize fills in what the catalog fills in.
|
||||
func normalize(m client.Model) client.Model {
|
||||
if m.Scaling.Max == 0 && m.Scaling.Min > 0 {
|
||||
m.Scaling.Max = m.Scaling.Min
|
||||
}
|
||||
for i := range m.Profiles {
|
||||
p := &m.Profiles[i]
|
||||
if p.Engine == "" {
|
||||
p.Engine = "vllm"
|
||||
}
|
||||
if p.TP <= 0 {
|
||||
p.TP = 1
|
||||
}
|
||||
if p.Nodes <= 0 {
|
||||
p.Nodes = 1
|
||||
}
|
||||
if p.StartupTimeout <= 0 {
|
||||
p.StartupTimeout = 20 * time.Minute
|
||||
}
|
||||
if p.Weight <= 0 {
|
||||
p.Weight = 1
|
||||
}
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
func (a *API) listModels(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
out := []client.ModelView{}
|
||||
for _, n := range a.order {
|
||||
m := a.models[n]
|
||||
v := client.ModelView{Name: m.Name, Description: m.Description, Aliases: m.Aliases, Source: m.Source,
|
||||
Min: m.Scaling.Min, Max: m.Scaling.Max, Desired: m.Scaling.Min, Class: m.Class, Owner: m.Owner}
|
||||
if v.Class == "" {
|
||||
v.Class = "standard"
|
||||
}
|
||||
for _, p := range m.Profiles {
|
||||
v.Profiles = append(v.Profiles, p.Name)
|
||||
}
|
||||
if m.External != nil {
|
||||
v.External = m.External.URL + " · " + m.External.Model
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (a *API) addModel(w http.ResponseWriter, r *http.Request) {
|
||||
var m client.Model
|
||||
if err := json.NewDecoder(r.Body).Decode(&m); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the model definition is not valid JSON: "+err.Error())
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
if m.Name == "" || (m.Source == "" && m.External == nil) {
|
||||
writeError(w, http.StatusBadRequest, "a model needs a name and a source")
|
||||
return
|
||||
}
|
||||
if _, ok := a.models[m.Name]; ok {
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("model %q is already in the catalog", m.Name))
|
||||
return
|
||||
}
|
||||
a.models[m.Name] = normalize(m)
|
||||
a.order = append(a.order, m.Name)
|
||||
writeJSON(w, http.StatusOK, map[string]string{"added": m.Name})
|
||||
}
|
||||
|
||||
func (a *API) getModel(w http.ResponseWriter, r *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m, ok := a.models[r.PathValue("name")]
|
||||
if !ok {
|
||||
writeError(w, http.StatusNotFound, "no model named "+r.PathValue("name")+" is in the catalog")
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
}
|
||||
|
||||
func (a *API) removeModel(w http.ResponseWriter, r *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
name := r.PathValue("name")
|
||||
if _, ok := a.models[name]; !ok {
|
||||
// The real API says this with a 400: the model was not added at
|
||||
// runtime, or is not there at all.
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("model %q was not added at runtime", name))
|
||||
return
|
||||
}
|
||||
delete(a.models, name)
|
||||
a.order = slices.DeleteFunc(a.order, func(n string) bool { return n == name })
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": name})
|
||||
}
|
||||
|
||||
// change applies f to a model, answering 400 when there is none.
|
||||
func (a *API) change(w http.ResponseWriter, r *http.Request, body any, f func(*client.Model) error) (client.Model, bool) {
|
||||
if err := json.NewDecoder(r.Body).Decode(body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return client.Model{}, false
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
m, ok := a.models[r.PathValue("name")]
|
||||
if !ok {
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("model %q was not added at runtime", r.PathValue("name")))
|
||||
return client.Model{}, false
|
||||
}
|
||||
if err := f(&m); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return client.Model{}, false
|
||||
}
|
||||
m = normalize(m)
|
||||
a.models[m.Name] = m
|
||||
return m, true
|
||||
}
|
||||
|
||||
func (a *API) scale(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.ScaleRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
if req.Min < 0 || req.Max < 0 {
|
||||
return fmt.Errorf("a replica count cannot be negative")
|
||||
}
|
||||
m.Scaling = client.Scaling{Min: req.Min, Max: req.Max, IdleTimeout: req.IdleTimeout}
|
||||
return nil
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, map[string]any{"model": m.Name, "min": m.Scaling.Min, "max": m.Scaling.Max,
|
||||
"idle_timeout": m.Scaling.IdleTimeout, "on_demand": m.Scaling.Min == 0 && m.Scaling.Max > 0})
|
||||
}
|
||||
}
|
||||
|
||||
func (a *API) class(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.ClassRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
switch req.Class {
|
||||
case "", "critical", "standard", "opportunistic":
|
||||
m.Class = req.Class
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("class %q is not critical, standard or opportunistic", req.Class)
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "class": m.Class})
|
||||
}
|
||||
}
|
||||
|
||||
func (a *API) owner(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.OwnerRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
if req.Owner == "" {
|
||||
m.Owner = ""
|
||||
return nil
|
||||
}
|
||||
key, err := a.resolve(req.Owner)
|
||||
m.Owner = key
|
||||
return err
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, client.OwnerView{Model: m.Name, Owner: m.Owner})
|
||||
}
|
||||
}
|
||||
|
||||
func (a *API) adapters(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.AdaptersRequest
|
||||
m, ok := a.change(w, r, &req, func(m *client.Model) error {
|
||||
m.Adapters = req.Adapters
|
||||
if len(m.Adapters) == 0 {
|
||||
m.Adapters = nil
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if ok {
|
||||
writeJSON(w, http.StatusOK, client.AdaptersRequest{Adapters: m.Adapters})
|
||||
}
|
||||
}
|
||||
|
||||
// resolve reads org:X, team:X or project:X, X an ID or a path, as scope:ID.
|
||||
// Must be called holding mu.
|
||||
func (a *API) resolve(s string) (string, error) {
|
||||
kind, what, ok := strings.Cut(s, ":")
|
||||
if !ok || what == "" {
|
||||
return "", fmt.Errorf("%q is not org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT", s)
|
||||
}
|
||||
if _, ok := a.scopes[s]; ok {
|
||||
return s, nil
|
||||
}
|
||||
norm := func(p string) string {
|
||||
parts := strings.Split(p, "/")
|
||||
for i := range parts {
|
||||
parts[i] = strings.ToLower(strings.TrimSpace(parts[i]))
|
||||
}
|
||||
return strings.Join(parts, "/")
|
||||
}
|
||||
for key, name := range a.scopes {
|
||||
if strings.HasPrefix(key, kind+":") && norm(name) == norm(what) {
|
||||
return key, nil
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("there is no %s %q", kind, what)
|
||||
}
|
||||
|
||||
func (a *API) listQuotas(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
out := []client.QuotaView{}
|
||||
for _, q := range a.quotas {
|
||||
out = append(out, q)
|
||||
}
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].Scope < out[j].Scope })
|
||||
writeJSON(w, http.StatusOK, out)
|
||||
}
|
||||
|
||||
func (a *API) putQuota(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.QuotaRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
key, err := a.resolve(req.Scope)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
empty := func(r client.Resources) bool { return r.GPUs == nil && r.CPUCores == nil && r.MemoryGB == nil }
|
||||
if empty(req.Guarantee) && empty(req.Limit) {
|
||||
writeError(w, http.StatusBadRequest, "a quota needs a guarantee or a limit; to remove one, clear it")
|
||||
return
|
||||
}
|
||||
a.quotas[key] = client.QuotaView{Scope: key, Name: a.scopes[key], Guarantee: req.Guarantee, Limit: req.Limit}
|
||||
writeJSON(w, http.StatusOK, client.QuotaSet{Scope: key})
|
||||
}
|
||||
|
||||
func (a *API) deleteQuota(w http.ResponseWriter, r *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
key, err := a.resolve(r.PathValue("scope"))
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
delete(a.quotas, key)
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": key})
|
||||
}
|
||||
|
||||
func (a *API) guard(w http.ResponseWriter, _ *http.Request) {
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
type view struct {
|
||||
client.Attachment
|
||||
Name string `json:"name,omitempty"`
|
||||
}
|
||||
atts := []view{}
|
||||
for _, at := range a.attachments {
|
||||
atts = append(atts, view{Attachment: at, Name: a.scopes[at.Scope+":"+at.ScopeID]})
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]any{
|
||||
"policies": []string{"baseline", "finance"}, "default": []string{"baseline"}, "attachments": atts,
|
||||
})
|
||||
}
|
||||
|
||||
func (a *API) attach(on bool) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
var req client.GuardAttachmentRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
kind, what, _ := strings.Cut(req.Target, ":")
|
||||
at := client.Attachment{Scope: kind, ScopeID: what, Policy: req.Policy}
|
||||
if kind == "org" || kind == "team" || kind == "project" {
|
||||
key, err := a.resolve(req.Target)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
_, at.ScopeID, _ = strings.Cut(key, ":")
|
||||
} else if kind != "key" && kind != "user" {
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("%q: attach to org, team, project, key or user", kind))
|
||||
return
|
||||
}
|
||||
i := slices.Index(a.attachments, at)
|
||||
switch {
|
||||
case on && req.Policy != "baseline" && req.Policy != "finance":
|
||||
writeError(w, http.StatusBadRequest, fmt.Sprintf("no policy %q in server.yaml's guard.policies", req.Policy))
|
||||
return
|
||||
case on && i < 0:
|
||||
a.attachments = append(a.attachments, at)
|
||||
case !on && i < 0:
|
||||
writeError(w, http.StatusNotFound, "no such attachment")
|
||||
return
|
||||
case !on:
|
||||
a.attachments = slices.Delete(a.attachments, i, i+1)
|
||||
}
|
||||
writeJSON(w, http.StatusOK, at)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,209 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/datasource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/datasource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// --- llmgw_models ---
|
||||
|
||||
type modelsDataSource struct{ c *client.Client }
|
||||
|
||||
func newModelsDataSource() datasource.DataSource { return &modelsDataSource{} }
|
||||
|
||||
type modelsData struct {
|
||||
Models []modelSummary `tfsdk:"models"`
|
||||
}
|
||||
|
||||
type modelSummary struct {
|
||||
Name types.String `tfsdk:"name"`
|
||||
Description types.String `tfsdk:"description"`
|
||||
Source types.String `tfsdk:"source"`
|
||||
Aliases []string `tfsdk:"aliases"`
|
||||
Profiles []string `tfsdk:"profiles"`
|
||||
Class types.String `tfsdk:"class"`
|
||||
Owner types.String `tfsdk:"owner"`
|
||||
External types.String `tfsdk:"external"`
|
||||
MinReplicas types.Int64 `tfsdk:"min_replicas"`
|
||||
MaxReplicas types.Int64 `tfsdk:"max_replicas"`
|
||||
ReadyReplicas types.Int64 `tfsdk:"ready_replicas"`
|
||||
TotalReplicas types.Int64 `tfsdk:"total_replicas"`
|
||||
DesiredReplicas types.Int64 `tfsdk:"desired_replicas"`
|
||||
Parked types.Bool `tfsdk:"parked"`
|
||||
Reason types.String `tfsdk:"reason"`
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Metadata(_ context.Context, req datasource.MetadataRequest, resp *datasource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_models"
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Schema(_ context.Context, _ datasource.SchemaRequest, resp *datasource.SchemaResponse) {
|
||||
str := func(desc string) schema.StringAttribute {
|
||||
return schema.StringAttribute{Computed: true, Description: desc}
|
||||
}
|
||||
num := func(desc string) schema.Int64Attribute {
|
||||
return schema.Int64Attribute{Computed: true, Description: desc}
|
||||
}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "Every model in the catalog, from configuration and added at runtime, with its replicas now.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"models": schema.ListNestedAttribute{Computed: true, Description: "The models, in the catalog's order.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"name": str("The name clients ask for."),
|
||||
"description": str("What it is for."),
|
||||
"source": str("Where its weights are."),
|
||||
"aliases": schema.ListAttribute{Computed: true, ElementType: types.StringType, Description: "Other names it answers to."},
|
||||
"profiles": schema.ListAttribute{Computed: true, ElementType: types.StringType, Description: "Its profiles' names."},
|
||||
"class": str("Its capacity class."),
|
||||
"owner": str("Its owner, scope:ID; empty is shared."),
|
||||
"external": str("Where an external model is served: the provider's host and its name for it."),
|
||||
"min_replicas": num("Replicas kept running."),
|
||||
"max_replicas": num("The most replicas."),
|
||||
"ready_replicas": num("Replicas serving now."),
|
||||
"total_replicas": num("Replicas in any state."),
|
||||
"desired_replicas": num("Replicas it should have now."),
|
||||
"parked": schema.BoolAttribute{Computed: true, Description: "It failed to start too often and is no longer retried."},
|
||||
"reason": str("The last error, when parked."),
|
||||
}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Configure(_ context.Context, req datasource.ConfigureRequest, _ *datasource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
d.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func (d *modelsDataSource) Read(ctx context.Context, _ datasource.ReadRequest, resp *datasource.ReadResponse) {
|
||||
ms, err := d.c.Models(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Listing the models", err.Error())
|
||||
return
|
||||
}
|
||||
out := modelsData{Models: []modelSummary{}}
|
||||
for _, m := range ms {
|
||||
out.Models = append(out.Models, modelSummary{
|
||||
Name: types.StringValue(m.Name), Description: types.StringValue(m.Description), Source: types.StringValue(m.Source),
|
||||
Aliases: nonNil(m.Aliases), Profiles: nonNil(m.Profiles), Class: types.StringValue(m.Class),
|
||||
Owner: types.StringValue(m.Owner), External: types.StringValue(m.External),
|
||||
MinReplicas: types.Int64Value(m.Min), MaxReplicas: types.Int64Value(m.Max), ReadyReplicas: types.Int64Value(m.Ready),
|
||||
TotalReplicas: types.Int64Value(m.Total), DesiredReplicas: types.Int64Value(m.Desired),
|
||||
Parked: types.BoolValue(m.Parked), Reason: types.StringValue(m.Reason),
|
||||
})
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &out)...)
|
||||
}
|
||||
|
||||
// nonNil is an empty list for none, which reads better in an expression than
|
||||
// a null one: length() of it is 0 rather than an error.
|
||||
func nonNil(s []string) []string {
|
||||
if s == nil {
|
||||
return []string{}
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
// --- llmgw_nodes ---
|
||||
|
||||
type nodesDataSource struct{ c *client.Client }
|
||||
|
||||
func newNodesDataSource() datasource.DataSource { return &nodesDataSource{} }
|
||||
|
||||
type nodesData struct {
|
||||
Nodes []nodeSummary `tfsdk:"nodes"`
|
||||
}
|
||||
|
||||
type nodeSummary struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Hostname types.String `tfsdk:"hostname"`
|
||||
Address types.String `tfsdk:"address"`
|
||||
State types.String `tfsdk:"state"`
|
||||
Labels map[string]string `tfsdk:"labels"`
|
||||
GPUTotal types.Int64 `tfsdk:"gpu_total"`
|
||||
GPUFree types.Int64 `tfsdk:"gpu_free"`
|
||||
GPUClasses []string `tfsdk:"gpu_classes"`
|
||||
MemoryGB types.Int64 `tfsdk:"memory_gb"`
|
||||
CPUCount types.Int64 `tfsdk:"cpu_count"`
|
||||
Engines []string `tfsdk:"engines"`
|
||||
Fabrics []string `tfsdk:"fabrics"`
|
||||
OS types.String `tfsdk:"os"`
|
||||
AgentVersion types.String `tfsdk:"agent_version"`
|
||||
Replicas types.Int64 `tfsdk:"replicas"`
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Metadata(_ context.Context, req datasource.MetadataRequest, resp *datasource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_nodes"
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Schema(_ context.Context, _ datasource.SchemaRequest, resp *datasource.SchemaResponse) {
|
||||
str := func(desc string) schema.StringAttribute {
|
||||
return schema.StringAttribute{Computed: true, Description: desc}
|
||||
}
|
||||
num := func(desc string) schema.Int64Attribute {
|
||||
return schema.Int64Attribute{Computed: true, Description: desc}
|
||||
}
|
||||
list := func(desc string) schema.ListAttribute {
|
||||
return schema.ListAttribute{Computed: true, ElementType: types.StringType, Description: desc}
|
||||
}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "Every node the control plane knows, with its cards and state now.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"nodes": schema.ListNestedAttribute{Computed: true, Description: "The nodes.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"id": str("The node's ID."),
|
||||
"hostname": str("Its hostname."),
|
||||
"address": str("Its management address."),
|
||||
"state": str("ready, cordoned, draining, unreachable or lost."),
|
||||
"labels": schema.MapAttribute{Computed: true, ElementType: types.StringType, Description: "Its labels."},
|
||||
"gpu_total": num("Its GPUs."),
|
||||
"gpu_free": num("GPUs not allocated."),
|
||||
"gpu_classes": list("The classes of its GPUs, h100-80g."),
|
||||
"memory_gb": num("Its memory, 0 when not reported."),
|
||||
"cpu_count": num("Its CPUs, 0 when not reported."),
|
||||
"engines": list("The inference engines installed."),
|
||||
"fabrics": list("The fabrics its adapters are on."),
|
||||
"os": str("Its operating system and architecture."),
|
||||
"agent_version": str("The agent's version."),
|
||||
"replicas": num("Replicas with a shard on it."),
|
||||
}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Configure(_ context.Context, req datasource.ConfigureRequest, _ *datasource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
d.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func (d *nodesDataSource) Read(ctx context.Context, _ datasource.ReadRequest, resp *datasource.ReadResponse) {
|
||||
ns, err := d.c.Nodes(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Listing the nodes", err.Error())
|
||||
return
|
||||
}
|
||||
out := nodesData{Nodes: []nodeSummary{}}
|
||||
for _, n := range ns {
|
||||
labels := n.Labels
|
||||
if labels == nil {
|
||||
labels = map[string]string{}
|
||||
}
|
||||
out.Nodes = append(out.Nodes, nodeSummary{
|
||||
ID: types.StringValue(n.ID), Hostname: types.StringValue(n.Hostname), Address: types.StringValue(n.MgmtAddr),
|
||||
State: types.StringValue(n.State), Labels: labels,
|
||||
GPUTotal: types.Int64Value(n.GPUTotal), GPUFree: types.Int64Value(n.GPUFree), GPUClasses: nonNil(n.Classes),
|
||||
MemoryGB: types.Int64Value(n.MemoryGB), CPUCount: types.Int64Value(n.CPUCount),
|
||||
Engines: nonNil(n.Engines), Fabrics: nonNil(n.Fabrics), OS: types.StringValue(n.OS),
|
||||
AgentVersion: types.StringValue(n.AgentVersion), Replicas: types.Int64Value(n.Replicas),
|
||||
})
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &out)...)
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
// Package provider is the llmgw Terraform provider: models, quotas and
|
||||
// guardrail attachments managed through a control plane's admin API.
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/datasource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/provider"
|
||||
"github.com/hashicorp/terraform-plugin-framework/provider/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// DefaultEndpoint is the admin listener's default address.
|
||||
const DefaultEndpoint = "http://127.0.0.1:9090"
|
||||
|
||||
// DefaultTokenFile is where the server writes the admin token under systemd.
|
||||
const DefaultTokenFile = "/var/lib/llmgw/admin-token"
|
||||
|
||||
type llmgwProvider struct{ version string }
|
||||
|
||||
// New returns the provider, for providerserver.
|
||||
func New(version string) func() provider.Provider {
|
||||
return func() provider.Provider { return &llmgwProvider{version: version} }
|
||||
}
|
||||
|
||||
type providerConfig struct {
|
||||
Endpoint types.String `tfsdk:"endpoint"`
|
||||
Token types.String `tfsdk:"token"`
|
||||
TokenFile types.String `tfsdk:"token_file"`
|
||||
Actor types.String `tfsdk:"actor"`
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Metadata(_ context.Context, _ provider.MetadataRequest, resp *provider.MetadataResponse) {
|
||||
resp.TypeName = "llmgw"
|
||||
resp.Version = p.version
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Schema(_ context.Context, _ provider.SchemaRequest, resp *provider.SchemaResponse) {
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "Manages an llmgw cluster through its control plane's admin API.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"endpoint": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Description: "The admin listener, " + DefaultEndpoint + " by default, or LLMGW_ADMIN_URL. " +
|
||||
"It answers to an address, not a name; reach a remote one through an SSH tunnel.",
|
||||
},
|
||||
"token": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Sensitive: true,
|
||||
Description: "The admin token, or LLMGW_ADMIN_TOKEN. When neither is set it is read from token_file.",
|
||||
},
|
||||
"token_file": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Description: "A file holding the admin token; " + DefaultTokenFile + " by default.",
|
||||
},
|
||||
"actor": schema.StringAttribute{
|
||||
Optional: true,
|
||||
Description: "Who changes are recorded as made by in the audit log; terraform by default.",
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Configure(ctx context.Context, req provider.ConfigureRequest, resp *provider.ConfigureResponse) {
|
||||
var cfg providerConfig
|
||||
resp.Diagnostics.Append(req.Config.Get(ctx, &cfg)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
pick := func(v types.String, env, def string) string {
|
||||
if !v.IsNull() && !v.IsUnknown() && v.ValueString() != "" {
|
||||
return v.ValueString()
|
||||
}
|
||||
if s := os.Getenv(env); env != "" && s != "" {
|
||||
return s
|
||||
}
|
||||
return def
|
||||
}
|
||||
endpoint := pick(cfg.Endpoint, "LLMGW_ADMIN_URL", DefaultEndpoint)
|
||||
token := pick(cfg.Token, "LLMGW_ADMIN_TOKEN", "")
|
||||
if token == "" {
|
||||
file := pick(cfg.TokenFile, "", DefaultTokenFile)
|
||||
b, err := os.ReadFile(file) //nolint:gosec // the file the configuration names
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("No admin token",
|
||||
"Set token or LLMGW_ADMIN_TOKEN, or make "+file+" readable (it is root's): "+err.Error())
|
||||
return
|
||||
}
|
||||
token = strings.TrimSpace(string(b))
|
||||
}
|
||||
c, err := client.New(endpoint, token, pick(cfg.Actor, "", "terraform"))
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Cannot reach the admin API", err.Error())
|
||||
return
|
||||
}
|
||||
resp.ResourceData = c
|
||||
resp.DataSourceData = c
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) Resources(context.Context) []func() resource.Resource {
|
||||
return []func() resource.Resource{newModelResource, newQuotaResource, newGuardAttachmentResource}
|
||||
}
|
||||
|
||||
func (p *llmgwProvider) DataSources(context.Context) []func() datasource.DataSource {
|
||||
return []func() datasource.DataSource{newModelsDataSource, newNodesDataSource}
|
||||
}
|
||||
|
||||
// clientFrom takes the client Configure built, which is absent while
|
||||
// Terraform validates a configuration before the provider is configured.
|
||||
func clientFrom(data any) (*client.Client, bool) {
|
||||
c, ok := data.(*client.Client)
|
||||
return c, ok && c != nil
|
||||
}
|
||||
@@ -0,0 +1,307 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/providerserver"
|
||||
"github.com/hashicorp/terraform-plugin-go/tfprotov6"
|
||||
"github.com/hashicorp/terraform-plugin-testing/helper/resource"
|
||||
"github.com/hashicorp/terraform-plugin-testing/plancheck"
|
||||
"github.com/hashicorp/terraform-plugin-testing/terraform"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/fakeapi"
|
||||
)
|
||||
|
||||
// These run Terraform itself against the provider and an admin API in
|
||||
// memory, and so need TF_ACC=1 and a terraform binary (on PATH, or
|
||||
// TF_ACC_TERRAFORM_PATH); without TF_ACC they are skipped. Each step's apply
|
||||
// is followed by a plan that must be empty, which is what shows the state
|
||||
// read back does not differ from the configuration on every run.
|
||||
|
||||
var factories = map[string]func() (tfprotov6.ProviderServer, error){
|
||||
"llmgw": providerserver.NewProtocol6WithError(New("test")()),
|
||||
}
|
||||
|
||||
func fake(t *testing.T) (*fakeapi.API, string) {
|
||||
t.Helper()
|
||||
api := fakeapi.New("secret",
|
||||
client.NodeView{ID: "gpu01", State: "ready", GPUTotal: 8, GPUFree: 8, Classes: []string{"h100-80g"}},
|
||||
client.NodeView{ID: "cpu01", State: "cordoned", CPUCount: 32, MemoryGB: 128})
|
||||
srv := httptest.NewServer(api)
|
||||
t.Cleanup(srv.Close)
|
||||
return api, fmt.Sprintf(`
|
||||
provider "llmgw" {
|
||||
endpoint = %q
|
||||
token = "secret"
|
||||
}
|
||||
`, srv.URL)
|
||||
}
|
||||
|
||||
const modelV1 = `
|
||||
resource "llmgw_model" "qwen" {
|
||||
name = "qwen-small"
|
||||
source = "/srv/models/qwen.gguf"
|
||||
aliases = ["qwen"]
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 4
|
||||
cpu_cores = 4
|
||||
}]
|
||||
scaling = {
|
||||
min = 1
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
}
|
||||
`
|
||||
|
||||
// In place: scaling, class and owner.
|
||||
const modelV2 = `
|
||||
resource "llmgw_model" "qwen" {
|
||||
name = "qwen-small"
|
||||
source = "/srv/models/qwen.gguf"
|
||||
aliases = ["qwen"]
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 4
|
||||
cpu_cores = 4
|
||||
}]
|
||||
scaling = {
|
||||
min = 0
|
||||
max = 2
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
class = "opportunistic"
|
||||
owner = "team:Acme/ml"
|
||||
}
|
||||
`
|
||||
|
||||
// A profile changed: the model is replaced.
|
||||
const modelV3 = `
|
||||
resource "llmgw_model" "qwen" {
|
||||
name = "qwen-small"
|
||||
source = "/srv/models/qwen.gguf"
|
||||
aliases = ["qwen"]
|
||||
profiles = [{
|
||||
name = "cpu"
|
||||
engine = "llamacpp"
|
||||
min_memory_gb = 6
|
||||
cpu_cores = 4
|
||||
}]
|
||||
scaling = {
|
||||
min = 0
|
||||
max = 2
|
||||
idle_timeout = "30m"
|
||||
}
|
||||
class = "opportunistic"
|
||||
owner = "team:Acme/ml"
|
||||
}
|
||||
`
|
||||
|
||||
func TestAccAModelIsAddedChangedInPlaceReplacedAndRemoved(t *testing.T) {
|
||||
api, cfg := fake(t)
|
||||
inCluster := func(check func(client.Model) error) resource.TestCheckFunc {
|
||||
return func(*terraform.State) error {
|
||||
m, ok := api.Model("qwen-small")
|
||||
if !ok {
|
||||
return fmt.Errorf("qwen-small is not in the catalog")
|
||||
}
|
||||
return check(m)
|
||||
}
|
||||
}
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
CheckDestroy: func(*terraform.State) error {
|
||||
if _, ok := api.Model("qwen-small"); ok {
|
||||
return fmt.Errorf("qwen-small is still in the catalog")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + modelV1,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "id", "qwen-small"),
|
||||
// Filled in by the catalog: a max not written follows min.
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "scaling.max", "1"),
|
||||
// As written, not as the catalog defaulted it.
|
||||
resource.TestCheckNoResourceAttr("llmgw_model.qwen", "profiles.0.tp"),
|
||||
inCluster(func(m client.Model) error {
|
||||
if m.Scaling.IdleTimeout != 30*time.Minute || m.Profiles[0].MinMemoryGB != 4 {
|
||||
return fmt.Errorf("added as %+v", m)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
),
|
||||
},
|
||||
{
|
||||
Config: cfg + modelV2,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionUpdate),
|
||||
}},
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "owner", "team:Acme/ml"),
|
||||
resource.TestCheckResourceAttr("llmgw_model.qwen", "owner_id", "team:t1"),
|
||||
inCluster(func(m client.Model) error {
|
||||
if m.Scaling.Min != 0 || m.Scaling.Max != 2 || m.Class != "opportunistic" || m.Owner != "team:t1" {
|
||||
return fmt.Errorf("changed to %+v", m)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
),
|
||||
},
|
||||
{
|
||||
// Scaled from a terminal: the next plan puts it back, in place.
|
||||
PreConfig: func() { api.ScaleBehindTerraform("qwen-small", 3, 3) },
|
||||
Config: cfg + modelV2,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionUpdate),
|
||||
}},
|
||||
Check: inCluster(func(m client.Model) error {
|
||||
if m.Scaling.Min != 0 || m.Scaling.Max != 2 {
|
||||
return fmt.Errorf("not put back: %+v", m.Scaling)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
},
|
||||
{
|
||||
Config: cfg + modelV3,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionDestroyBeforeCreate),
|
||||
}},
|
||||
Check: inCluster(func(m client.Model) error {
|
||||
if m.Profiles[0].MinMemoryGB != 6 || m.Owner != "team:t1" {
|
||||
return fmt.Errorf("replaced as %+v", m)
|
||||
}
|
||||
return nil
|
||||
}),
|
||||
},
|
||||
{
|
||||
// Removed from a terminal: planned again.
|
||||
PreConfig: func() { api.RemoveBehindTerraform("qwen-small") },
|
||||
Config: cfg + modelV3,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_model.qwen", plancheck.ResourceActionCreate),
|
||||
}},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestAccAQuotaIsSetChangedImportedAndCleared(t *testing.T) {
|
||||
api, cfg := fake(t)
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
CheckDestroy: func(*terraform.State) error {
|
||||
if _, ok := api.Quota("team:t1"); ok {
|
||||
return fmt.Errorf("the quota is still set")
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + `
|
||||
resource "llmgw_quota" "ml" {
|
||||
scope = "team:Acme/ml"
|
||||
limit = { gpus = 8 }
|
||||
}
|
||||
`,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_quota.ml", "id", "team:t1"),
|
||||
resource.TestCheckResourceAttr("llmgw_quota.ml", "name", "Acme / ml"),
|
||||
resource.TestCheckResourceAttr("llmgw_quota.ml", "limit.gpus", "8"),
|
||||
),
|
||||
},
|
||||
{
|
||||
Config: cfg + `
|
||||
resource "llmgw_quota" "ml" {
|
||||
scope = "team:Acme/ml"
|
||||
guarantee = { gpus = 2 }
|
||||
limit = { gpus = 8, memory_gb = 256 }
|
||||
}
|
||||
`,
|
||||
ConfigPlanChecks: resource.ConfigPlanChecks{PreApply: []plancheck.PlanCheck{
|
||||
plancheck.ExpectResourceAction("llmgw_quota.ml", plancheck.ResourceActionUpdate),
|
||||
}},
|
||||
Check: func(*terraform.State) error {
|
||||
q, ok := api.Quota("team:t1")
|
||||
if !ok || q.Guarantee.GPUs == nil || *q.Guarantee.GPUs != 2 || q.Limit.MemoryGB == nil || *q.Limit.MemoryGB != 256 {
|
||||
return fmt.Errorf("quota %+v", q)
|
||||
}
|
||||
return nil
|
||||
},
|
||||
},
|
||||
{
|
||||
ResourceName: "llmgw_quota.ml",
|
||||
ImportState: true,
|
||||
ImportStateId: "team:t1",
|
||||
ImportStateVerify: true,
|
||||
// Imported by its ID, which is then the only name it has.
|
||||
ImportStateVerifyIgnore: []string{"scope"},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestAccAPolicyIsAttachedImportedAndDetached(t *testing.T) {
|
||||
api, cfg := fake(t)
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
CheckDestroy: func(*terraform.State) error {
|
||||
if n := len(api.Attachments()); n != 0 {
|
||||
return fmt.Errorf("%d attachments left", n)
|
||||
}
|
||||
return nil
|
||||
},
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + `
|
||||
resource "llmgw_guard_attachment" "ml" {
|
||||
target = "team:Acme/ml"
|
||||
policy = "finance"
|
||||
}
|
||||
`,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("llmgw_guard_attachment.ml", "id", "team:t1:finance"),
|
||||
resource.TestCheckResourceAttr("llmgw_guard_attachment.ml", "scope_id", "t1"),
|
||||
),
|
||||
},
|
||||
{
|
||||
ResourceName: "llmgw_guard_attachment.ml",
|
||||
ImportState: true,
|
||||
ImportStateVerify: true,
|
||||
ImportStateVerifyIgnore: []string{"target"},
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
|
||||
func TestAccTheModelsAndNodesAreRead(t *testing.T) {
|
||||
_, cfg := fake(t)
|
||||
resource.Test(t, resource.TestCase{
|
||||
ProtoV6ProviderFactories: factories,
|
||||
Steps: []resource.TestStep{
|
||||
{
|
||||
Config: cfg + modelV1 + `
|
||||
data "llmgw_nodes" "all" {}
|
||||
data "llmgw_models" "all" {
|
||||
depends_on = [llmgw_model.qwen]
|
||||
}
|
||||
`,
|
||||
Check: resource.ComposeAggregateTestCheckFunc(
|
||||
resource.TestCheckResourceAttr("data.llmgw_nodes.all", "nodes.#", "2"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_nodes.all", "nodes.0.gpu_classes.0", "h100-80g"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_nodes.all", "nodes.1.state", "cordoned"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_models.all", "models.#", "1"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_models.all", "models.0.name", "qwen-small"),
|
||||
resource.TestCheckResourceAttr("data.llmgw_models.all", "models.0.profiles.0", "cpu"),
|
||||
),
|
||||
},
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,139 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// The guard attachment resource is a guardrail policy attached to a part of
|
||||
// the tenancy tree, a key or a person. The policies themselves are defined
|
||||
// in server.yaml's guard section, not here: what this manages is where they
|
||||
// apply.
|
||||
|
||||
type guardAttachmentResource struct{ c *client.Client }
|
||||
|
||||
func newGuardAttachmentResource() resource.Resource { return &guardAttachmentResource{} }
|
||||
|
||||
type guardAttachmentModel struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Target types.String `tfsdk:"target"`
|
||||
Policy types.String `tfsdk:"policy"`
|
||||
Scope types.String `tfsdk:"scope"`
|
||||
ScopeID types.String `tfsdk:"scope_id"`
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_guard_attachment"
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Schema(_ context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {
|
||||
replace := []planmodifier.String{stringplanmodifier.RequiresReplace()}
|
||||
keep := []planmodifier.String{stringplanmodifier.UseStateForUnknown()}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "A guardrail policy from server.yaml's guard.policies, attached to an organization, team, project, key or person.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"id": schema.StringAttribute{Computed: true, PlanModifiers: keep,
|
||||
Description: "scope:SCOPE-ID:policy. Import by it."},
|
||||
"target": schema.StringAttribute{Required: true, PlanModifiers: replace,
|
||||
Description: "org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT (or the ID after the colon), key:KEY-ID or user:USER-ID."},
|
||||
"policy": schema.StringAttribute{Required: true, PlanModifiers: replace,
|
||||
Description: "The policy's name in server.yaml's guard.policies."},
|
||||
"scope": schema.StringAttribute{Computed: true, PlanModifiers: keep,
|
||||
Description: "org, team, project, key or user."},
|
||||
"scope_id": schema.StringAttribute{Computed: true, PlanModifiers: keep,
|
||||
Description: "The ID the target resolved to."},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Configure(_ context.Context, req resource.ConfigureRequest, _ *resource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
r.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func attachmentID(a client.Attachment) string { return a.Scope + ":" + a.ScopeID + ":" + a.Policy }
|
||||
|
||||
func (r *guardAttachmentResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
|
||||
var plan guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
a, err := r.c.Attach(ctx, plan.Target.ValueString(), plan.Policy.ValueString())
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Attaching "+plan.Policy.ValueString()+" to "+plan.Target.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
plan.ID = types.StringValue(attachmentID(a))
|
||||
plan.Scope = types.StringValue(a.Scope)
|
||||
plan.ScopeID = types.StringValue(a.ScopeID)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
|
||||
var st guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
g, err := r.c.Guard(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the guard attachments", err.Error())
|
||||
return
|
||||
}
|
||||
for _, a := range g.Attachments {
|
||||
if attachmentID(a) == st.ID.ValueString() {
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
return
|
||||
}
|
||||
}
|
||||
resp.State.RemoveResource(ctx)
|
||||
}
|
||||
|
||||
// Update is never called: every attribute replaces the attachment.
|
||||
func (r *guardAttachmentResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
|
||||
var plan guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
|
||||
var st guardAttachmentModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
// By the ID it resolved to, which still names it if the team has since
|
||||
// been renamed.
|
||||
err := r.c.Detach(ctx, st.Scope.ValueString()+":"+st.ScopeID.ValueString(), st.Policy.ValueString())
|
||||
if err != nil && !client.IsNotFound(err) {
|
||||
resp.Diagnostics.AddError("Detaching "+st.Policy.ValueString()+" from "+st.Target.ValueString(), err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func (r *guardAttachmentResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) {
|
||||
// scope:SCOPE-ID:policy. The policy is after the last colon and the scope
|
||||
// before the first; an ID between them is whatever is left.
|
||||
scope, rest, ok1 := strings.Cut(req.ID, ":")
|
||||
i := strings.LastIndex(rest, ":")
|
||||
if !ok1 || i <= 0 || i == len(rest)-1 {
|
||||
resp.Diagnostics.AddError("Not an attachment ID", fmt.Sprintf("%q is not scope:SCOPE-ID:policy, as id shows it", req.ID))
|
||||
return
|
||||
}
|
||||
id, policy := rest[:i], rest[i+1:]
|
||||
st := guardAttachmentModel{
|
||||
ID: types.StringValue(req.ID), Target: types.StringValue(scope + ":" + id), Policy: types.StringValue(policy),
|
||||
Scope: types.StringValue(scope), ScopeID: types.StringValue(id),
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
}
|
||||
@@ -0,0 +1,424 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/diag"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/int64planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/listplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/objectplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// The model resource is a catalog entry added at runtime, as `llmgw ctl
|
||||
// model add` adds one.
|
||||
//
|
||||
// What the admin API can change in place — scaling, class, owner, adapters —
|
||||
// is changed in place; anything else in the definition replaces the model,
|
||||
// because the API has no way to edit it and removing and adding is what an
|
||||
// operator would do by hand. A replacement stops the model's replicas.
|
||||
//
|
||||
// The state keeps the definition as it was written rather than as the
|
||||
// catalog filled it in, so a profile written without tp does not differ on
|
||||
// every plan from one the catalog gave tp 1. Only what can change in place is
|
||||
// read back from the cluster, where a change made from a terminal shows as a
|
||||
// difference to put right. A model removed from the cluster is planned again.
|
||||
|
||||
type modelResource struct{ c *client.Client }
|
||||
|
||||
func newModelResource() resource.Resource { return &modelResource{} }
|
||||
|
||||
type modelResourceModel struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Name types.String `tfsdk:"name"`
|
||||
Source types.String `tfsdk:"source"`
|
||||
Description types.String `tfsdk:"description"`
|
||||
Aliases types.List `tfsdk:"aliases"`
|
||||
Fallbacks types.List `tfsdk:"fallbacks"`
|
||||
RoutingProfile types.String `tfsdk:"routing_profile"`
|
||||
ContextLength types.Int64 `tfsdk:"context_length"`
|
||||
MaxOutput types.Int64 `tfsdk:"max_output"`
|
||||
Class types.String `tfsdk:"class"`
|
||||
Owner types.String `tfsdk:"owner"`
|
||||
OwnerID types.String `tfsdk:"owner_id"`
|
||||
Capabilities *capabilitiesModel `tfsdk:"capabilities"`
|
||||
External *externalModel `tfsdk:"external"`
|
||||
Profiles []profileModel `tfsdk:"profiles"`
|
||||
Scaling *scalingModel `tfsdk:"scaling"`
|
||||
Adapters []adapterModel `tfsdk:"adapters"`
|
||||
}
|
||||
|
||||
type capabilitiesModel struct {
|
||||
Tools types.Bool `tfsdk:"tools"`
|
||||
Vision types.Bool `tfsdk:"vision"`
|
||||
Embeddings types.Bool `tfsdk:"embeddings"`
|
||||
}
|
||||
|
||||
type externalModel struct {
|
||||
URL types.String `tfsdk:"url"`
|
||||
Model types.String `tfsdk:"model"`
|
||||
APIKeyFile types.String `tfsdk:"api_key_file"`
|
||||
}
|
||||
|
||||
type profileModel struct {
|
||||
Name types.String `tfsdk:"name"`
|
||||
Engine types.String `tfsdk:"engine"`
|
||||
GPUClass types.String `tfsdk:"gpu_class"`
|
||||
TP types.Int64 `tfsdk:"tp"`
|
||||
PP types.Int64 `tfsdk:"pp"`
|
||||
GPUsPerNode types.Int64 `tfsdk:"gpus_per_node"`
|
||||
Nodes types.Int64 `tfsdk:"nodes"`
|
||||
Args types.List `tfsdk:"args"`
|
||||
Env types.Map `tfsdk:"env"`
|
||||
MinMemoryGB types.Int64 `tfsdk:"min_memory_gb"`
|
||||
CPUCores types.Int64 `tfsdk:"cpu_cores"`
|
||||
MinDiskGB types.Int64 `tfsdk:"min_disk_gb"`
|
||||
NodeSelector types.Map `tfsdk:"node_selector"`
|
||||
Weight types.Float64 `tfsdk:"weight"`
|
||||
Priority types.Int64 `tfsdk:"priority"`
|
||||
StartupTimeout types.String `tfsdk:"startup_timeout"`
|
||||
}
|
||||
|
||||
type scalingModel struct {
|
||||
Min types.Int64 `tfsdk:"min"`
|
||||
Max types.Int64 `tfsdk:"max"`
|
||||
IdleTimeout types.String `tfsdk:"idle_timeout"`
|
||||
}
|
||||
|
||||
type adapterModel struct {
|
||||
Name types.String `tfsdk:"name"`
|
||||
Source types.String `tfsdk:"source"`
|
||||
Scale types.Float64 `tfsdk:"scale"`
|
||||
}
|
||||
|
||||
func (r *modelResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_model"
|
||||
}
|
||||
|
||||
func (r *modelResource) Schema(_ context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {
|
||||
replace := []planmodifier.String{stringplanmodifier.RequiresReplace()}
|
||||
replaceInt := []planmodifier.Int64{int64planmodifier.RequiresReplace()}
|
||||
replaceList := []planmodifier.List{listplanmodifier.RequiresReplace()}
|
||||
replaceObject := []planmodifier.Object{objectplanmodifier.RequiresReplace()}
|
||||
str := func(desc string) schema.StringAttribute {
|
||||
return schema.StringAttribute{Optional: true, Description: desc}
|
||||
}
|
||||
num := func(desc string) schema.Int64Attribute {
|
||||
return schema.Int64Attribute{Optional: true, Description: desc}
|
||||
}
|
||||
strMap := func(desc string) schema.MapAttribute {
|
||||
return schema.MapAttribute{Optional: true, ElementType: types.StringType, Description: desc}
|
||||
}
|
||||
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "A model in the catalog, added at runtime and kept in the store, as `llmgw ctl model add` adds one. " +
|
||||
"Scaling, class, owner and adapters change in place; any other change replaces the model, stopping its replicas. " +
|
||||
"A model declared in the control plane's configuration cannot be managed here.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"id": schema.StringAttribute{Computed: true, Description: "The model's name.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
|
||||
"name": schema.StringAttribute{Required: true, PlanModifiers: replace,
|
||||
Description: "The name clients ask for."},
|
||||
"source": schema.StringAttribute{Optional: true, PlanModifiers: replace,
|
||||
Description: "Where the weights are: a path on the nodes, an https URL, or a registry repository. Absent for an external model."},
|
||||
"description": schema.StringAttribute{Optional: true, PlanModifiers: replace,
|
||||
Description: "What the model is for."},
|
||||
"aliases": schema.ListAttribute{Optional: true, ElementType: types.StringType, PlanModifiers: replaceList,
|
||||
Description: "Other names it answers to."},
|
||||
"fallbacks": schema.ListAttribute{Optional: true, ElementType: types.StringType, PlanModifiers: replaceList,
|
||||
Description: "Models tried in order when this one cannot answer."},
|
||||
"routing_profile": schema.StringAttribute{Optional: true, PlanModifiers: replace,
|
||||
Description: "The routing profile, from the router's configuration."},
|
||||
"context_length": schema.Int64Attribute{Optional: true, PlanModifiers: replaceInt,
|
||||
Description: "The context window, in tokens."},
|
||||
"max_output": schema.Int64Attribute{Optional: true, PlanModifiers: replaceInt,
|
||||
Description: "The most tokens one answer may have."},
|
||||
"class": schema.StringAttribute{Optional: true,
|
||||
Description: "The capacity class: critical, standard or opportunistic. Absent is what the scaling implies."},
|
||||
"owner": schema.StringAttribute{Optional: true,
|
||||
Description: "The part of the tenancy tree the model's capacity counts against: org:NAME, team:ORG/TEAM, " +
|
||||
"project:ORG/TEAM/PROJECT, or their IDs. Absent is shared."},
|
||||
"owner_id": schema.StringAttribute{Computed: true,
|
||||
Description: "The owner as the cluster records it, scope:ID."},
|
||||
"capabilities": schema.SingleNestedAttribute{Optional: true, PlanModifiers: replaceObject,
|
||||
Description: "What the model can do beyond chat.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"tools": schema.BoolAttribute{Optional: true, Description: "It calls tools."},
|
||||
"vision": schema.BoolAttribute{Optional: true, Description: "It reads images."},
|
||||
"embeddings": schema.BoolAttribute{Optional: true, Description: "It answers /v1/embeddings."},
|
||||
}},
|
||||
"external": schema.SingleNestedAttribute{Optional: true, PlanModifiers: replaceObject,
|
||||
Description: "A model served by a provider outside the cluster, in place of source and profiles.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"url": schema.StringAttribute{Required: true, Description: "The provider's API, https://api.openai.com/v1."},
|
||||
"model": schema.StringAttribute{Required: true, Description: "The provider's name for the model."},
|
||||
"api_key_file": schema.StringAttribute{Optional: true, Description: "The provider's key: a file on the control plane, or an env: or vault: reference."},
|
||||
}},
|
||||
"profiles": schema.ListNestedAttribute{Optional: true, PlanModifiers: replaceList,
|
||||
Description: "How the model runs: one profile per kind of node, tried in order. " +
|
||||
"What is left out takes the catalog's default, as in server.yaml.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"name": str("The profile's name."),
|
||||
"engine": str("vllm, sglang or llamacpp; vllm when absent."),
|
||||
"gpu_class": str("The GPU class it runs on, h100-80g; absent runs on CPUs."),
|
||||
"tp": num("Tensor parallelism."),
|
||||
"pp": num("Pipeline parallelism."),
|
||||
"gpus_per_node": num("GPUs on each node."),
|
||||
"nodes": num("Nodes one replica spans."),
|
||||
"args": schema.ListAttribute{Optional: true, ElementType: types.StringType, Description: "Arguments added to the engine's."},
|
||||
"env": strMap("Environment for the engine."),
|
||||
"min_memory_gb": num("Host memory one replica reserves, in GB."),
|
||||
"cpu_cores": num("CPU cores one replica without a GPU computes on."),
|
||||
"min_disk_gb": num("Free disk a node needs for the weights, in GB."),
|
||||
"node_selector": strMap("Labels a node must have."),
|
||||
"weight": schema.Float64Attribute{Optional: true, Description: "The share of requests this profile's replicas take."},
|
||||
"priority": num("Placement order among profiles."),
|
||||
"startup_timeout": str("How long a replica may take to start, 20m when absent."),
|
||||
}},
|
||||
},
|
||||
"scaling": schema.SingleNestedAttribute{Optional: true,
|
||||
Description: "How many replicas run. Absent is none: the model is in the catalog and not running. " +
|
||||
"Changes in place.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"min": schema.Int64Attribute{Optional: true, Computed: true, Description: "Replicas kept running; 0 is on demand."},
|
||||
"max": schema.Int64Attribute{Optional: true, Computed: true, Description: "The most replicas; min when absent."},
|
||||
"idle_timeout": schema.StringAttribute{Optional: true, Computed: true,
|
||||
Description: "How long an on-demand model may go unasked-for before it is stopped, 30m; the cluster's default when absent."},
|
||||
}},
|
||||
"adapters": schema.ListNestedAttribute{Optional: true,
|
||||
Description: "LoRA adapters it serves, each a name clients ask for. Changes in place; each new file is checked first.",
|
||||
NestedObject: schema.NestedAttributeObject{Attributes: map[string]schema.Attribute{
|
||||
"name": schema.StringAttribute{Required: true, Description: "The name clients ask for."},
|
||||
"source": schema.StringAttribute{Required: true, Description: "Where the adapter's file is."},
|
||||
"scale": schema.Float64Attribute{Optional: true, Description: "How strongly it applies, 1 when absent."},
|
||||
}},
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (r *modelResource) Configure(_ context.Context, req resource.ConfigureRequest, _ *resource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
r.c = c
|
||||
}
|
||||
}
|
||||
|
||||
// definition is the model to add, from a plan. The owner is left out: it is
|
||||
// set afterwards through its own endpoint, which resolves a path to an ID.
|
||||
func definition(ctx context.Context, p modelResourceModel, diags *diag.Diagnostics) client.Model {
|
||||
m := client.Model{
|
||||
Name: p.Name.ValueString(), Source: p.Source.ValueString(), Description: p.Description.ValueString(),
|
||||
Aliases: listOf(ctx, p.Aliases, diags), Fallbacks: listOf(ctx, p.Fallbacks, diags),
|
||||
RoutingProfile: p.RoutingProfile.ValueString(), ContextLength: p.ContextLength.ValueInt64(),
|
||||
MaxOutput: p.MaxOutput.ValueInt64(), Class: p.Class.ValueString(),
|
||||
Adapters: adapters(p.Adapters),
|
||||
}
|
||||
if c := p.Capabilities; c != nil {
|
||||
m.Capabilities = &client.Capabilities{Tools: c.Tools.ValueBool(), Vision: c.Vision.ValueBool(), Embeddings: c.Embeddings.ValueBool()}
|
||||
}
|
||||
if e := p.External; e != nil {
|
||||
m.External = &client.External{URL: e.URL.ValueString(), Model: e.Model.ValueString(), APIKeyFile: e.APIKeyFile.ValueString()}
|
||||
}
|
||||
for _, pr := range p.Profiles {
|
||||
m.Profiles = append(m.Profiles, client.Profile{
|
||||
Name: pr.Name.ValueString(), Engine: pr.Engine.ValueString(), GPUClass: pr.GPUClass.ValueString(),
|
||||
TP: pr.TP.ValueInt64(), PP: pr.PP.ValueInt64(), GPUsPerNode: pr.GPUsPerNode.ValueInt64(), Nodes: pr.Nodes.ValueInt64(),
|
||||
Args: listOf(ctx, pr.Args, diags), Env: mapOf(ctx, pr.Env, diags),
|
||||
MinMemoryGB: pr.MinMemoryGB.ValueInt64(), CPUCores: pr.CPUCores.ValueInt64(), MinDiskGB: pr.MinDiskGB.ValueInt64(),
|
||||
NodeSelector: mapOf(ctx, pr.NodeSelector, diags), Weight: pr.Weight.ValueFloat64(), Priority: pr.Priority.ValueInt64(),
|
||||
StartupTimeout: duration(pr.StartupTimeout, "startup_timeout", diags),
|
||||
})
|
||||
}
|
||||
m.Scaling = client.Scaling(scaleRequest(p.Scaling, diags))
|
||||
return m
|
||||
}
|
||||
|
||||
func adapters(in []adapterModel) []client.Adapter {
|
||||
var out []client.Adapter
|
||||
for _, a := range in {
|
||||
out = append(out, client.Adapter{Name: a.Name.ValueString(), Source: a.Source.ValueString(), Scale: a.Scale.ValueFloat64()})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// scaleRequest is the scaling a plan asks for. What is unknown is left for
|
||||
// the cluster to fill in: a max not written follows min.
|
||||
func scaleRequest(s *scalingModel, diags *diag.Diagnostics) client.ScaleRequest {
|
||||
if s == nil {
|
||||
return client.ScaleRequest{}
|
||||
}
|
||||
return client.ScaleRequest{Min: s.Min.ValueInt64(), Max: s.Max.ValueInt64(),
|
||||
IdleTimeout: duration(s.IdleTimeout, "scaling.idle_timeout", diags)}
|
||||
}
|
||||
|
||||
// refresh takes what can change in place from the cluster's copy. adopt is
|
||||
// set when reading: a scaling block the configuration left out is taken
|
||||
// from the cluster when the cluster has one, so a model scaled from a
|
||||
// terminal shows the difference.
|
||||
func refresh(m client.Model, st *modelResourceModel, adopt bool) {
|
||||
st.ID = types.StringValue(m.Name)
|
||||
switch {
|
||||
case st.Scaling != nil:
|
||||
st.Scaling.Min = types.Int64Value(m.Scaling.Min)
|
||||
st.Scaling.Max = types.Int64Value(m.Scaling.Max)
|
||||
st.Scaling.IdleTimeout = keepDuration(st.Scaling.IdleTimeout, m.Scaling.IdleTimeout)
|
||||
case adopt && m.Scaling != (client.Scaling{}):
|
||||
st.Scaling = &scalingModel{Min: types.Int64Value(m.Scaling.Min), Max: types.Int64Value(m.Scaling.Max),
|
||||
IdleTimeout: types.StringValue(m.Scaling.IdleTimeout.String())}
|
||||
}
|
||||
st.Class = strOrNull(m.Class)
|
||||
if m.Owner != st.OwnerID.ValueString() {
|
||||
st.Owner = strOrNull(m.Owner)
|
||||
}
|
||||
st.OwnerID = strOrNull(m.Owner)
|
||||
switch {
|
||||
case len(m.Adapters) > 0:
|
||||
st.Adapters = nil
|
||||
for _, a := range m.Adapters {
|
||||
st.Adapters = append(st.Adapters, adapterModel{Name: types.StringValue(a.Name),
|
||||
Source: types.StringValue(a.Source), Scale: floatOrNull(a.Scale)})
|
||||
}
|
||||
case st.Adapters != nil:
|
||||
st.Adapters = []adapterModel{}
|
||||
}
|
||||
}
|
||||
|
||||
func (r *modelResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
|
||||
var plan modelResourceModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
m := definition(ctx, plan, &resp.Diagnostics)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
name := m.Name
|
||||
if err := r.c.AddModel(ctx, m); err != nil {
|
||||
resp.Diagnostics.AddError("Adding the model "+name, err.Error())
|
||||
return
|
||||
}
|
||||
plan.OwnerID = types.StringNull()
|
||||
if owner := plan.Owner.ValueString(); owner != "" {
|
||||
id, err := r.c.SetOwner(ctx, name, owner)
|
||||
if err != nil {
|
||||
// Taken out again rather than left half made: Terraform would
|
||||
// not know it exists, and the next apply would fail on the name.
|
||||
_ = r.c.RemoveModel(ctx, name)
|
||||
resp.Diagnostics.AddError("Setting the owner of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
plan.OwnerID = strOrNull(id)
|
||||
}
|
||||
got, err := r.c.Model(ctx, name)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the model "+name+" back", err.Error())
|
||||
return
|
||||
}
|
||||
refresh(got, &plan, false)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *modelResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
|
||||
var st modelResourceModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
m, err := r.c.Model(ctx, st.Name.ValueString())
|
||||
if client.IsNotFound(err) {
|
||||
resp.State.RemoveResource(ctx)
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the model "+st.Name.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
refresh(m, &st, true)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
}
|
||||
|
||||
func (r *modelResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
|
||||
var plan, st modelResourceModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
name := plan.Name.ValueString()
|
||||
|
||||
want, had := scaleRequest(plan.Scaling, &resp.Diagnostics), scaleRequest(st.Scaling, &resp.Diagnostics)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if want != had || (plan.Scaling == nil) != (st.Scaling == nil) {
|
||||
if err := r.c.Scale(ctx, name, want); err != nil {
|
||||
resp.Diagnostics.AddError("Scaling "+name, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
if !plan.Class.Equal(st.Class) {
|
||||
if err := r.c.SetClass(ctx, name, plan.Class.ValueString()); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the class of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
plan.OwnerID = st.OwnerID
|
||||
if !plan.Owner.Equal(st.Owner) {
|
||||
id, err := r.c.SetOwner(ctx, name, plan.Owner.ValueString())
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Setting the owner of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
plan.OwnerID = strOrNull(id)
|
||||
}
|
||||
if !sameAdapters(plan.Adapters, st.Adapters) {
|
||||
if err := r.c.SetAdapters(ctx, name, adapters(plan.Adapters)); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the adapters of "+name, err.Error())
|
||||
return
|
||||
}
|
||||
}
|
||||
got, err := r.c.Model(ctx, name)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the model "+name+" back", err.Error())
|
||||
return
|
||||
}
|
||||
refresh(got, &plan, false)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func sameAdapters(a, b []adapterModel) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for i := range a {
|
||||
if !a[i].Name.Equal(b[i].Name) || !a[i].Source.Equal(b[i].Source) || a[i].Scale.ValueFloat64() != b[i].Scale.ValueFloat64() {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func (r *modelResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
|
||||
var st modelResourceModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
name := st.Name.ValueString()
|
||||
if err := r.c.RemoveModel(ctx, name); err != nil {
|
||||
// The API refuses removing a model that is not there with the same
|
||||
// status as one it may not remove; asking tells them apart, and one
|
||||
// already gone is what deleting wanted.
|
||||
if _, gerr := r.c.Model(ctx, name); client.IsNotFound(gerr) {
|
||||
return
|
||||
}
|
||||
resp.Diagnostics.AddError("Removing the model "+name, err.Error())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,180 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/path"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/client"
|
||||
)
|
||||
|
||||
// The quota resource is one scope's capacity quota: what an organization,
|
||||
// team or project is guaranteed of the cluster and the most it may hold.
|
||||
//
|
||||
// The scope is kept as written — team:Acme/ml — and beside it, as id, the
|
||||
// scope the cluster resolved it to, team:ID. The quota is found again by the
|
||||
// ID, so renaming the team does not lose it.
|
||||
|
||||
type quotaResource struct{ c *client.Client }
|
||||
|
||||
func newQuotaResource() resource.Resource { return "aResource{} }
|
||||
|
||||
type quotaModel struct {
|
||||
ID types.String `tfsdk:"id"`
|
||||
Scope types.String `tfsdk:"scope"`
|
||||
Name types.String `tfsdk:"name"`
|
||||
Guarantee *resourcesModel `tfsdk:"guarantee"`
|
||||
Limit *resourcesModel `tfsdk:"limit"`
|
||||
}
|
||||
|
||||
type resourcesModel struct {
|
||||
GPUs types.Int64 `tfsdk:"gpus"`
|
||||
CPUCores types.Int64 `tfsdk:"cpu_cores"`
|
||||
MemoryGB types.Int64 `tfsdk:"memory_gb"`
|
||||
}
|
||||
|
||||
func (r *quotaResource) Metadata(_ context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
|
||||
resp.TypeName = req.ProviderTypeName + "_quota"
|
||||
}
|
||||
|
||||
func (r *quotaResource) Schema(_ context.Context, _ resource.SchemaRequest, resp *resource.SchemaResponse) {
|
||||
amount := func(desc string) schema.SingleNestedAttribute {
|
||||
return schema.SingleNestedAttribute{Optional: true, Description: desc,
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"gpus": schema.Int64Attribute{Optional: true, Description: "GPUs; absent is not governed."},
|
||||
"cpu_cores": schema.Int64Attribute{Optional: true, Description: "CPU cores of replicas without a GPU; absent is not governed."},
|
||||
"memory_gb": schema.Int64Attribute{Optional: true, Description: "Host memory of those replicas, in GB; absent is not governed."},
|
||||
}}
|
||||
}
|
||||
resp.Schema = schema.Schema{
|
||||
Description: "A capacity quota on an organization, team or project. At least one of guarantee and limit must be set.",
|
||||
Attributes: map[string]schema.Attribute{
|
||||
"id": schema.StringAttribute{Computed: true,
|
||||
Description: "The scope as the cluster records it, team:ID. Import by it.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
|
||||
"scope": schema.StringAttribute{Required: true,
|
||||
Description: "org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT, or the ID after the colon.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.RequiresReplace()}},
|
||||
"name": schema.StringAttribute{Computed: true, Description: "The scope's path, Acme / ml.",
|
||||
PlanModifiers: []planmodifier.String{stringplanmodifier.UseStateForUnknown()}},
|
||||
"guarantee": amount("What is held for the scope: while it holds less, what others borrowed beyond their own guarantees is taken back for it."),
|
||||
"limit": amount("The most the scope may hold."),
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (r *quotaResource) Configure(_ context.Context, req resource.ConfigureRequest, _ *resource.ConfigureResponse) {
|
||||
if c, ok := clientFrom(req.ProviderData); ok {
|
||||
r.c = c
|
||||
}
|
||||
}
|
||||
|
||||
func amountOf(m *resourcesModel) client.Resources {
|
||||
if m == nil {
|
||||
return client.Resources{}
|
||||
}
|
||||
return client.Resources{GPUs: ptrOf(m.GPUs), CPUCores: ptrOf(m.CPUCores), MemoryGB: ptrOf(m.MemoryGB)}
|
||||
}
|
||||
|
||||
// amountModel is the cluster's amount for the state: a block left out stays
|
||||
// out while the cluster governs nothing in it.
|
||||
func amountModel(had *resourcesModel, r client.Resources) *resourcesModel {
|
||||
if had == nil && r.GPUs == nil && r.CPUCores == nil && r.MemoryGB == nil {
|
||||
return nil
|
||||
}
|
||||
return &resourcesModel{GPUs: ptrOrNull(r.GPUs), CPUCores: ptrOrNull(r.CPUCores), MemoryGB: ptrOrNull(r.MemoryGB)}
|
||||
}
|
||||
|
||||
func (r *quotaResource) put(ctx context.Context, plan *quotaModel) error {
|
||||
id, err := r.c.SetQuota(ctx, client.QuotaRequest{Scope: plan.Scope.ValueString(),
|
||||
Guarantee: amountOf(plan.Guarantee), Limit: amountOf(plan.Limit)})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
plan.ID = types.StringValue(id)
|
||||
plan.Name = types.StringNull()
|
||||
qs, err := r.c.Quotas(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, q := range qs {
|
||||
if q.Scope == id {
|
||||
plan.Name = strOrNull(q.Name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *quotaResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
|
||||
var plan quotaModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if err := r.put(ctx, &plan); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the quota on "+plan.Scope.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *quotaResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
|
||||
var st quotaModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
qs, err := r.c.Quotas(ctx)
|
||||
if err != nil {
|
||||
resp.Diagnostics.AddError("Reading the quotas", err.Error())
|
||||
return
|
||||
}
|
||||
for _, q := range qs {
|
||||
if q.Scope != st.ID.ValueString() {
|
||||
continue
|
||||
}
|
||||
if st.Scope.IsNull() {
|
||||
// Imported: the ID is the only name there is.
|
||||
st.Scope = types.StringValue(q.Scope)
|
||||
}
|
||||
st.Name = strOrNull(q.Name)
|
||||
st.Guarantee = amountModel(st.Guarantee, q.Guarantee)
|
||||
st.Limit = amountModel(st.Limit, q.Limit)
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &st)...)
|
||||
return
|
||||
}
|
||||
resp.State.RemoveResource(ctx)
|
||||
}
|
||||
|
||||
func (r *quotaResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
|
||||
var plan quotaModel
|
||||
resp.Diagnostics.Append(req.Plan.Get(ctx, &plan)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if err := r.put(ctx, &plan); err != nil {
|
||||
resp.Diagnostics.AddError("Setting the quota on "+plan.Scope.ValueString(), err.Error())
|
||||
return
|
||||
}
|
||||
resp.Diagnostics.Append(resp.State.Set(ctx, &plan)...)
|
||||
}
|
||||
|
||||
func (r *quotaResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
|
||||
var st quotaModel
|
||||
resp.Diagnostics.Append(req.State.Get(ctx, &st)...)
|
||||
if resp.Diagnostics.HasError() {
|
||||
return
|
||||
}
|
||||
if err := r.c.ClearQuota(ctx, st.ID.ValueString()); err != nil {
|
||||
resp.Diagnostics.AddError("Removing the quota on "+st.Scope.ValueString(), err.Error())
|
||||
}
|
||||
}
|
||||
|
||||
func (r *quotaResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) {
|
||||
resource.ImportStatePassthroughID(ctx, path.Root("id"), req, resp)
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
package provider
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/diag"
|
||||
"github.com/hashicorp/terraform-plugin-framework/types"
|
||||
)
|
||||
|
||||
// Conversions between Terraform's values and the client's.
|
||||
|
||||
// strOrNull is null for empty: the API leaves an unset field out, and a
|
||||
// configuration that does not set it holds null, not "".
|
||||
func strOrNull(s string) types.String {
|
||||
if s == "" {
|
||||
return types.StringNull()
|
||||
}
|
||||
return types.StringValue(s)
|
||||
}
|
||||
|
||||
func intOrNull(n int64) types.Int64 {
|
||||
if n == 0 {
|
||||
return types.Int64Null()
|
||||
}
|
||||
return types.Int64Value(n)
|
||||
}
|
||||
|
||||
func floatOrNull(f float64) types.Float64 {
|
||||
if f == 0 {
|
||||
return types.Float64Null()
|
||||
}
|
||||
return types.Float64Value(f)
|
||||
}
|
||||
|
||||
func ptrOrNull(p *int64) types.Int64 {
|
||||
if p == nil {
|
||||
return types.Int64Null()
|
||||
}
|
||||
return types.Int64Value(*p)
|
||||
}
|
||||
|
||||
func ptrOf(v types.Int64) *int64 {
|
||||
if v.IsNull() || v.IsUnknown() {
|
||||
return nil
|
||||
}
|
||||
n := v.ValueInt64()
|
||||
return &n
|
||||
}
|
||||
|
||||
func listOf(ctx context.Context, l types.List, diags *diag.Diagnostics) []string {
|
||||
if l.IsNull() || l.IsUnknown() {
|
||||
return nil
|
||||
}
|
||||
var out []string
|
||||
diags.Append(l.ElementsAs(ctx, &out, false)...)
|
||||
return out
|
||||
}
|
||||
|
||||
func mapOf(ctx context.Context, m types.Map, diags *diag.Diagnostics) map[string]string {
|
||||
if m.IsNull() || m.IsUnknown() {
|
||||
return nil
|
||||
}
|
||||
out := map[string]string{}
|
||||
diags.Append(m.ElementsAs(ctx, &out, false)...)
|
||||
return out
|
||||
}
|
||||
|
||||
// duration reads a duration written as Go writes them, "30m" or "1h30m".
|
||||
func duration(v types.String, what string, diags *diag.Diagnostics) time.Duration {
|
||||
if v.IsNull() || v.IsUnknown() || v.ValueString() == "" {
|
||||
return 0
|
||||
}
|
||||
d, err := time.ParseDuration(v.ValueString())
|
||||
if err != nil || d < 0 {
|
||||
diags.AddError("Not a duration", fmt.Sprintf("%s is %q: write a duration like 30m or 1h30m", what, v.ValueString()))
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
// keepDuration is the API's duration as the configuration wrote it when they
|
||||
// mean the same: "30m" and "30m0s" are one value, and showing a difference
|
||||
// between them on every plan would be noise.
|
||||
func keepDuration(had types.String, d time.Duration) types.String {
|
||||
if !had.IsNull() && !had.IsUnknown() {
|
||||
if was, err := time.ParseDuration(had.ValueString()); err == nil && was == d {
|
||||
return had
|
||||
}
|
||||
}
|
||||
return types.StringValue(d.String())
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
// Command terraform-provider-llmgw is the llmgw Terraform provider: models,
|
||||
// capacity quotas and guardrail attachments managed through a control
|
||||
// plane's admin API.
|
||||
//
|
||||
// A module of its own, so the gateway's binary does not carry Terraform's
|
||||
// dependencies.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"log"
|
||||
|
||||
"github.com/hashicorp/terraform-plugin-framework/providerserver"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/terraform-provider-llmgw/internal/provider"
|
||||
)
|
||||
|
||||
// version is set when a release is built: -ldflags "-X main.version=v0.1.0".
|
||||
var version = "dev"
|
||||
|
||||
// Address is the provider's source address, as a configuration's
|
||||
// required_providers names it.
|
||||
const Address = "gitea.mynodes.xyz/ilker/llmgw"
|
||||
|
||||
func main() {
|
||||
var debug bool
|
||||
flag.BoolVar(&debug, "debug", false, "run under a debugger, printing what TF_REATTACH_PROVIDERS needs")
|
||||
flag.Parse()
|
||||
|
||||
err := providerserver.Serve(context.Background(), provider.New(version), providerserver.ServeOpts{
|
||||
Address: Address,
|
||||
Debug: debug,
|
||||
})
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user