The admin API described in OpenAPI 3.1: generated from a route table by docgen, served at /api/v1/openapi.json
adminapi.Routes lists every endpoint with its query parameters and Go values
of the bodies it reads and answers with; internal/docgen/openapi turns that
into docs/admin-api.openapi.json by reflection, the way encoding/json encodes
the types, with the fields' comments as descriptions. A test fails when an
endpoint is registered and not in the table, another when the committed
document is stale. The binary embeds it and serves it with the token.
Handlers that wrote map literals and decoded anonymous structs now use named
types, so the document can see them. GET /api/v1/models/{name} answers with a
model's whole definition as it is in force, which a tool managing it needs.
docs/admin-api.md: reaching the API, the token, what a request must carry.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
30f7f3474a
commit
a95c519bc5
17 files changed
+7643
-62
No files matched your search
@@ -1,4 +1,5 @@
|
||||
// Package llmgw holds what the binary carries besides code: its manual.
|
||||
// Package llmgw holds what the binary carries besides code: its manual, and
|
||||
// the description of its admin API.
|
||||
package llmgw
|
||||
|
||||
import "embed"
|
||||
@@ -8,3 +9,10 @@ import "embed"
|
||||
//
|
||||
//go:embed README.md CHANGELOG.md docs/*.md
|
||||
var Docs embed.FS
|
||||
|
||||
// AdminOpenAPI is the admin API described in OpenAPI 3.1, generated from its
|
||||
// route table by `go generate` in internal/docgen and served at
|
||||
// /api/v1/openapi.json, so a client can be pointed at the server it talks to.
|
||||
//
|
||||
//go:embed docs/admin-api.openapi.json
|
||||
var AdminOpenAPI []byte
|
||||
@@ -0,0 +1,160 @@
|
||||
# The admin API
|
||||
|
||||
Written for whoever automates a cluster rather than typing at it: how to reach
|
||||
the control plane's management API, what it asks of a request, and where its
|
||||
description is.
|
||||
|
||||
`llmgw ctl` and the console are clients of this API and nothing more, so
|
||||
anything they do can be done with `curl`, a generated client or the
|
||||
[Terraform provider](../terraform-provider-llmgw/README.md). It adds and
|
||||
removes models, scales them, cordons and upgrades nodes, sets capacity
|
||||
quotas, attaches guardrail policies, and reads the fleet, usage and the audit
|
||||
log.
|
||||
|
||||
## Reaching it
|
||||
|
||||
The API is under `/api/v1` on the admin listener, `admin_listen` in
|
||||
`/etc/llmgw/server.yaml`: `127.0.0.1:9090` unless changed. It is on loopback
|
||||
on purpose. The same listener serves pprof when it is turned on, and the
|
||||
server refuses to start with it on any other address unless
|
||||
`admin_remote: true` — the token would cross the network in clear.
|
||||
|
||||
From another machine, tunnel to it rather than moving it:
|
||||
|
||||
```sh
|
||||
ssh -N -L 9090:127.0.0.1:9090 admin@control-plane &
|
||||
curl -s -H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" http://127.0.0.1:9090/api/v1/summary
|
||||
```
|
||||
|
||||
With `admin_remote: true` and `admin_listen: 10.0.0.5:9090`, call that address
|
||||
directly — on a network you trust, or behind a TLS proxy that keeps the
|
||||
`Host` header an address.
|
||||
|
||||
## The token
|
||||
|
||||
Every path under `/api` needs the admin token as a bearer token:
|
||||
|
||||
```
|
||||
Authorization: Bearer 6f1c…
|
||||
```
|
||||
|
||||
The server writes it the first time it starts, 0600, to
|
||||
`/var/lib/llmgw/admin-token` under its systemd unit (`admin_token_file`
|
||||
moves it; a server run by hand keeps it in `secrets/admin-token` beside its
|
||||
configuration). `llmgw ctl` reads it from there, which is why it runs with
|
||||
sudo, or from `LLMGW_ADMIN_TOKEN`. To give it to automation:
|
||||
|
||||
```sh
|
||||
sudo cat /var/lib/llmgw/admin-token
|
||||
```
|
||||
|
||||
The token is the whole of the API's authorisation: whoever holds it can do
|
||||
everything the API does. Keep it where you keep other root credentials.
|
||||
|
||||
## What a request must look like
|
||||
|
||||
Besides the token, the listener refuses three things before a handler sees
|
||||
them, as defences against a browser being turned against it:
|
||||
|
||||
- **A name in `Host`.** The listener answers to an address or `localhost`,
|
||||
not a DNS name, so a page on some other site cannot reach it by rebinding
|
||||
a name to 127.0.0.1. Call it as `http://127.0.0.1:9090`, not
|
||||
`http://cp.example.com:9090`; behind a proxy, have the proxy send an
|
||||
address.
|
||||
- **A cross-site request.** One whose `Origin` is another host, or whose
|
||||
`Sec-Fetch-Site` says another site sent it.
|
||||
- **A change that is not JSON.** Every method but `GET` and `HEAD` must carry
|
||||
`Content-Type: application/json`, even a `DELETE` or a `POST` with no body.
|
||||
A form post from a page cannot set that header.
|
||||
|
||||
A change is recorded in the audit log with who made it, as the request says
|
||||
in `X-Llmgw-Actor` (letters, digits and `._-@`, at most 64; `unknown` when
|
||||
absent). `llmgw ctl` sends the name of whoever ran it; automation should send
|
||||
its own, `terraform` or `ci@deploy`, so `llmgw ctl audit` can tell them apart.
|
||||
|
||||
A refusal is a 4xx or 5xx with `{"error": "…"}` saying why. 501 means this
|
||||
control plane lacks what the endpoint needs — a store, a mirror, guard
|
||||
policies — and says which.
|
||||
|
||||
## Some calls
|
||||
|
||||
```sh
|
||||
H=(-H "Authorization: Bearer $LLMGW_ADMIN_TOKEN" -H "Content-Type: application/json" -H "X-Llmgw-Actor: $USER")
|
||||
API=http://127.0.0.1:9090/api/v1
|
||||
|
||||
# The fleet in one answer, and every model.
|
||||
curl -s "${H[@]}" $API/summary
|
||||
curl -s "${H[@]}" $API/models
|
||||
|
||||
# Add a model, read back its definition with the defaults filled in, scale it.
|
||||
curl -s "${H[@]}" -X POST $API/models -d '{
|
||||
"name": "qwen-small",
|
||||
"source": "/srv/models/qwen2.5-1.5b-instruct-q4_k_m.gguf",
|
||||
"profiles": [{"name": "cpu", "engine": "llamacpp", "min_memory_gb": 4, "cpu_cores": 4}],
|
||||
"scaling": {"min": 1, "max": 2}
|
||||
}'
|
||||
curl -s "${H[@]}" $API/models/qwen-small
|
||||
curl -s "${H[@]}" -X POST $API/models/qwen-small/scale -d '{"min": 0, "max": 2, "idle_timeout": 1800000000000}'
|
||||
|
||||
# A team's capacity quota.
|
||||
curl -s "${H[@]}" -X PUT $API/quotas -d '{"scope": "team:Acme/ml", "limit": {"gpus": 8}}'
|
||||
|
||||
# Remove the model; a DELETE carries the content type too.
|
||||
curl -s "${H[@]}" -X DELETE $API/models/qwen-small
|
||||
```
|
||||
|
||||
Durations in bodies are integer nanoseconds, as Go encodes them: half an hour
|
||||
is `1800000000000`.
|
||||
|
||||
## The description
|
||||
|
||||
The API is described in OpenAPI 3.1:
|
||||
|
||||
- served by every control plane at `GET /api/v1/openapi.json` (with the
|
||||
token, like the rest), as built into that binary, so it describes the
|
||||
server you are talking to;
|
||||
- committed as [admin-api.openapi.json](admin-api.openapi.json) for reading
|
||||
without a cluster.
|
||||
|
||||
```sh
|
||||
curl -s "${H[@]}" $API/openapi.json > llmgw-admin.json
|
||||
```
|
||||
|
||||
It lists every endpoint, its path and query parameters, and the schema of
|
||||
each request and answer, with the meaning of each field. Load it into any
|
||||
OpenAPI tool — Swagger UI, Redoc, an editor — or generate a client from it:
|
||||
|
||||
```sh
|
||||
docker run --rm -v "$PWD:/w" openapitools/openapi-generator-cli generate \
|
||||
-i /w/llmgw-admin.json -g python -o /w/llmgw-admin-client
|
||||
```
|
||||
|
||||
### How it is kept true
|
||||
|
||||
The document is generated, not written. `internal/control/adminapi/routes.go`
|
||||
holds a table of every endpoint: its method and path, a summary, its query
|
||||
parameters, and Go values of the types it reads and answers with. The
|
||||
generator in `internal/docgen/openapi` reads those types by reflection, the
|
||||
way `encoding/json` encodes them, and takes each field's description from its
|
||||
comment in the source.
|
||||
|
||||
Two tests keep it from drifting. One registers the API on a recording mux and
|
||||
fails when an endpoint is served and not in the table, or in the table and not
|
||||
served. The other regenerates the document and fails when it differs from the
|
||||
committed one. After changing an endpoint or a type it carries:
|
||||
|
||||
```sh
|
||||
go generate ./internal/docgen
|
||||
```
|
||||
|
||||
## What it does not cover
|
||||
|
||||
- **Organizations, teams, projects, people and API keys.** These live in the
|
||||
store and are managed with `llmgw org`, `llmgw users` and `llmgw keys` on the
|
||||
control plane, or in the web console's `/admin` pages on the client port
|
||||
behind a session. The admin API reads the tree only where a quota or a
|
||||
policy names a part of it.
|
||||
- **The client-facing API** (`/v1/chat/completions` and the rest), which is
|
||||
OpenAI's and Anthropic's, on `listen`, with API keys.
|
||||
- **The rest of the admin listener**: `/metrics`, `/health`, the dashboard
|
||||
pages and `/debug/pprof`, which are not under `/api`.
|
||||
File diff suppressed because it is too large.
Load diff
+2
-1
@@ -252,7 +252,8 @@ knowing before you edit it:
|
||||
`LLMGW_ADMIN_TOKEN` gives it from elsewhere. pprof, whose heap holds the API
|
||||
keys requests presented, is off unless `debug_pprof: true`, and then behind
|
||||
the token too. To see the dashboard from elsewhere use `/admin` on the
|
||||
client port — do not move this listener.
|
||||
client port — do not move this listener. The API itself is described in
|
||||
[admin-api.md](admin-api.md).
|
||||
- `default_rpm`, `default_tpm` and `default_price` are what make limits and
|
||||
budgets mean anything. Without a price every request costs zero and a budget
|
||||
never binds; without the defaults, a project created without limits is
|
||||
|
||||
@@ -75,13 +75,26 @@ func (a *API) SetRevoker(f func(ctx context.Context, nodeID string, at time.Time
|
||||
a.revoke = f
|
||||
}
|
||||
|
||||
// Mux is where the endpoints are mounted. An *http.ServeMux is one; a test
|
||||
// that records the patterns is another, and is how the route table is checked
|
||||
// against what is actually served.
|
||||
type Mux interface {
|
||||
HandleFunc(pattern string, handler func(http.ResponseWriter, *http.Request))
|
||||
}
|
||||
|
||||
// Register mounts the endpoints on a mux under /api/v1.
|
||||
func (a *API) Register(mux *http.ServeMux) {
|
||||
//
|
||||
// Every pattern registered here, or by the register helpers it calls, must
|
||||
// also be in Routes: that table is what the OpenAPI document is generated
|
||||
// from, and a test fails when the two disagree.
|
||||
func (a *API) Register(mux Mux) {
|
||||
mux.HandleFunc("GET /api/v1/summary", a.summary)
|
||||
// Adding a model is a write, and the only one this API has. It lives here
|
||||
// rather than on the client-facing listener for the same reason the rest of
|
||||
// it does: reaching this port already means holding the control plane.
|
||||
mux.HandleFunc("GET /api/v1/openapi.json", serveOpenAPI)
|
||||
// Changes to the catalog live here rather than on the client-facing
|
||||
// listener for the same reason the rest of this does: reaching this port
|
||||
// already means holding the control plane.
|
||||
mux.HandleFunc("POST /api/v1/models", a.addModel)
|
||||
mux.HandleFunc("GET /api/v1/models/{name}", a.getModel)
|
||||
mux.HandleFunc("DELETE /api/v1/models/{name}", a.removeModel)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/scale", a.scaleModel)
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/resources", a.setResources)
|
||||
@@ -97,9 +110,7 @@ func (a *API) Register(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane has no store")
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Class string `json:"class"`
|
||||
}
|
||||
var req ClassRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
@@ -109,7 +120,7 @@ func (a *API) Register(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "class": m.EffectiveClass()})
|
||||
writeJSON(w, http.StatusOK, ClassView{Model: m.Name, Class: m.EffectiveClass()})
|
||||
})
|
||||
mux.HandleFunc("PUT /api/v1/models/{name}/capacity", a.putCapacity)
|
||||
mux.HandleFunc("GET /api/v1/capacity", a.listCapacity)
|
||||
@@ -953,7 +964,25 @@ func (a *API) addModel(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"added": m.Name})
|
||||
writeJSON(w, http.StatusOK, ModelAdded{Added: m.Name})
|
||||
}
|
||||
|
||||
// getModel answers with one model's whole definition as it is in force: what
|
||||
// was added, with the defaults the catalog filled in. The listing shows a
|
||||
// model's state and only the names of its profiles; this is what a tool
|
||||
// managing the definition — Terraform, say — compares against what it wants.
|
||||
//
|
||||
// By its name only, not an alias: an alias answering would let a tool believe
|
||||
// a model it manages exists when it is another one.
|
||||
func (a *API) getModel(w http.ResponseWriter, r *http.Request) {
|
||||
name := r.PathValue("name")
|
||||
for _, m := range a.ctl.Catalog().Models() {
|
||||
if m.Name == name {
|
||||
writeJSON(w, http.StatusOK, m)
|
||||
return
|
||||
}
|
||||
}
|
||||
writeError(w, http.StatusNotFound, "no model named "+name+" is in the catalog")
|
||||
}
|
||||
|
||||
// ScaleRequest is how many replicas a model should run.
|
||||
@@ -1039,7 +1068,7 @@ func (a *API) removeModel(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": name})
|
||||
writeJSON(w, http.StatusOK, Removed{Removed: name})
|
||||
}
|
||||
|
||||
func (a *API) summary(w http.ResponseWriter, _ *http.Request) {
|
||||
@@ -1137,15 +1166,13 @@ func (a *API) removeNode(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
inv.Remove(id)
|
||||
a.ctl.Reconciler().Notify()
|
||||
writeJSON(w, http.StatusOK, map[string]string{"id": id, "removed": "true"})
|
||||
writeJSON(w, http.StatusOK, NodeRemoved{ID: id, Removed: "true"})
|
||||
}
|
||||
|
||||
func (a *API) setNodeState(w http.ResponseWriter, r *http.Request) {
|
||||
id := r.PathValue("id")
|
||||
|
||||
var body struct {
|
||||
State string `json:"state"`
|
||||
}
|
||||
var body NodeStateRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
@@ -1173,7 +1200,7 @@ func (a *API) setNodeState(w http.ResponseWriter, r *http.Request) {
|
||||
// is usually about to do something to it.
|
||||
a.ctl.Reconciler().Notify()
|
||||
|
||||
writeJSON(w, http.StatusOK, map[string]string{"id": id, "state": st.String()})
|
||||
writeJSON(w, http.StatusOK, NodeStateView{ID: id, State: st.String()})
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
@@ -1183,7 +1210,7 @@ func writeJSON(w http.ResponseWriter, code int, v any) {
|
||||
}
|
||||
|
||||
func writeError(w http.ResponseWriter, code int, msg string) {
|
||||
writeJSON(w, code, map[string]string{"error": msg})
|
||||
writeJSON(w, code, ErrorView{Error: msg})
|
||||
}
|
||||
|
||||
// profileChanged reports whether a replica is running a profile the catalog no
|
||||
@@ -1384,7 +1411,7 @@ func (a *API) retryModel(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
was := a.ctl.Reconciler().Retry(name)
|
||||
writeJSON(w, http.StatusOK, map[string]bool{"was_parked": was})
|
||||
writeJSON(w, http.StatusOK, RetryView{WasParked: was})
|
||||
}
|
||||
|
||||
// requestFinder looks a request up by the id it was given. Asked of the usage
|
||||
@@ -1477,7 +1504,7 @@ func (a *API) upgradeNode(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadGateway, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"node": r.PathValue("id"), "version": req.Version})
|
||||
writeJSON(w, http.StatusOK, UpgradeView{Node: r.PathValue("id"), Version: req.Version})
|
||||
}
|
||||
|
||||
// SetServe supplies what serves a model from the registry.
|
||||
@@ -1577,7 +1604,7 @@ func (a *API) rollbackModel(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"result": msg})
|
||||
writeJSON(w, http.StatusOK, Result{Result: msg})
|
||||
}
|
||||
|
||||
// OriginRequest names the repository a model's weights came from.
|
||||
|
||||
@@ -8,7 +8,13 @@ import (
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||
)
|
||||
|
||||
func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
// CanaryPercentRequest changes the share of a model's requests its canary
|
||||
// takes.
|
||||
type CanaryPercentRequest struct {
|
||||
Percent int `json:"percent"`
|
||||
}
|
||||
|
||||
func (a *API) registerCanary(mux Mux) {
|
||||
need := func(w http.ResponseWriter) bool {
|
||||
if a.serve == nil {
|
||||
writeError(w, http.StatusNotImplemented, "a canary needs a mirror and a store, and this control plane lacks one")
|
||||
@@ -45,9 +51,7 @@ func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
if !need(w) {
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Percent int `json:"percent"`
|
||||
}
|
||||
var req CanaryPercentRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
@@ -57,7 +61,7 @@ func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"result": msg})
|
||||
writeJSON(w, http.StatusOK, Result{Result: msg})
|
||||
})
|
||||
mux.HandleFunc("DELETE /api/v1/models/{name}/canary", func(w http.ResponseWriter, r *http.Request) {
|
||||
if !need(w) {
|
||||
@@ -68,7 +72,7 @@ func (a *API) registerCanary(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"result": msg})
|
||||
writeJSON(w, http.StatusOK, Result{Result: msg})
|
||||
})
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/promote", func(w http.ResponseWriter, r *http.Request) {
|
||||
if !need(w) {
|
||||
|
||||
@@ -34,10 +34,7 @@ type RailView struct {
|
||||
// sends traffic over the fabric for seconds a rail, so it is a POST and only
|
||||
// ever run when asked.
|
||||
func (a *API) fabricTest(w http.ResponseWriter, r *http.Request) {
|
||||
var body struct {
|
||||
A, B, Fabric string
|
||||
Seconds int
|
||||
}
|
||||
var body FabricTestRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
|
||||
@@ -66,7 +66,17 @@ func (a *API) guardTarget(ctx context.Context, target string) (guard.Attachment,
|
||||
return guard.Attachment{}, fmt.Errorf("%q: attach to org, team, project, key or user", kind)
|
||||
}
|
||||
|
||||
func (a *API) registerGuard(mux *http.ServeMux) {
|
||||
// GuardAttachmentRequest attaches a policy to a part of the tenancy tree, a
|
||||
// key or a person, or with DELETE takes it off again.
|
||||
type GuardAttachmentRequest struct {
|
||||
// Target is org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT (or the ID
|
||||
// after the colon), key:KEY-ID or user:USER-ID.
|
||||
Target string `json:"target"`
|
||||
// Policy is a policy's name in server.yaml's guard.policies.
|
||||
Policy string `json:"policy"`
|
||||
}
|
||||
|
||||
func (a *API) registerGuard(mux Mux) {
|
||||
mux.HandleFunc("GET /api/v1/guard", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.guard == nil {
|
||||
writeError(w, http.StatusNotImplemented, "no guard policies: define them under guard: in server.yaml")
|
||||
@@ -97,12 +107,7 @@ func (a *API) registerGuard(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusNotImplemented, "no guard policies: define them under guard: in server.yaml")
|
||||
return
|
||||
}
|
||||
var at struct {
|
||||
// Target is org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT
|
||||
// (or the ID after the colon), key:KEY-ID or user:USER-ID.
|
||||
Target string `json:"target"`
|
||||
Policy string `json:"policy"`
|
||||
}
|
||||
var at GuardAttachmentRequest
|
||||
if err := json.NewDecoder(r.Body).Decode(&at); err != nil {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
|
||||
@@ -50,7 +50,26 @@ type QuotaRequest struct {
|
||||
Limit store.Resources `json:"limit"`
|
||||
}
|
||||
|
||||
func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
// QuotaSet is the scope a quota was set on, as org:ID, team:ID or
|
||||
// project:ID whatever name it was asked by.
|
||||
type QuotaSet struct {
|
||||
Scope string `json:"scope"`
|
||||
}
|
||||
|
||||
// OwnerRequest sets the part of the tenancy tree a model's capacity counts
|
||||
// against: org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT, or their IDs.
|
||||
// Empty makes the model shared.
|
||||
type OwnerRequest struct {
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
// OwnerView is a model's owner now, as scope:ID.
|
||||
type OwnerView struct {
|
||||
Model string `json:"model"`
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
|
||||
func (a *API) registerQuotas(mux Mux) {
|
||||
mux.HandleFunc("GET /api/v1/quotas", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.quotas == nil {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane has no store")
|
||||
@@ -78,7 +97,7 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"scope": key})
|
||||
writeJSON(w, http.StatusOK, QuotaSet{Scope: key})
|
||||
})
|
||||
mux.HandleFunc("DELETE /api/v1/quotas/{scope}", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.quotas == nil {
|
||||
@@ -90,16 +109,14 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"removed": key})
|
||||
writeJSON(w, http.StatusOK, Removed{Removed: key})
|
||||
})
|
||||
mux.HandleFunc("POST /api/v1/models/{name}/owner", func(w http.ResponseWriter, r *http.Request) {
|
||||
if a.models == nil || a.quotas == nil {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane has no store")
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Owner string `json:"owner"`
|
||||
}
|
||||
var req OwnerRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil {
|
||||
writeError(w, http.StatusBadRequest, "the request is not valid JSON: "+err.Error())
|
||||
return
|
||||
@@ -118,7 +135,7 @@ func (a *API) registerQuotas(mux *http.ServeMux) {
|
||||
writeError(w, http.StatusBadRequest, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"model": m.Name, "owner": m.Owner})
|
||||
writeJSON(w, http.StatusOK, OwnerView{Model: m.Name, Owner: m.Owner})
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,84 @@
|
||||
package adminapi
|
||||
|
||||
// The small bodies the endpoints read and answer with. Named types rather
|
||||
// than maps and anonymous structs, so the OpenAPI document generated from
|
||||
// Routes describes exactly what a handler decodes and writes: a map literal
|
||||
// in a handler is a shape no generator can see.
|
||||
|
||||
// ErrorView is every refusal: a 4xx or 5xx status with what went wrong.
|
||||
type ErrorView struct {
|
||||
Error string `json:"error"`
|
||||
}
|
||||
|
||||
// Result is what a change that has no other answer says it did.
|
||||
type Result struct {
|
||||
Result string `json:"result"`
|
||||
}
|
||||
|
||||
// ModelAdded names the model added to the catalog.
|
||||
type ModelAdded struct {
|
||||
Added string `json:"added"`
|
||||
}
|
||||
|
||||
// Removed names what was taken away: a model, or a quota's scope.
|
||||
type Removed struct {
|
||||
Removed string `json:"removed"`
|
||||
}
|
||||
|
||||
// ClassRequest sets a model's capacity class: critical, standard or
|
||||
// opportunistic, or empty to go back to what its scaling implies.
|
||||
type ClassRequest struct {
|
||||
Class string `json:"class"`
|
||||
}
|
||||
|
||||
// ClassView is a model's capacity class now.
|
||||
type ClassView struct {
|
||||
Model string `json:"model"`
|
||||
Class string `json:"class"`
|
||||
}
|
||||
|
||||
// RetryView says whether the model had been parked, given up on after
|
||||
// failing to start, before it was retried.
|
||||
type RetryView struct {
|
||||
WasParked bool `json:"was_parked"`
|
||||
}
|
||||
|
||||
// NodeStateRequest cordons, drains or readies a node: ready (or uncordon),
|
||||
// cordoned (or cordon), draining (or drain).
|
||||
type NodeStateRequest struct {
|
||||
State string `json:"state"`
|
||||
}
|
||||
|
||||
// NodeStateView is a node's state after the change.
|
||||
type NodeStateView struct {
|
||||
ID string `json:"id"`
|
||||
State string `json:"state"`
|
||||
}
|
||||
|
||||
// NodeRemoved names the node forgotten. Removed is the string "true".
|
||||
type NodeRemoved struct {
|
||||
ID string `json:"id"`
|
||||
Removed string `json:"removed"`
|
||||
}
|
||||
|
||||
// UpgradeView is the release a node was told to run.
|
||||
type UpgradeView struct {
|
||||
Node string `json:"node"`
|
||||
Version string `json:"version"`
|
||||
}
|
||||
|
||||
// FabricTestRequest is a bandwidth test between two nodes.
|
||||
type FabricTestRequest struct {
|
||||
A string `json:"a"`
|
||||
B string `json:"b"`
|
||||
// Fabric is the declared fabric to test over; empty is the one the two
|
||||
// nodes share.
|
||||
Fabric string `json:"fabric,omitempty"`
|
||||
// Seconds each rail is measured for, from 1 to 30.
|
||||
Seconds int `json:"seconds,omitempty"`
|
||||
}
|
||||
|
||||
// ReplayRequest names the model a kept request is sent to again.
|
||||
type ReplayRequest struct {
|
||||
Model string `json:"model"`
|
||||
}
|
||||
@@ -60,9 +60,7 @@ func (a *API) replayRequest(w http.ResponseWriter, r *http.Request) {
|
||||
writeError(w, http.StatusNotImplemented, "this control plane does not route requests")
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Model string `json:"model"`
|
||||
}
|
||||
var req ReplayRequest
|
||||
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<12)).Decode(&req); err != nil || req.Model == "" {
|
||||
writeError(w, http.StatusBadRequest, "say which model: {\"model\": \"...\"}")
|
||||
return
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
package adminapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
llmgw "gitea.mynodes.xyz/ilker/llmgw"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/bench"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/fabrics"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/modelserve"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/nvlink"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/guard"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/notify"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
||||
)
|
||||
|
||||
// Route is one endpoint as the OpenAPI document describes it.
|
||||
//
|
||||
// A table written by hand beside Register rather than read out of the
|
||||
// handlers, because what a handler decodes and answers is not visible from
|
||||
// outside it. The table is checked against Register by a test, so an endpoint
|
||||
// cannot be added without being described; the bodies are Go values whose
|
||||
// types the generator reads, so a field added to one is in the document the
|
||||
// next time it is generated.
|
||||
type Route struct {
|
||||
Method string
|
||||
Path string
|
||||
// Summary is one line; Description, when there is more to say, the rest.
|
||||
Summary string
|
||||
Description string
|
||||
// Query is the query parameters the handler reads.
|
||||
Query []Param
|
||||
// Request is a value of the body's type; nil reads none.
|
||||
Request any
|
||||
// OptionalBody is set when the body may be left out.
|
||||
OptionalBody bool
|
||||
// Response is a value of the type a success answers with.
|
||||
Response any
|
||||
// Status is the success status; zero is 200.
|
||||
Status int
|
||||
// CSV is set when the answer is CSV unless JSON is asked for.
|
||||
CSV bool
|
||||
}
|
||||
|
||||
// Param is a query parameter.
|
||||
type Param struct {
|
||||
Name string
|
||||
Description string
|
||||
// Type is a JSON Schema type: string when empty, or integer, number,
|
||||
// boolean.
|
||||
Type string
|
||||
Required bool
|
||||
}
|
||||
|
||||
// PathParams describes the parameters in Routes' paths, by the segment
|
||||
// before them and their name.
|
||||
var PathParams = map[string]string{
|
||||
"models/{name}": "The model's name in the catalog.",
|
||||
"nodes/{id}": "The node's ID, as `llmgw ctl nodes` lists it.",
|
||||
"jobs/{id}": "The job's ID, as starting it answered.",
|
||||
"requests/{id}": "The request's ID: the x-request-id a client was answered with.",
|
||||
"fabrics/{id}": "The physical fabric's ID, as GET /api/v1/fabrics lists it.",
|
||||
"quotas/{scope}": "The scope: org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT, or their IDs after the colon.",
|
||||
}
|
||||
|
||||
// Routes is every endpoint Register mounts.
|
||||
var Routes = []Route{
|
||||
// The document itself.
|
||||
{Method: "GET", Path: "/api/v1/openapi.json", Summary: "This document",
|
||||
Description: "The admin API in OpenAPI 3.1, generated from the route table and built into the binary.",
|
||||
Response: map[string]any{}},
|
||||
|
||||
// The fleet.
|
||||
{Method: "GET", Path: "/api/v1/summary", Summary: "The whole fleet in one answer",
|
||||
Response: Summary{}},
|
||||
{Method: "GET", Path: "/api/v1/nodes", Summary: "Every node, with its cards, load and fabrics",
|
||||
Response: []NodeView{}},
|
||||
{Method: "POST", Path: "/api/v1/nodes/{id}/state", Summary: "Cordon, drain or ready a node",
|
||||
Request: NodeStateRequest{}, Response: NodeStateView{}},
|
||||
{Method: "POST", Path: "/api/v1/nodes/{id}/upgrade", Summary: "Have a node run a release",
|
||||
Request: UpgradeRequest{}, Response: UpgradeView{}},
|
||||
{Method: "POST", Path: "/api/v1/nodes/{id}/engines", Summary: "Choose an engine's build for a node",
|
||||
Description: "The node installs the build chosen. Build is auto, a build such as cu129, or none.",
|
||||
Request: NodeEngineRequest{}, Response: NodeEngineRequest{}},
|
||||
{Method: "DELETE", Path: "/api/v1/nodes/{id}", Summary: "Forget a node that has gone",
|
||||
Description: "Only a node that is lost, or unreachable with nothing on it; its certificate is revoked.",
|
||||
Response: NodeRemoved{}},
|
||||
{Method: "GET", Path: "/api/v1/nodes/{id}/logs", Summary: "The last lines of a node's logs",
|
||||
Query: []Param{
|
||||
{Name: "lines", Type: "integer", Description: "How many lines; 300 when absent, at most 2000."},
|
||||
{Name: "source", Description: "Which log: agent when absent, or an engine the node lists in sources."},
|
||||
},
|
||||
Response: LogsView{}},
|
||||
{Method: "GET", Path: "/api/v1/fabrics", Summary: "The physical fabrics and the names nodes give them",
|
||||
Response: fabrics.Report{}},
|
||||
{Method: "GET", Path: "/api/v1/fabrics/{id}/topology", Summary: "One fabric's map, with the cluster's adapters live on it",
|
||||
Response: TopologyView{}},
|
||||
{Method: "GET", Path: "/api/v1/nvlink", Summary: "The multi-node NVLink domains and what keeps nodes out of them",
|
||||
Response: nvlink.Report{}},
|
||||
{Method: "POST", Path: "/api/v1/fabric-test", Summary: "Measure bandwidth between two nodes, rail by rail",
|
||||
Request: FabricTestRequest{}, Response: FabricTestView{}},
|
||||
{Method: "GET", Path: "/api/v1/weights/orphans", Summary: "Weights on nodes that no model uses",
|
||||
Response: []OrphanView{}},
|
||||
{Method: "POST", Path: "/api/v1/weights/collect", Summary: "Delete weights that no model uses",
|
||||
Request: CollectRequest{}, OptionalBody: true, Response: []CollectResult{}},
|
||||
|
||||
// The catalog.
|
||||
{Method: "GET", Path: "/api/v1/models", Summary: "Every model, with its replicas and state",
|
||||
Response: []ModelView{}},
|
||||
{Method: "POST", Path: "/api/v1/models", Summary: "Add a model to the catalog",
|
||||
Description: "Kept in the store across restarts. Refused when the name is taken, a model in " +
|
||||
"configuration has it, or the definition would make the catalog invalid.",
|
||||
Request: catalog.Model{}, Response: ModelAdded{}},
|
||||
{Method: "GET", Path: "/api/v1/models/{name}", Summary: "One model's definition, as it is in force",
|
||||
Description: "With the defaults the catalog filled in. By its name only, not an alias.",
|
||||
Response: catalog.Model{}},
|
||||
{Method: "DELETE", Path: "/api/v1/models/{name}", Summary: "Remove a model added at runtime",
|
||||
Description: "A model declared in configuration is refused: it would come back on the next restart.",
|
||||
Response: Removed{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/scale", Summary: "Set how many replicas a model runs",
|
||||
Request: ScaleRequest{}, Response: ScaleResponse{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/resources", Summary: "Set what a model's profiles reserve of a node",
|
||||
Request: ResourcesRequest{}, Response: catalog.Model{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/class", Summary: "Set a model's capacity class",
|
||||
Request: ClassRequest{}, Response: ClassView{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/owner", Summary: "Set the part of the tenancy tree a model's capacity counts against",
|
||||
Request: OwnerRequest{}, Response: OwnerView{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/adapters", Summary: "Replace a model's LoRA adapters",
|
||||
Description: "Each new file is read first, to check it is an adapter for the model's architecture.",
|
||||
Request: AdaptersRequest{}, Response: AdaptersRequest{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/retry", Summary: "Try a parked model again",
|
||||
Response: RetryView{}},
|
||||
{Method: "GET", Path: "/api/v1/models/{name}/target", Summary: "How many replicas a throughput needs, and whether they fit",
|
||||
Query: []Param{{Name: "tokens_per_sec", Type: "number", Required: true,
|
||||
Description: "The output tokens a second wanted of the model."}},
|
||||
Response: TargetView{}},
|
||||
{Method: "GET", Path: "/api/v1/models/{name}/headroom", Summary: "How many more replicas of a model fit now, and where",
|
||||
Query: []Param{{Name: "limit", Type: "integer",
|
||||
Description: "The most to try placing; 16 when absent or past 64."}},
|
||||
Response: HeadroomView{}},
|
||||
{Method: "PUT", Path: "/api/v1/models/{name}/capacity", Summary: "Record what a replica was measured to serve",
|
||||
Description: "Replaces an earlier measurement of the same model on the same profile.",
|
||||
Request: bench.Capacity{}, Response: bench.Capacity{}},
|
||||
{Method: "GET", Path: "/api/v1/capacity", Summary: "Every recorded measurement",
|
||||
Response: []CapacityView{}},
|
||||
{Method: "GET", Path: "/api/v1/replicas", Summary: "Every replica, with its shards",
|
||||
Query: []Param{{Name: "model", Description: "Only this model's replicas."}},
|
||||
Response: []ReplicaView{}},
|
||||
{Method: "GET", Path: "/api/v1/explain-placement", Summary: "Why a model could not be placed",
|
||||
Query: []Param{{Name: "model", Required: true, Description: "The model's name or an alias."}},
|
||||
Response: ExplainView{}},
|
||||
{Method: "POST", Path: "/api/v1/restart", Summary: "Restart replicas, every replica of some models, or all",
|
||||
Request: RestartRequest{}, Response: RestartResult{}, Status: http.StatusAccepted},
|
||||
|
||||
// Serving from the registry, and newer weights.
|
||||
{Method: "GET", Path: "/api/v1/plan", Summary: "How a registry repository would be served here",
|
||||
Query: []Param{
|
||||
{Name: "repo", Required: true, Description: "The repository, owner/name."},
|
||||
{Name: "format", Description: "gguf or safetensors; gguf when absent."},
|
||||
{Name: "context", Type: "integer", Description: "The context length to plan for, in tokens."},
|
||||
},
|
||||
Response: modelserve.Plan{}},
|
||||
{Method: "POST", Path: "/api/v1/models/serve", Summary: "Fetch a registry repository and serve it",
|
||||
Request: modelserve.Request{}, Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
{Method: "GET", Path: "/api/v1/jobs", Summary: "Every job since the control plane started",
|
||||
Response: []modelserve.Job{}},
|
||||
{Method: "GET", Path: "/api/v1/jobs/{id}", Summary: "One job",
|
||||
Response: modelserve.Job{}},
|
||||
{Method: "GET", Path: "/api/v1/updates", Summary: "Newer weights the registry has for models",
|
||||
Response: []modelserve.UpdateInfo{}},
|
||||
{Method: "POST", Path: "/api/v1/updates/check", Summary: "Ask the registry now, about one model or every one",
|
||||
Request: CheckRequest{}, OptionalBody: true, Response: []modelserve.UpdateInfo{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/update", Summary: "Move a model to its repository's newer weights",
|
||||
Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/rollback", Summary: "Go back to the weights the last update replaced",
|
||||
Response: Result{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/origin", Summary: "Record the repository a model's weights came from",
|
||||
Request: OriginRequest{}, Response: OriginView{}},
|
||||
{Method: "GET", Path: "/api/v1/canaries", Summary: "Every canary, and how its two sides serve",
|
||||
Response: []modelserve.CanaryView{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/canary", Summary: "Start a canary of a model's newer weights",
|
||||
Request: catalog.Canary{}, Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
{Method: "PUT", Path: "/api/v1/models/{name}/canary", Summary: "Change the share of requests a canary takes",
|
||||
Request: CanaryPercentRequest{}, Response: Result{}},
|
||||
{Method: "DELETE", Path: "/api/v1/models/{name}/canary", Summary: "Take a canary out: its share first, then the candidate",
|
||||
Response: Result{}},
|
||||
{Method: "POST", Path: "/api/v1/models/{name}/promote", Summary: "Make a canary's weights the model's",
|
||||
Response: modelserve.Job{}, Status: http.StatusAccepted},
|
||||
|
||||
// Tenancy: quotas and guardrails.
|
||||
{Method: "GET", Path: "/api/v1/quotas", Summary: "Every capacity quota, with what its scope holds now",
|
||||
Response: []QuotaView{}},
|
||||
{Method: "PUT", Path: "/api/v1/quotas", Summary: "Set a scope's capacity quota",
|
||||
Description: "Replaces the scope's quota whole. The scope may be named by path or by ID; the answer names it by ID.",
|
||||
Request: QuotaRequest{}, Response: QuotaSet{}},
|
||||
{Method: "DELETE", Path: "/api/v1/quotas/{scope}", Summary: "Remove a scope's capacity quota",
|
||||
Response: Removed{}},
|
||||
{Method: "GET", Path: "/api/v1/guard", Summary: "The guardrail policies and where they are attached",
|
||||
Response: GuardView{}},
|
||||
{Method: "POST", Path: "/api/v1/guard/attachments", Summary: "Attach a guardrail policy",
|
||||
Request: GuardAttachmentRequest{}, Response: guard.Attachment{}},
|
||||
{Method: "DELETE", Path: "/api/v1/guard/attachments", Summary: "Detach a guardrail policy",
|
||||
Request: GuardAttachmentRequest{}, Response: guard.Attachment{}},
|
||||
|
||||
// Usage, requests and the record of changes.
|
||||
{Method: "GET", Path: "/api/v1/usage", Summary: "What each group of the tenancy tree spent",
|
||||
Query: []Param{
|
||||
{Name: "by", Description: "org, team or project; team when absent."},
|
||||
{Name: "since", Description: "A duration back from now, like 24h; 24h when absent."},
|
||||
},
|
||||
Response: []ScopeUsageView{}},
|
||||
{Method: "GET", Path: "/api/v1/usage/export", Summary: "Usage totals by period and dimension, as CSV or JSON",
|
||||
Query: []Param{
|
||||
{Name: "from", Description: "A date (2026-09-01) or an RFC 3339 time; the first of this month when absent."},
|
||||
{Name: "to", Description: "A date or an RFC 3339 time; now when absent."},
|
||||
{Name: "period", Description: "hour, day or month; day when absent."},
|
||||
{Name: "by", Description: "Dimensions, comma-separated: org, team, project, user, key, model."},
|
||||
{Name: "format", Description: "json for JSON; CSV otherwise."},
|
||||
},
|
||||
Response: []store.ExportRow{}, CSV: true},
|
||||
{Method: "GET", Path: "/api/v1/requests/{id}", Summary: "What one request went through",
|
||||
Response: RequestView{}},
|
||||
{Method: "GET", Path: "/api/v1/requests/{id}/body", Summary: "A kept request and its answer",
|
||||
Response: store.RequestBody{}},
|
||||
{Method: "POST", Path: "/api/v1/requests/{id}/replay", Summary: "Send a kept request again, to any model",
|
||||
Request: ReplayRequest{}, Response: ReplayView{}},
|
||||
{Method: "GET", Path: "/api/v1/audit", Summary: "Changes made, newest first",
|
||||
Query: []Param{
|
||||
{Name: "since", Description: "How far back: 90m, 24h, 7d; 7d when absent."},
|
||||
{Name: "limit", Type: "integer", Description: "The most events to answer with."},
|
||||
{Name: "actor", Description: "Only changes by this actor."},
|
||||
{Name: "contains", Description: "Only events whose action or target contains this."},
|
||||
},
|
||||
Response: []store.AuditEvent{}},
|
||||
|
||||
// Operations.
|
||||
{Method: "POST", Path: "/api/v1/notify/test", Summary: "Send a test notification to every webhook",
|
||||
Response: []notify.Result{}},
|
||||
{Method: "POST", Path: "/api/v1/metrics/range", Summary: "Several Prometheus range queries at once",
|
||||
Description: "Read from the Prometheus monitoring.prometheus names; configured is false without one.",
|
||||
Request: MetricsRequest{}, Response: MetricsResult{}},
|
||||
}
|
||||
|
||||
// serveOpenAPI answers with the document generated from Routes, as it was
|
||||
// when this binary was built.
|
||||
func serveOpenAPI(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write(llmgw.AdminOpenAPI)
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
package adminapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"slices"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||
)
|
||||
|
||||
// patterns records what Register mounts.
|
||||
type patterns []string
|
||||
|
||||
func (p *patterns) HandleFunc(pattern string, _ func(http.ResponseWriter, *http.Request)) {
|
||||
*p = append(*p, pattern)
|
||||
}
|
||||
|
||||
func TestEveryRegisteredEndpointIsInTheRouteTable(t *testing.T) {
|
||||
// The OpenAPI document is generated from Routes, so an endpoint missing
|
||||
// from it is one no client generated from the document can call, and one
|
||||
// in it that is not served is a promise the server breaks.
|
||||
a, _ := setup(t, testModels())
|
||||
var got patterns
|
||||
a.Register(&got)
|
||||
|
||||
table := map[string]bool{}
|
||||
for _, r := range Routes {
|
||||
key := r.Method + " " + r.Path
|
||||
if table[key] {
|
||||
t.Errorf("%s is in Routes twice", key)
|
||||
}
|
||||
table[key] = true
|
||||
}
|
||||
served := map[string]bool{}
|
||||
for _, p := range got {
|
||||
served[p] = true
|
||||
if !table[p] {
|
||||
t.Errorf("%s is registered and not in Routes: describe it there", p)
|
||||
}
|
||||
}
|
||||
for key := range table {
|
||||
if !served[key] {
|
||||
t.Errorf("%s is in Routes and not registered", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryRouteSaysWhatItAnswers(t *testing.T) {
|
||||
for _, r := range Routes {
|
||||
if r.Summary == "" || r.Response == nil {
|
||||
t.Errorf("%s %s needs a summary and a response type", r.Method, r.Path)
|
||||
}
|
||||
if r.Request != nil && (r.Method == "GET" || r.Method == "HEAD") {
|
||||
t.Errorf("%s %s reads a body on a GET", r.Method, r.Path)
|
||||
}
|
||||
for _, seg := range strings.Split(strings.Trim(r.Path, "/"), "/") {
|
||||
if !strings.HasPrefix(seg, "{") {
|
||||
continue
|
||||
}
|
||||
found := false
|
||||
for k := range PathParams {
|
||||
if strings.HasSuffix(k, "/"+seg) && strings.Contains(r.Path, k) {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Errorf("%s %s: %s is not described in PathParams", r.Method, r.Path, seg)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTheAPIServesItsOwnDescription(t *testing.T) {
|
||||
a, _ := setup(t, testModels())
|
||||
var doc struct {
|
||||
OpenAPI string `json:"openapi"`
|
||||
Paths map[string]map[string]any `json:"paths"`
|
||||
}
|
||||
if code := call(t, a, "GET", "/api/v1/openapi.json", "", &doc); code != 200 {
|
||||
t.Fatalf("status %d", code)
|
||||
}
|
||||
if doc.OpenAPI != "3.1.0" {
|
||||
t.Errorf("openapi %q", doc.OpenAPI)
|
||||
}
|
||||
// What is built in is the document generated when it was last committed;
|
||||
// internal/docgen's test is what fails when that is stale. This checks the
|
||||
// two ends meet: every route the table has is in what is served.
|
||||
for _, r := range Routes {
|
||||
if _, ok := doc.Paths[r.Path][strings.ToLower(r.Method)]; !ok {
|
||||
t.Errorf("the served document has no %s %s", r.Method, r.Path)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestOneModelsDefinitionIsReadByItsNameAlone(t *testing.T) {
|
||||
a, _ := setup(t, testModels())
|
||||
var m catalog.Model
|
||||
if code := call(t, a, "GET", "/api/v1/models/llama", "", &m); code != 200 {
|
||||
t.Fatalf("status %d", code)
|
||||
}
|
||||
if m.Name != "llama" || m.Source != "/m" || len(m.Profiles) != 1 || m.Profiles[0].TP != 8 {
|
||||
t.Errorf("got %+v", m)
|
||||
}
|
||||
if m.Scaling.Min != 2 || m.Scaling.Max != 4 {
|
||||
t.Errorf("scaling %+v", m.Scaling)
|
||||
}
|
||||
if !slices.Equal(m.Aliases, []string{"l"}) {
|
||||
t.Errorf("aliases %v", m.Aliases)
|
||||
}
|
||||
// An alias answering would let a tool that manages "l" believe it exists
|
||||
// when what it found is llama.
|
||||
if code := call(t, a, "GET", "/api/v1/models/l", "", nil); code != http.StatusNotFound {
|
||||
t.Errorf("an alias answered %d", code)
|
||||
}
|
||||
if code := call(t, a, "GET", "/api/v1/models/nothing", "", nil); code != http.StatusNotFound {
|
||||
t.Errorf("a missing model answered %d", code)
|
||||
}
|
||||
}
|
||||
@@ -65,6 +65,20 @@ func (a *API) listOrphans(w http.ResponseWriter, _ *http.Request) {
|
||||
writeJSON(w, http.StatusOK, a.Orphans())
|
||||
}
|
||||
|
||||
// CollectRequest limits a sweep to one node. Empty means every node, which
|
||||
// is the common case after removing a model from the catalog.
|
||||
type CollectRequest struct {
|
||||
Node string `json:"node,omitempty"`
|
||||
}
|
||||
|
||||
// CollectResult is what was deleted on one node, or why nothing was.
|
||||
type CollectResult struct {
|
||||
Node string `json:"node"`
|
||||
Files int `json:"files"`
|
||||
Bytes int64 `json:"bytes"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
// collectWeights deletes them.
|
||||
//
|
||||
// A proposal, not an instruction: each node checks every path again before
|
||||
@@ -79,26 +93,16 @@ func (a *API) collectWeights(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
var body struct {
|
||||
// Node limits it to one node. Empty means every node, which is the
|
||||
// common case after removing a model from the catalog.
|
||||
Node string `json:"node,omitempty"`
|
||||
}
|
||||
var body CollectRequest
|
||||
// An empty body is the "all nodes" case and must not be an error.
|
||||
_ = json.NewDecoder(r.Body).Decode(&body)
|
||||
|
||||
type result struct {
|
||||
Node string `json:"node"`
|
||||
Files int `json:"files"`
|
||||
Bytes int64 `json:"bytes"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
out := make([]result, 0)
|
||||
out := make([]CollectResult, 0)
|
||||
for _, o := range a.Orphans() {
|
||||
if body.Node != "" && o.Node != body.Node {
|
||||
continue
|
||||
}
|
||||
res := result{Node: o.Node, Files: len(o.Paths), Bytes: o.Bytes}
|
||||
res := CollectResult{Node: o.Node, Files: len(o.Paths), Bytes: o.Bytes}
|
||||
// A node that has gone since the list was built is reported rather than
|
||||
// failing the whole sweep: the other nodes' space is still worth
|
||||
// freeing, and this one will be offered again next time.
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
// Command gen writes docs/reference.md from the configuration structs.
|
||||
// Command gen writes docs/reference.md from the configuration structs, and
|
||||
// docs/admin-api.openapi.json from the admin API route table.
|
||||
//
|
||||
// Run by `go generate ./...`, and checked by a test, so the reference cannot
|
||||
// quietly stop describing the code.
|
||||
@@ -10,7 +11,9 @@ import (
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/docgen"
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/docgen/openapi"
|
||||
)
|
||||
|
||||
func main() {
|
||||
@@ -43,4 +46,15 @@ func main() {
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("docs/reference.md and internal/docgen/reference_gen.go: %d blocks\n", len(blocks))
|
||||
|
||||
spec, err := openapi.Generate(root, adminapi.Routes, adminapi.PathParams)
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "docs", "admin-api.openapi.json"), spec, 0o644); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "error:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
fmt.Printf("docs/admin-api.openapi.json: %d routes\n", len(adminapi.Routes))
|
||||
}
|
||||
@@ -0,0 +1,672 @@
|
||||
// Package openapi generates the admin API's OpenAPI 3.1 document from its
|
||||
// route table, adminapi.Routes.
|
||||
//
|
||||
// The schemas are read from the Go types the handlers decode and write, by
|
||||
// reflection, following encoding/json's rules — tag names, omitted fields,
|
||||
// embedded structs flattened — so the document says what goes over the wire
|
||||
// rather than what someone remembered to write down. Their descriptions are
|
||||
// the types' and fields' doc comments, read from the source, which is why
|
||||
// generating needs the repository and is done by `go generate` rather than
|
||||
// at run time: the binary serves the document built into it.
|
||||
package openapi
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"go/ast"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"slices"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
|
||||
)
|
||||
|
||||
// Module is the import path the repository's packages are under; a type
|
||||
// from any other is described without comments.
|
||||
const Module = "gitea.mynodes.xyz/ilker/llmgw"
|
||||
|
||||
// Generate renders the document for routes, reading doc comments from the
|
||||
// repository at root.
|
||||
func Generate(root string, routes []adminapi.Route, pathParams map[string]string) ([]byte, error) {
|
||||
g := &generator{root: root, docs: map[string]map[string]typeDoc{}, names: map[reflect.Type]string{}}
|
||||
|
||||
// Every named struct reachable from a body first, so a name two
|
||||
// packages share can be told apart before either is written: by the name
|
||||
// alone where it is unique, with its package where it is not.
|
||||
for _, r := range routes {
|
||||
for _, v := range []any{r.Request, r.Response} {
|
||||
if v != nil {
|
||||
g.collect(reflect.TypeOf(v))
|
||||
}
|
||||
}
|
||||
}
|
||||
g.collect(errorType)
|
||||
g.assignNames()
|
||||
errorSchema := g.schema(errorType)
|
||||
|
||||
paths := map[string]*object{}
|
||||
var order []string
|
||||
tags := map[string]bool{}
|
||||
for _, r := range routes {
|
||||
item, ok := paths[r.Path]
|
||||
if !ok {
|
||||
item = &object{}
|
||||
paths[r.Path] = item
|
||||
order = append(order, r.Path)
|
||||
}
|
||||
op, err := g.operation(r, pathParams)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tags[tagOf(r.Path)] = true
|
||||
item.set(strings.ToLower(r.Method), op)
|
||||
}
|
||||
slices.Sort(order)
|
||||
var pathsObj object
|
||||
for _, p := range order {
|
||||
pathsObj.set(p, *paths[p])
|
||||
}
|
||||
|
||||
var schemas object
|
||||
names := make([]string, 0, len(g.schemas))
|
||||
for n := range g.schemas {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, n := range names {
|
||||
schemas.set(n, g.schemas[n])
|
||||
}
|
||||
|
||||
var tagList []any
|
||||
for _, t := range sortedKeys(tags) {
|
||||
tagList = append(tagList, obj("name", t))
|
||||
}
|
||||
|
||||
doc := obj(
|
||||
"openapi", "3.1.0",
|
||||
"info", obj(
|
||||
"title", "llmgw admin API",
|
||||
"version", "v1",
|
||||
"description", infoDescription,
|
||||
),
|
||||
"servers", []any{obj("url", "http://127.0.0.1:9090",
|
||||
"description", "The admin listener at its default address, admin_listen.")},
|
||||
"security", []any{obj("adminToken", []any{})},
|
||||
"tags", tagList,
|
||||
"paths", pathsObj,
|
||||
"components", obj(
|
||||
"securitySchemes", obj("adminToken", obj(
|
||||
"type", "http",
|
||||
"scheme", "bearer",
|
||||
"description", "The admin token: the contents of /var/lib/llmgw/admin-token on the "+
|
||||
"control plane (admin_token_file), readable by root and the server. "+
|
||||
"Sent as Authorization: Bearer TOKEN.",
|
||||
)),
|
||||
"parameters", obj("Actor", obj(
|
||||
"name", "X-Llmgw-Actor",
|
||||
"in", "header",
|
||||
"required", false,
|
||||
"description", "Who the change is recorded as made by in the audit log: letters, digits "+
|
||||
"and ._-@, at most 64. unknown when absent.",
|
||||
"schema", obj("type", "string"),
|
||||
)),
|
||||
"responses", obj(
|
||||
"Error", obj("description", "Refused: the error says why.",
|
||||
"content", obj("application/json", obj("schema", errorSchema))),
|
||||
"Unauthorized", obj("description", "The admin token is missing or wrong.",
|
||||
"content", obj("application/json", obj("schema", errorSchema))),
|
||||
),
|
||||
"schemas", schemas,
|
||||
),
|
||||
)
|
||||
b, err := marshal(doc)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out bytes.Buffer
|
||||
if err := json.Indent(&out, b, "", " "); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out.WriteByte('\n')
|
||||
return out.Bytes(), nil
|
||||
}
|
||||
|
||||
const infoDescription = "The control plane's management API: what `llmgw ctl` and the console use.\n\n" +
|
||||
"It is served on the admin listener (admin_listen, 127.0.0.1:9090 by default), which stays on " +
|
||||
"loopback unless admin_remote is set: reach it from elsewhere through an SSH tunnel. Every path " +
|
||||
"under /api needs the admin token as a bearer token.\n\n" +
|
||||
"The listener also refuses, before the token is looked at: a Host header that is a name rather " +
|
||||
"than an address or localhost; a request another site's page sent; and any method but GET and " +
|
||||
"HEAD whose Content-Type is not application/json, even one with no body.\n\n" +
|
||||
"Durations in bodies are integer nanoseconds, as Go encodes them. Generated from the route " +
|
||||
"table in internal/control/adminapi; see docs/admin-api.md."
|
||||
|
||||
// operation builds one route's operation object.
|
||||
func (g *generator) operation(r adminapi.Route, pathParams map[string]string) (object, error) {
|
||||
var op object
|
||||
op.set("operationId", operationID(r.Method, r.Path))
|
||||
op.set("summary", r.Summary)
|
||||
if r.Description != "" {
|
||||
op.set("description", r.Description)
|
||||
}
|
||||
op.set("tags", []any{tagOf(r.Path)})
|
||||
|
||||
var params []any
|
||||
segs := strings.Split(strings.Trim(r.Path, "/"), "/")
|
||||
for i, s := range segs {
|
||||
if !strings.HasPrefix(s, "{") {
|
||||
continue
|
||||
}
|
||||
name := strings.Trim(s, "{}")
|
||||
key := s
|
||||
if i > 0 {
|
||||
key = segs[i-1] + "/" + s
|
||||
}
|
||||
desc, ok := pathParams[key]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s %s: no description of %s in adminapi.PathParams", r.Method, r.Path, key)
|
||||
}
|
||||
params = append(params, obj("name", name, "in", "path", "required", true,
|
||||
"description", desc, "schema", obj("type", "string")))
|
||||
}
|
||||
for _, q := range r.Query {
|
||||
t := q.Type
|
||||
if t == "" {
|
||||
t = "string"
|
||||
}
|
||||
p := obj("name", q.Name, "in", "query")
|
||||
if q.Required {
|
||||
p.set("required", true)
|
||||
}
|
||||
p.set("description", q.Description)
|
||||
p.set("schema", obj("type", t))
|
||||
params = append(params, p)
|
||||
}
|
||||
if r.Method != "GET" && r.Method != "HEAD" {
|
||||
params = append(params, obj("$ref", "#/components/parameters/Actor"))
|
||||
}
|
||||
if len(params) > 0 {
|
||||
op.set("parameters", params)
|
||||
}
|
||||
|
||||
if r.Request != nil {
|
||||
body := obj("required", !r.OptionalBody,
|
||||
"content", obj("application/json", obj("schema", g.schema(reflect.TypeOf(r.Request)))))
|
||||
op.set("requestBody", body)
|
||||
}
|
||||
|
||||
status := r.Status
|
||||
if status == 0 {
|
||||
status = 200
|
||||
}
|
||||
content := obj("application/json", obj("schema", g.schema(reflect.TypeOf(r.Response))))
|
||||
if r.CSV {
|
||||
content.set("text/csv", obj("schema", obj("type", "string")))
|
||||
}
|
||||
desc := "OK"
|
||||
if status == 202 {
|
||||
desc = "Accepted: started, and running on."
|
||||
}
|
||||
op.set("responses", obj(
|
||||
fmt.Sprint(status), obj("description", desc, "content", content),
|
||||
"401", obj("$ref", "#/components/responses/Unauthorized"),
|
||||
"default", obj("$ref", "#/components/responses/Error"),
|
||||
))
|
||||
return op, nil
|
||||
}
|
||||
|
||||
// operationID is the method and the path's words, getModelsNameTarget.
|
||||
func operationID(method, p string) string {
|
||||
var b strings.Builder
|
||||
b.WriteString(strings.ToLower(method))
|
||||
for _, s := range strings.Split(strings.TrimPrefix(p, "/api/v1/"), "/") {
|
||||
for _, w := range strings.FieldsFunc(s, func(r rune) bool { return !unicode.IsLetter(r) && !unicode.IsDigit(r) }) {
|
||||
b.WriteString(strings.ToUpper(w[:1]) + w[1:])
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// tagOf groups an operation by the first word of its path.
|
||||
func tagOf(p string) string {
|
||||
first, _, _ := strings.Cut(strings.TrimPrefix(p, "/api/v1/"), "/")
|
||||
return strings.TrimSuffix(first, ".json")
|
||||
}
|
||||
|
||||
// --- schemas ---
|
||||
|
||||
type typeDoc struct {
|
||||
doc string
|
||||
fields map[string]string
|
||||
}
|
||||
|
||||
type generator struct {
|
||||
root string
|
||||
docs map[string]map[string]typeDoc
|
||||
names map[reflect.Type]string
|
||||
schemas map[string]any
|
||||
seen []reflect.Type
|
||||
}
|
||||
|
||||
var (
|
||||
timeType = reflect.TypeOf(time.Time{})
|
||||
durationType = reflect.TypeOf(time.Duration(0))
|
||||
rawType = reflect.TypeOf(json.RawMessage{})
|
||||
errorType = reflect.TypeOf(adminapi.ErrorView{})
|
||||
marshalerType = reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||
textType = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
|
||||
)
|
||||
|
||||
// collect records every named struct reachable from t.
|
||||
func (g *generator) collect(t reflect.Type) {
|
||||
for t.Kind() == reflect.Pointer || t.Kind() == reflect.Slice || t.Kind() == reflect.Array || t.Kind() == reflect.Map {
|
||||
if t == rawType {
|
||||
return
|
||||
}
|
||||
t = t.Elem()
|
||||
}
|
||||
if t.Kind() != reflect.Struct || special(t) {
|
||||
return
|
||||
}
|
||||
if t.Name() != "" {
|
||||
if slices.Contains(g.seen, t) {
|
||||
return
|
||||
}
|
||||
g.seen = append(g.seen, t)
|
||||
}
|
||||
for _, f := range fields(t) {
|
||||
g.collect(f.typ)
|
||||
}
|
||||
}
|
||||
|
||||
// assignNames names each struct's schema: by its name alone when it is the
|
||||
// admin API's own, with its package's otherwise — catalog.Model,
|
||||
// modelserve.Job. Qualified always rather than only on a collision, so a
|
||||
// type added elsewhere one day cannot rename a schema clients already use.
|
||||
func (g *generator) assignNames() {
|
||||
own := reflect.TypeOf(adminapi.Route{}).PkgPath()
|
||||
for _, t := range g.seen {
|
||||
n := t.Name()
|
||||
if t.PkgPath() != own {
|
||||
n = path.Base(t.PkgPath()) + "." + n
|
||||
}
|
||||
g.names[t] = n
|
||||
}
|
||||
g.schemas = map[string]any{}
|
||||
}
|
||||
|
||||
func (g *generator) nameOf(t reflect.Type) string { return g.names[t] }
|
||||
|
||||
// special is a type encoded other than by its fields.
|
||||
func special(t reflect.Type) bool {
|
||||
if t == timeType || t == durationType || t == rawType {
|
||||
return true
|
||||
}
|
||||
return t.Implements(marshalerType) || reflect.PointerTo(t).Implements(marshalerType) ||
|
||||
t.Implements(textType) || reflect.PointerTo(t).Implements(textType)
|
||||
}
|
||||
|
||||
func ref(name string) object { return obj("$ref", "#/components/schemas/"+name) }
|
||||
|
||||
// schema describes t, adding the named structs it reaches to the components.
|
||||
func (g *generator) schema(t reflect.Type) any {
|
||||
switch {
|
||||
case t == timeType:
|
||||
return obj("type", "string", "format", "date-time")
|
||||
case t == durationType:
|
||||
return obj("type", "integer", "format", "int64", "description", "A duration, in nanoseconds.")
|
||||
case t == rawType:
|
||||
return obj("description", "Any JSON value.")
|
||||
case t.Implements(textType) || reflect.PointerTo(t).Implements(textType):
|
||||
return obj("type", "string")
|
||||
case t.Implements(marshalerType) || reflect.PointerTo(t).Implements(marshalerType):
|
||||
return obj("description", "Encoded by the type's own JSON encoding.")
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Bool:
|
||||
return obj("type", "boolean")
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Uint8, reflect.Uint16, reflect.Uint32:
|
||||
return obj("type", "integer")
|
||||
case reflect.Int64, reflect.Uint, reflect.Uint64:
|
||||
return obj("type", "integer", "format", "int64")
|
||||
case reflect.Float32, reflect.Float64:
|
||||
return obj("type", "number")
|
||||
case reflect.String:
|
||||
return obj("type", "string")
|
||||
case reflect.Pointer:
|
||||
return g.schema(t.Elem())
|
||||
case reflect.Slice, reflect.Array:
|
||||
if t.Elem().Kind() == reflect.Uint8 {
|
||||
return obj("type", "string", "contentEncoding", "base64")
|
||||
}
|
||||
return obj("type", "array", "items", g.schema(t.Elem()))
|
||||
case reflect.Map:
|
||||
return obj("type", "object", "additionalProperties", g.schema(t.Elem()))
|
||||
case reflect.Interface:
|
||||
return obj()
|
||||
case reflect.Struct:
|
||||
if t.Name() == "" {
|
||||
return g.structSchema(t, "")
|
||||
}
|
||||
name := g.names[t]
|
||||
if name == "" {
|
||||
// Reached only through a body collect did not walk, which would be
|
||||
// a bug here; named anyway rather than inlined so it shows.
|
||||
name = t.Name()
|
||||
g.names[t] = name
|
||||
}
|
||||
if _, ok := g.schemas[name]; !ok {
|
||||
g.schemas[name] = nil // placeholder: a type that contains itself
|
||||
g.schemas[name] = g.structSchema(t, g.typeDocOf(t).doc)
|
||||
}
|
||||
return ref(name)
|
||||
}
|
||||
return obj()
|
||||
}
|
||||
|
||||
func (g *generator) structSchema(t reflect.Type, doc string) object {
|
||||
s := obj("type", "object")
|
||||
if doc != "" {
|
||||
s.set("description", doc)
|
||||
}
|
||||
var props object
|
||||
for _, f := range fields(t) {
|
||||
var p any
|
||||
if f.asString {
|
||||
p = obj("type", "string")
|
||||
} else {
|
||||
p = g.schema(f.typ)
|
||||
}
|
||||
if d := g.typeDocOf(f.owner).fields[f.goName]; d != "" {
|
||||
p = withDescription(p, d)
|
||||
}
|
||||
props.set(f.name, p)
|
||||
}
|
||||
s.set("properties", props)
|
||||
return s
|
||||
}
|
||||
|
||||
// withDescription puts a field's comment on its schema. A reference keeps
|
||||
// its own description where it is declared; in 3.1 a sibling description
|
||||
// beside $ref is allowed and is the field's.
|
||||
func withDescription(p any, d string) any {
|
||||
o, ok := p.(object)
|
||||
if !ok {
|
||||
return p
|
||||
}
|
||||
if prev, ok := o.get("description"); ok {
|
||||
d = d + "\n\n" + prev.(string)
|
||||
}
|
||||
o = slices.Clone(o)
|
||||
o.set("description", d)
|
||||
return o
|
||||
}
|
||||
|
||||
// field is one key of a struct's JSON.
|
||||
type field struct {
|
||||
name string
|
||||
goName string
|
||||
owner reflect.Type // the struct it is declared in, for its comment
|
||||
typ reflect.Type
|
||||
asString bool
|
||||
depth int
|
||||
}
|
||||
|
||||
// fields lists a struct's JSON keys as encoding/json writes them: exported
|
||||
// fields, by tag name, "-" left out, embedded structs without a name of
|
||||
// their own flattened, and a key at a shallower depth winning over the same
|
||||
// key from deeper in.
|
||||
func fields(t reflect.Type) []field {
|
||||
var out []field
|
||||
var walk func(t reflect.Type, depth int)
|
||||
walk = func(t reflect.Type, depth int) {
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
sf := t.Field(i)
|
||||
tag := sf.Tag.Get("json")
|
||||
if tag == "-" {
|
||||
continue
|
||||
}
|
||||
name, opts, _ := strings.Cut(tag, ",")
|
||||
ft := sf.Type
|
||||
if sf.Anonymous && name == "" {
|
||||
et := ft
|
||||
if et.Kind() == reflect.Pointer {
|
||||
et = et.Elem()
|
||||
}
|
||||
if et.Kind() == reflect.Struct && !special(et) {
|
||||
walk(et, depth+1)
|
||||
continue
|
||||
}
|
||||
}
|
||||
if !sf.IsExported() {
|
||||
continue
|
||||
}
|
||||
if name == "" {
|
||||
name = sf.Name
|
||||
}
|
||||
out = append(out, field{name: name, goName: sf.Name, owner: t, typ: ft, depth: depth,
|
||||
asString: slices.Contains(strings.Split(opts, ","), "string")})
|
||||
}
|
||||
}
|
||||
walk(t, 0)
|
||||
// The shallowest of each name, in the order they were met.
|
||||
best := map[string]int{}
|
||||
for _, f := range out {
|
||||
if d, ok := best[f.name]; !ok || f.depth < d {
|
||||
best[f.name] = f.depth
|
||||
}
|
||||
}
|
||||
var kept []field
|
||||
taken := map[string]bool{}
|
||||
for _, f := range out {
|
||||
if f.depth == best[f.name] && !taken[f.name] {
|
||||
taken[f.name] = true
|
||||
kept = append(kept, f)
|
||||
}
|
||||
}
|
||||
return kept
|
||||
}
|
||||
|
||||
// typeDocOf reads a type's comment and its fields' from its package's source.
|
||||
func (g *generator) typeDocOf(t reflect.Type) typeDoc {
|
||||
pkg := t.PkgPath()
|
||||
if pkg != Module && !strings.HasPrefix(pkg, Module+"/") {
|
||||
return typeDoc{}
|
||||
}
|
||||
docs, ok := g.docs[pkg]
|
||||
if !ok {
|
||||
docs = parseDocs(filepath.Join(g.root, filepath.FromSlash(strings.TrimPrefix(strings.TrimPrefix(pkg, Module), "/"))))
|
||||
g.docs[pkg] = docs
|
||||
}
|
||||
return docs[t.Name()]
|
||||
}
|
||||
|
||||
// parseDocs reads the struct types declared in a directory and their
|
||||
// comments. A directory it cannot read gives no comments rather than an
|
||||
// error: the document is still right, only quieter.
|
||||
func parseDocs(dir string) map[string]typeDoc {
|
||||
out := map[string]typeDoc{}
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return out
|
||||
}
|
||||
fset := token.NewFileSet()
|
||||
for _, e := range entries {
|
||||
n := e.Name()
|
||||
if e.IsDir() || !strings.HasSuffix(n, ".go") || strings.HasSuffix(n, "_test.go") {
|
||||
continue
|
||||
}
|
||||
f, err := parser.ParseFile(fset, filepath.Join(dir, n), nil, parser.ParseComments)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for _, d := range f.Decls {
|
||||
gd, ok := d.(*ast.GenDecl)
|
||||
if !ok || gd.Tok != token.TYPE {
|
||||
continue
|
||||
}
|
||||
for _, sp := range gd.Specs {
|
||||
ts := sp.(*ast.TypeSpec)
|
||||
doc := ts.Doc
|
||||
if doc == nil && len(gd.Specs) == 1 {
|
||||
doc = gd.Doc
|
||||
}
|
||||
td := typeDoc{doc: text(doc), fields: map[string]string{}}
|
||||
if st, ok := ts.Type.(*ast.StructType); ok {
|
||||
for _, fl := range st.Fields.List {
|
||||
c := fl.Doc
|
||||
if c == nil {
|
||||
c = fl.Comment
|
||||
}
|
||||
for _, name := range fieldNames(fl) {
|
||||
td.fields[name] = text(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
out[ts.Name.Name] = td
|
||||
}
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func fieldNames(fl *ast.Field) []string {
|
||||
if len(fl.Names) > 0 {
|
||||
var out []string
|
||||
for _, n := range fl.Names {
|
||||
out = append(out, n.Name)
|
||||
}
|
||||
return out
|
||||
}
|
||||
// Embedded: named by its type.
|
||||
t := fl.Type
|
||||
if s, ok := t.(*ast.StarExpr); ok {
|
||||
t = s.X
|
||||
}
|
||||
switch x := t.(type) {
|
||||
case *ast.Ident:
|
||||
return []string{x.Name}
|
||||
case *ast.SelectorExpr:
|
||||
return []string{x.Sel.Name}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// text is a comment as prose: the comment markers gone, lines within a
|
||||
// paragraph joined, paragraphs kept apart.
|
||||
func text(g *ast.CommentGroup) string {
|
||||
if g == nil {
|
||||
return ""
|
||||
}
|
||||
var paras []string
|
||||
var cur []string
|
||||
for _, line := range strings.Split(g.Text(), "\n") {
|
||||
line = strings.TrimSpace(line)
|
||||
if line == "" {
|
||||
if len(cur) > 0 {
|
||||
paras = append(paras, strings.Join(cur, " "))
|
||||
cur = nil
|
||||
}
|
||||
continue
|
||||
}
|
||||
cur = append(cur, line)
|
||||
}
|
||||
if len(cur) > 0 {
|
||||
paras = append(paras, strings.Join(cur, " "))
|
||||
}
|
||||
return strings.Join(paras, "\n\n")
|
||||
}
|
||||
|
||||
// --- an object that keeps its keys in order ---
|
||||
|
||||
type kv struct {
|
||||
k string
|
||||
v any
|
||||
}
|
||||
|
||||
// object is a JSON object written in the order its keys were set, so the
|
||||
// document reads in the order it was meant to — openapi, info, paths — and
|
||||
// a struct's properties in the order its fields are declared.
|
||||
type object []kv
|
||||
|
||||
func obj(pairs ...any) object {
|
||||
var o object
|
||||
for i := 0; i+1 < len(pairs); i += 2 {
|
||||
o.set(pairs[i].(string), pairs[i+1])
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
func (o *object) set(k string, v any) {
|
||||
for i := range *o {
|
||||
if (*o)[i].k == k {
|
||||
(*o)[i].v = v
|
||||
return
|
||||
}
|
||||
}
|
||||
*o = append(*o, kv{k, v})
|
||||
}
|
||||
|
||||
func (o object) get(k string) (any, bool) {
|
||||
for _, p := range o {
|
||||
if p.k == k {
|
||||
return p.v, true
|
||||
}
|
||||
}
|
||||
return nil, false
|
||||
}
|
||||
|
||||
// MarshalJSON writes the keys in order.
|
||||
func (o object) MarshalJSON() ([]byte, error) {
|
||||
var b bytes.Buffer
|
||||
b.WriteByte('{')
|
||||
for i, p := range o {
|
||||
if i > 0 {
|
||||
b.WriteByte(',')
|
||||
}
|
||||
k, err := marshal(p.k)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.Write(k)
|
||||
b.WriteByte(':')
|
||||
v, err := marshal(p.v)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b.Write(v)
|
||||
}
|
||||
b.WriteByte('}')
|
||||
return b.Bytes(), nil
|
||||
}
|
||||
|
||||
// marshal is json.Marshal without escaping <, > and &, which are common in
|
||||
// the descriptions and harmless in a document nobody puts in a page.
|
||||
func marshal(v any) ([]byte, error) {
|
||||
var b bytes.Buffer
|
||||
enc := json.NewEncoder(&b)
|
||||
enc.SetEscapeHTML(false)
|
||||
if err := enc.Encode(v); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return bytes.TrimRight(b.Bytes(), "\n"), nil
|
||||
}
|
||||
|
||||
func sortedKeys(m map[string]bool) []string {
|
||||
out := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
out = append(out, k)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package openapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
|
||||
)
|
||||
|
||||
func repoRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("cannot locate this test")
|
||||
}
|
||||
return filepath.Join(filepath.Dir(file), "..", "..", "..")
|
||||
}
|
||||
|
||||
func generated(t *testing.T) map[string]any {
|
||||
t.Helper()
|
||||
b, err := Generate(repoRoot(t), adminapi.Routes, adminapi.PathParams)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var doc map[string]any
|
||||
if err := json.Unmarshal(b, &doc); err != nil {
|
||||
t.Fatalf("the document is not JSON: %v", err)
|
||||
}
|
||||
return doc
|
||||
}
|
||||
|
||||
// at walks a decoded document by keys.
|
||||
func at(t *testing.T, v any, keys ...string) any {
|
||||
t.Helper()
|
||||
for _, k := range keys {
|
||||
m, ok := v.(map[string]any)
|
||||
if !ok {
|
||||
t.Fatalf("%s: not an object", strings.Join(keys, "."))
|
||||
}
|
||||
v, ok = m[k]
|
||||
if !ok {
|
||||
t.Fatalf("no %s (of %s)", k, strings.Join(keys, "."))
|
||||
}
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
func TestTheAdminAPIDocumentIsNotStale(t *testing.T) {
|
||||
// Like the configuration reference: a description of the API that is not
|
||||
// regenerated describes some earlier week's API, and the server serves the
|
||||
// committed copy to every client generated from it.
|
||||
want, err := Generate(repoRoot(t), adminapi.Routes, adminapi.PathParams)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := os.ReadFile(filepath.Join(repoRoot(t), "docs", "admin-api.openapi.json"))
|
||||
if err != nil {
|
||||
t.Fatalf("docs/admin-api.openapi.json is missing; run go generate ./internal/docgen: %v", err)
|
||||
}
|
||||
if string(got) != string(want) {
|
||||
t.Error("docs/admin-api.openapi.json no longer matches the admin API's routes and types.\n" +
|
||||
"Run: go generate ./internal/docgen")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryReferenceInTheDocumentResolves(t *testing.T) {
|
||||
doc := generated(t)
|
||||
var walk func(v any)
|
||||
n := 0
|
||||
walk = func(v any) {
|
||||
switch x := v.(type) {
|
||||
case map[string]any:
|
||||
if r, ok := x["$ref"].(string); ok {
|
||||
n++
|
||||
parts := strings.Split(strings.TrimPrefix(r, "#/"), "/")
|
||||
at(t, doc, parts...)
|
||||
}
|
||||
for _, e := range x {
|
||||
walk(e)
|
||||
}
|
||||
case []any:
|
||||
for _, e := range x {
|
||||
walk(e)
|
||||
}
|
||||
}
|
||||
}
|
||||
walk(doc)
|
||||
if n < 50 {
|
||||
t.Errorf("only %d references; the schemas are not being shared", n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestABodyIsDescribedByItsGoType(t *testing.T) {
|
||||
doc := generated(t)
|
||||
scale := at(t, doc, "paths", "/api/v1/models/{name}/scale", "post")
|
||||
if r := at(t, scale, "requestBody", "content", "application/json", "schema", "$ref"); r != "#/components/schemas/ScaleRequest" {
|
||||
t.Errorf("scale's body is %v", r)
|
||||
}
|
||||
props := at(t, doc, "components", "schemas", "ScaleRequest", "properties").(map[string]any)
|
||||
// The tag's name, not the field's.
|
||||
for _, k := range []string{"min", "max", "idle_timeout"} {
|
||||
if _, ok := props[k]; !ok {
|
||||
t.Errorf("ScaleRequest has no %s: %v", k, props)
|
||||
}
|
||||
}
|
||||
// The field's comment is its description, and a duration says its unit.
|
||||
if d := at(t, props, "max", "description").(string); !strings.Contains(d, "follows Min") {
|
||||
t.Errorf("max lost its comment: %q", d)
|
||||
}
|
||||
if d := at(t, props, "idle_timeout", "description").(string); !strings.Contains(d, "nanoseconds") {
|
||||
t.Errorf("idle_timeout does not say it is nanoseconds: %q", d)
|
||||
}
|
||||
// A type from another package is named with its package.
|
||||
if r := at(t, doc, "paths", "/api/v1/models", "post", "requestBody", "content", "application/json", "schema", "$ref"); r != "#/components/schemas/catalog.Model" {
|
||||
t.Errorf("adding a model takes %v", r)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnEmbeddedStructsFieldsAreItsOwn(t *testing.T) {
|
||||
// CapacityView embeds bench.Capacity, and encoding/json writes the
|
||||
// embedded fields at the top level: so must the schema.
|
||||
doc := generated(t)
|
||||
props := at(t, doc, "components", "schemas", "CapacityView", "properties").(map[string]any)
|
||||
for _, k := range []string{"measured_by", "model", "profile"} {
|
||||
if _, ok := props[k]; !ok {
|
||||
t.Errorf("CapacityView has no %s: %v", k, props)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestEveryChangeNeedsTheTokenAndNamesItsActor(t *testing.T) {
|
||||
doc := generated(t)
|
||||
if s := at(t, doc, "components", "securitySchemes", "adminToken", "scheme"); s != "bearer" {
|
||||
t.Errorf("scheme %v", s)
|
||||
}
|
||||
if len(at(t, doc, "security").([]any)) != 1 {
|
||||
t.Error("the token is not required document-wide")
|
||||
}
|
||||
del := at(t, doc, "paths", "/api/v1/models/{name}", "delete")
|
||||
found := false
|
||||
for _, p := range at(t, del, "parameters").([]any) {
|
||||
if p.(map[string]any)["$ref"] == "#/components/parameters/Actor" {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
t.Error("removing a model does not take X-Llmgw-Actor")
|
||||
}
|
||||
if _, ok := at(t, del, "responses").(map[string]any)["401"]; !ok {
|
||||
t.Error("removing a model does not say it can be refused for the token")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPathParameterWithNoDescriptionIsAnError(t *testing.T) {
|
||||
routes := []adminapi.Route{{Method: "GET", Path: "/api/v1/things/{id}", Summary: "x", Response: adminapi.Result{}}}
|
||||
if _, err := Generate(repoRoot(t), routes, adminapi.PathParams); err == nil {
|
||||
t.Error("a parameter nobody described was let through")
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user