Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2005cea326 | ||
|
|
f27d722111 | ||
|
|
fe310576e7 | ||
|
|
688b205604 | ||
|
|
35ccabac28 | ||
|
|
2545fca174 | ||
|
|
b8274d6278 | ||
|
|
8164260e3b | ||
|
|
fbb5df05b8 |
No files matched your search
+153
-3
@@ -15,6 +15,41 @@ Conventions this codebase keeps, which every item here must keep too:
|
|||||||
- `go test ./...`, `go vet ./...` and `govulncheck ./...` pass. No release is
|
- `go test ./...`, `go vet ./...` and `govulncheck ./...` pass. No release is
|
||||||
cut and nothing is deployed to the PoC cluster without being told to.
|
cut and nothing is deployed to the PoC cluster without being told to.
|
||||||
|
|
||||||
|
## Open, in order (2026-10-16, after v0.14.7)
|
||||||
|
|
||||||
|
What is left, most needed first. The detail is in the item each names.
|
||||||
|
|
||||||
|
1. ~~**J1**~~ — done 2026-10-16: a restart no longer refused for room
|
||||||
|
another restart was giving back.
|
||||||
|
2. ~~**J5**~~ — done 2026-10-16: an upgrade refits an earlier install's
|
||||||
|
judge by itself.
|
||||||
|
3. ~~**I3**~~ — done 2026-10-16: `ctl bench overhead`, figures in the
|
||||||
|
README.
|
||||||
|
4. ~~**I4**~~ — done 2026-10-17: several keys for one outside provider.
|
||||||
|
5. ~~**Active Directory and Entra ID**~~ — done 2026-10-18: `auth.ldap`
|
||||||
|
sign-in and sync, Entra's 200-group overflow kept from taking roles.
|
||||||
|
Still to do: try it against a real domain controller (a Samba AD DC
|
||||||
|
on Proxmox): LDAPS, nested groups, a disabled user suspended by the
|
||||||
|
sync.
|
||||||
|
6. **J2** — a judge the leader adds by itself is sized as init's is. (S)
|
||||||
|
7. **J3** — HA setup puts the judges on nodes of their own. (S)
|
||||||
|
8. **J8** — a judge budget that fits the machine, measured at install. (S)
|
||||||
|
9. **F5** — large imports limited by the embedding engine's slots. (M)
|
||||||
|
10. **I1** — answers to similar questions from the cache. (M)
|
||||||
|
11. **J4** — the model mirror on more than one machine. (M)
|
||||||
|
12. **I2** — the customer's MCP servers behind llmgw. (L)
|
||||||
|
13. Smaller leftovers: audit before/after values, a tenant-admin role and
|
||||||
|
forms hidden from viewers (C1); Parquet and a scheduled push for usage
|
||||||
|
export (C4); English errors on Turkish pages (C24); H1 tried on the PoC;
|
||||||
|
licences in video and audio (H5); nginx in front of several control
|
||||||
|
planes (J7).
|
||||||
|
|
||||||
|
Not code, the operator's to decide or do: the PoC on v0.14.7 (it runs a
|
||||||
|
v0.14.5 dev build; no hurry); where Alertmanager sends (its receiver is
|
||||||
|
"none"). Settled 2026-10-16: Apertus 8B is on demand (min 0); the HA test
|
||||||
|
VMs 157–159, 165, 166 and the e2e VMs 151, 153–156 (a v0.11 HA test from
|
||||||
|
2026-10-03, idle since, holding 10.0.0.160 with VRRP id 51) were removed.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## A. Security — do first
|
## A. Security — do first
|
||||||
@@ -1429,6 +1464,14 @@ console show and add them.
|
|||||||
- Done 2026-10-15: llmgw's `/mcp` with picture, 3D, VLM and workflow tools,
|
- Done 2026-10-15: llmgw's `/mcp` with picture, 3D, VLM and workflow tools,
|
||||||
so a client needs no server of its own (asked by the game team); the
|
so a client needs no server of its own (asked by the game team); the
|
||||||
shared ComfyUI replica (H3's placement side).
|
shared ComfyUI replica (H3's placement side).
|
||||||
|
- Done 2026-10-16 (v0.14.6, v0.14.7): setup lists every part with what it
|
||||||
|
is and its settings; three machines each run the control plane, with a
|
||||||
|
floating address (VRRP id from the address); a lost store primary costs no
|
||||||
|
request (3 s connect per member, keepalive and TCP_USER_TIMEOUT, Patroni
|
||||||
|
ttl 20, a standby leaves the lease alone a round after the store returns);
|
||||||
|
a new leader waits for its judge within the budget; a judge is asked again
|
||||||
|
on another machine at half its budget; init sizes the judge (2 GB, no
|
||||||
|
prompt cache, two cores, one judge on up to four cores on one machine).
|
||||||
|
|
||||||
## I. What other gateways do that llmgw does not (written 2026-10-16)
|
## I. What other gateways do that llmgw does not (written 2026-10-16)
|
||||||
|
|
||||||
@@ -1485,7 +1528,16 @@ in every client with its own credentials and nobody seeing what was called.
|
|||||||
through it, a key without the grant does not see the server's tools, and
|
through it, a key without the grant does not see the server's tools, and
|
||||||
every call is in `ctl audit`.
|
every call is in `ctl audit`.
|
||||||
|
|
||||||
### I3. What llmgw adds to a request, measured (S)
|
### I3. What llmgw adds to a request, measured (S) — done 2026-10-16
|
||||||
|
`llmgw ctl bench overhead` (`internal/cli/bench_overhead.go`): an instant
|
||||||
|
engine and the gateway in one process, four modes (direct, bare, auth, auth
|
||||||
|
and guard patterns) at rising rates. On an Apple M5 the gateway added
|
||||||
|
0.05–0.42 ms at p50 and at most 0.49 ms at p99, in the README.
|
||||||
|
`TestTheGatewaysOverheadIsMeasured` runs a second of it in CI with no bound
|
||||||
|
asserted. Left: the CPU figure is rough with everything in one process; a
|
||||||
|
gateway in a process of its own would measure it properly. The original
|
||||||
|
item follows.
|
||||||
|
|
||||||
Bifrost's headline is "11 µs at 5k RPS" (its own figure, and its own pages
|
Bifrost's headline is "11 µs at 5k RPS" (its own figure, and its own pages
|
||||||
disagree with it). llmgw has none, so any question about its overhead is
|
disagree with it). llmgw has none, so any question about its overhead is
|
||||||
answered with a guess.
|
answered with a guess.
|
||||||
@@ -1500,7 +1552,13 @@ answered with a guess.
|
|||||||
- Done when: the README states the overhead, with how it was measured, and
|
- Done when: the README states the overhead, with how it was measured, and
|
||||||
CI runs the short version so a change that doubles it is seen.
|
CI runs the short version so a change that doubles it is seen.
|
||||||
|
|
||||||
### I4. Several keys for one outside provider (S)
|
### I4. Several keys for one outside provider (S) — done 2026-10-17
|
||||||
|
`external.api_key_files` (`catalog.ExternalKey`: file, weight), one endpoint
|
||||||
|
each in `control.externalEndpoints`, 401 retried on another key and
|
||||||
|
`external_key_refused` notified (`router.keyRefused`). With one key a 401 is
|
||||||
|
still the client's. `TestAKeyAtItsLimitOrRefusedIsPassedOverForAnother`. The
|
||||||
|
original item follows.
|
||||||
|
|
||||||
An external model has one `api_key_file` (`internal/control/control.go`,
|
An external model has one `api_key_file` (`internal/control/control.go`,
|
||||||
where external endpoints are made). A provider's limit is per key; one key
|
where external endpoints are made). A provider's limit is per key; one key
|
||||||
answering 429 makes the whole fallback answer 429.
|
answering 429 makes the whole fallback answer 429.
|
||||||
@@ -1513,9 +1571,101 @@ answering 429 makes the whole fallback answer 429.
|
|||||||
- Done when: with two keys and one returning 429, every request is answered
|
- Done when: with two keys and one returning 429, every request is answered
|
||||||
by the other, and the console shows the one breaker open.
|
by the other, and the console shows the one breaker open.
|
||||||
|
|
||||||
|
## J. Found testing HA and single-machine installs (written 2026-10-16)
|
||||||
|
|
||||||
|
Three VMs installed as HA from the setup screens, a fourth as a judge node,
|
||||||
|
a fifth as a single 4-core, 8 GB machine; machines powered off under load.
|
||||||
|
|
||||||
|
### J1. A restart refused for room another restart was giving back (S) — done 2026-10-16
|
||||||
|
Not a wait for ever, as first written: on the 4-core VM the judge's restart
|
||||||
|
was *refused* ("fits on no node even with this replica stopped") because the
|
||||||
|
starter, restarted at the same time to reserve fewer cores, still held its
|
||||||
|
old ones. Once refused, nothing tried again, so the judges kept their old
|
||||||
|
figures. Two fixes in `internal/control/reconciler`:
|
||||||
|
- a restart whose replacement fits once other restarts under way have gone
|
||||||
|
(queued, retiring or draining) waits for them, up to DrainGrace, instead
|
||||||
|
of being refused (`fitsAfterRestarts`, restart.go);
|
||||||
|
- a drained replica counts a retiring one that still answers as serving, so
|
||||||
|
one restart after another no longer leaves the first's drained replica,
|
||||||
|
and the room it holds, waiting out its grace (reconciler.go).
|
||||||
|
`TestARestartWaitsForRoomAnotherRestartWillGiveBack` is the 4-core case.
|
||||||
|
|
||||||
|
### J2. A judge the leader adds is sized as init's is (S)
|
||||||
|
When a policy names a judge the catalog lacks, the leader serves it itself
|
||||||
|
(`internal/cli/judgekeeper.go` via `knownJudge.serveRequest` in
|
||||||
|
`judges_known.go`, through modelserve). That judge reserves 3 GB (the
|
||||||
|
weights, an 8192-token KV cache and a fixed gigabyte), no cores, and runs
|
||||||
|
with llama.cpp's prompt cache, which grew one to 3.2 GB. init's judge
|
||||||
|
(`guardModel` in `internal/cli/serverinit_cluster.go`) reserves 2 GB and
|
||||||
|
its cores and runs with `--cache-ram 0`.
|
||||||
|
- `modelserve` gives a guard model `--cache-ram 0`, and the keeper's
|
||||||
|
request the same memory and cores rule as `cpuPlan`.
|
||||||
|
- Done when: a judge removed from the catalog comes back with the same
|
||||||
|
figures init would have given it.
|
||||||
|
|
||||||
|
### J3. HA setup puts the judges on nodes of their own (S)
|
||||||
|
Under load on a 4-core control plane that was also a node, 38 of 60
|
||||||
|
requests were refused for want of a judgment. With the judges on a node of
|
||||||
|
their own (4 cores, 32 GB, two cores each) the same test refused 2.
|
||||||
|
- The HA setup offers a judge node or two: machines given like the store's,
|
||||||
|
added as nodes with a label (`llmgw.role: judge`), and the judge's
|
||||||
|
profile selects that label.
|
||||||
|
- Done when: the setup installs the judges on the machines given for them,
|
||||||
|
and the load test above refuses no more than it did by hand.
|
||||||
|
|
||||||
|
### J4. The model mirror on more than one machine (M)
|
||||||
|
In an HA install the mirror stays on the first machine. While it is down,
|
||||||
|
models the nodes already have keep serving and new ones wait for it.
|
||||||
|
- A second mirror kept in step (each machine of the store, or one more), the
|
||||||
|
control planes and nodes given both, the first that answers used.
|
||||||
|
- Done when: with the first machine powered off, a model added then is
|
||||||
|
fetched and served.
|
||||||
|
|
||||||
|
### J5. An upgrade corrects an earlier install's judge (S) — done 2026-10-16
|
||||||
|
Built in the judge keeper (`internal/cli/judgekeeper.go`, `refit`), on the
|
||||||
|
leader: a judge whose entry is exactly what init before v0.14.7 wrote is
|
||||||
|
given 2 GB, `--cache-ram 0` and cores by init's rule (`refitCores`: up to
|
||||||
|
four for one judge, two each for more), through `modelsvc.SetProfiles`,
|
||||||
|
and its ready replicas are restarted (after J1). Its replica count is left
|
||||||
|
as it is. One that differs is left alone and said once in the log. The
|
||||||
|
original item follows.
|
||||||
|
|
||||||
|
v0.14.6 and earlier installed two judges reserving no cores or one, with
|
||||||
|
2 or 4 GB and the prompt cache on. On fewer than eight cores every judgment
|
||||||
|
missed its 5 s budget and every request was refused. v0.14.7's notes say
|
||||||
|
how to fix it by hand (`ctl model resources`, then `ctl restart`).
|
||||||
|
- On upgrade, or on `server init` run again, a judge whose profile is still
|
||||||
|
exactly what an earlier init wrote is moved to today's figures and
|
||||||
|
restarted (after J1). One changed by hand is left alone and named.
|
||||||
|
- Done when: a v0.14.6 single-machine install upgraded to it judges within
|
||||||
|
budget without anything typed.
|
||||||
|
|
||||||
|
### J6. Rate limits per process (note)
|
||||||
|
Rate limits are counted in each control plane's memory. A failover starts
|
||||||
|
every key's minute afresh. Documented in install.md; worth a shared count in
|
||||||
|
the store only if a customer's limits must hold across a failover.
|
||||||
|
|
||||||
|
### J7. nginx in front of several control planes (S)
|
||||||
|
`deploy/proxy/Caddyfile` sends to whichever control plane answers `/leader`.
|
||||||
|
nginx's open-source build has no active health checks, so its `upstream`
|
||||||
|
block would send to standbys and to a lost leader alike.
|
||||||
|
- nginx.conf: the floating address as its one upstream, and why; or the
|
||||||
|
`max_fails` passive approach with what it costs.
|
||||||
|
|
||||||
|
### J8. A judge budget that fits the machine (S)
|
||||||
|
A judgment takes 2.3 s on four cores and 4.2 on two, against the 5 s
|
||||||
|
default: two cores are close to it, and a busy machine passes it.
|
||||||
|
- At install, after the judge is up, time three judgments. If the slowest is
|
||||||
|
more than half the budget, say so in the plan's closing text with the
|
||||||
|
choices (more cores, a GPU judge, a longer `latency_budget`, `fail:
|
||||||
|
open`). The budget itself is not changed without being asked.
|
||||||
|
- Done when: an install on a machine too slow for its judge says so before
|
||||||
|
its first refused request.
|
||||||
|
|
||||||
## D. Facts worth knowing before starting
|
## D. Facts worth knowing before starting
|
||||||
|
|
||||||
- Admin API: `127.0.0.1:9090`, no auth, `ctlClient` in `internal/cli/ctl.go`.
|
- Admin API: `127.0.0.1:9090`, behind the token in
|
||||||
|
`/var/lib/llmgw/admin-token` (A5), `ctlClient` in `internal/cli/ctl.go`.
|
||||||
Metrics on `:9091` with `/prometheus/targets` beside them.
|
Metrics on `:9091` with `/prometheus/targets` beside them.
|
||||||
- Store: `internal/store`, migrations under `internal/store/migrations/{sqlite,postgres}`;
|
- Store: `internal/store`, migrations under `internal/store/migrations/{sqlite,postgres}`;
|
||||||
the PoC runs PostgreSQL under Patroni.
|
the PoC runs PostgreSQL under Patroni.
|
||||||
|
|||||||
+56
-1
@@ -18,12 +18,67 @@ is only released once its **exit criterion** is met, not when its code lands.
|
|||||||
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
|
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
|
||||||
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
||||||
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
||||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine) |
|
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider) |
|
||||||
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
|
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
|
||||||
|
|
||||||
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
|
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
|
||||||
APIs change between minors without a deprecation period.
|
APIs change between minors without a deprecation period.
|
||||||
|
|
||||||
|
## v0.14.8 — 2026-10-18
|
||||||
|
|
||||||
|
Sign-in with an on-premises Active Directory, and Entra ID made safe for
|
||||||
|
people in many groups. Also several keys for one outside provider, and a
|
||||||
|
measure of what the gateway adds to a request.
|
||||||
|
|
||||||
|
**Upgrading.**
|
||||||
|
- Migration 0048 adds the `user_directory` table, applied when the control
|
||||||
|
plane starts.
|
||||||
|
- Nothing changes until `auth.ldap` is set; docs/install.md, "Active
|
||||||
|
Directory", has an example. It has been tested against a fake directory,
|
||||||
|
not yet against a real domain controller.
|
||||||
|
- A judge installed by an earlier `server init` is refitted by the leader
|
||||||
|
by itself (below); the steps given for v0.14.7 are no longer needed.
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Sign-in with Active Directory** (`auth.ldap`). People sign in on the gateway's form with their AD username or mail
|
||||||
|
and password, checked against the domain controllers over LDAPS or StartTLS. Groups, nested ones included, give
|
||||||
|
staff roles and places in the tree, as with OIDC. Accounts are tied to the `objectGUID` and made at first sign-in
|
||||||
|
when `create_accounts` is on. Every 15 minutes the leader reads the directory. People disabled or deleted there are
|
||||||
|
suspended and reinstated when they come back; roles and memberships follow their groups. A sync that would suspend
|
||||||
|
more than half the accounts is refused. See docs/install.md, "Active Directory".
|
||||||
|
|
||||||
|
- **Several keys for one outside provider.** An external model takes `api_key_files`, each with an optional weight, in
|
||||||
|
place of `api_key_file`. Each key is an endpoint of its own (`external:MODEL#1`, `#2`), so a key answering 429 is
|
||||||
|
passed over while its breaker is open and the request goes to another. A key answering 401 is passed over too, and
|
||||||
|
`external_key_refused` is sent, naming the endpoint, never the key. Usage records which key answered by that name.
|
||||||
|
|
||||||
|
- **`llmgw ctl bench overhead`**: what the gateway adds to a request, measured against an engine that answers at
|
||||||
|
once. Gateway and engine run in one process on loopback, so no cluster is needed. The load is sent straight to the
|
||||||
|
engine and through the gateway: bare, with authentication and accounting, and with the guardrails' pattern checks.
|
||||||
|
It reports p50 and p99 latency, what the gateway added, and a rough CPU per thousand requests. On an Apple M5 the
|
||||||
|
gateway added 0.05–0.42 ms at p50 and at most 0.49 ms at p99 (README).
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
|
||||||
|
- **Entra ID's groups past 200 no longer take a role away.** For someone in more than 200 groups Entra sends none in
|
||||||
|
the token, only a pointer to Graph. That was read as no groups, so an administrator lost their role at the next
|
||||||
|
sign-in. Their role and memberships are now left as they were, and an error is logged saying what to change in the app
|
||||||
|
registration. docs/install.md has an Entra ID section.
|
||||||
|
- **An account made by SCIM or OIDC could not be reinstated.** Setting it active was refused as "never approved".
|
||||||
|
- **An upgrade refits the judge an earlier `server init` installed.** Upgrading to v0.14.7 left such a judge as it was:
|
||||||
|
no cores or one, 2 or 4 GB, the prompt cache on. On fewer than eight cores every request was refused until it was
|
||||||
|
fixed by hand. Now the leader finds a judge whose entry is exactly what an earlier init wrote and gives it 2 GB, no
|
||||||
|
prompt cache and cores by today's rule (up to four for one judge, two each for more). It then restarts the judge's
|
||||||
|
replicas onto that. The number of replicas is left as it is. A judge whose entry was changed by hand is left alone,
|
||||||
|
and the log says so once.
|
||||||
|
- **A restart is no longer refused for room another restart is giving back.** With every core of a node reserved, two
|
||||||
|
models restarted together, one to need more and one less (as after `ctl model resources` on a judge and a starter),
|
||||||
|
had the first refused: "fits on no node even with this replica stopped", though the second was about to give back
|
||||||
|
its cores. A restart now waits for other restarts under way, up to the drain grace, before it is refused. A drained
|
||||||
|
replica also counts a replica being restarted that still answers as serving, so replicas restarted one after
|
||||||
|
another no longer wait out the five-minute grace each.
|
||||||
|
|
||||||
## v0.14.7 — 2026-10-16
|
## v0.14.7 — 2026-10-16
|
||||||
|
|
||||||
The judge installed by `server init` sized for what it does. On a machine
|
The judge installed by `server init` sized for what it does. On a machine
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md).
|
|||||||
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
|
| v0.11.0 | — | High availability from setup, the setup and console screens, usage sums and per-tenant alerts | released |
|
||||||
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
| v0.12.0 | — | The cluster's settings in the store, in force when changed; the firewall switch; wipe | released |
|
||||||
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
|
||||||
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine) |
|
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider) |
|
||||||
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
|
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
|
||||||
|
|
||||||
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
|
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
|
||||||
@@ -257,6 +257,33 @@ Add `--json` to any command for machine-readable output.
|
|||||||
says nothing about which of six constraints bit, so it prints every candidate
|
says nothing about which of six constraints bit, so it prints every candidate
|
||||||
considered and the reason it was rejected.
|
considered and the reason it was rejected.
|
||||||
|
|
||||||
|
## What llmgw adds to a request
|
||||||
|
|
||||||
|
Measured with `llmgw ctl bench overhead`. It starts an engine that answers
|
||||||
|
at once and a gateway in front of it, in one process on loopback. It sends
|
||||||
|
the same load straight to the engine and through the gateway, and reports
|
||||||
|
what the gateway added. Run on an Apple M5 (10 CPUs, load about 2) on
|
||||||
|
2026-10-16, 10 s a step, 72,000 requests with none failed:
|
||||||
|
|
||||||
|
| Through the gateway | added at p50 | added at p99 |
|
||||||
|
|---|---|---|
|
||||||
|
| bare | 0.06–0.24 ms | 0.10–0.33 ms |
|
||||||
|
| with authentication and usage accounting | 0.05–0.32 ms | 0.09–0.37 ms |
|
||||||
|
| with the guardrails' pattern checks as well | 0.10–0.42 ms | 0.15–0.49 ms |
|
||||||
|
|
||||||
|
The range is across 100 to 2,000 requests a second. The overhead was
|
||||||
|
*lowest* at 2,000, most likely because a busier processor keeps its cores
|
||||||
|
awake. The guardrails' judge model is not in these figures: a judgment
|
||||||
|
takes seconds on CPUs (see [install.md](docs/install.md)) and is a model's
|
||||||
|
time, not the gateway's.
|
||||||
|
|
||||||
|
The command also prints the gateway's CPU per thousand requests. Gateway,
|
||||||
|
engine and load share one process there, so that figure is rough (0.04 to
|
||||||
|
0.65 s per thousand in the run above); the latency is the one to quote.
|
||||||
|
Figures from a VM on a busy host move with the host: run it where you will
|
||||||
|
deploy, and keep the load it prints beside the result. CI runs a one-second
|
||||||
|
version on every change (`TestTheGatewaysOverheadIsMeasured`).
|
||||||
|
|
||||||
## Running the router
|
## Running the router
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|||||||
@@ -9054,6 +9054,13 @@
|
|||||||
"type": "string",
|
"type": "string",
|
||||||
"description": "APIKeyFile holds the provider's key, on the control plane. A file rather than the key itself, so the key is in neither the catalog, the store nor the audit log; empty for a provider that asks for none."
|
"description": "APIKeyFile holds the provider's key, on the control plane. A file rather than the key itself, so the key is in neither the catalog, the store nor the audit log; empty for a provider that asks for none."
|
||||||
},
|
},
|
||||||
|
"api_key_files": {
|
||||||
|
"type": "array",
|
||||||
|
"items": {
|
||||||
|
"$ref": "#/components/schemas/catalog.ExternalKey"
|
||||||
|
},
|
||||||
|
"description": "APIKeyFiles are several keys for the provider, in place of APIKeyFile: one endpoint each (external:MODEL#1, #2…), so a key that answers 429 or 401 is passed over while its breaker is open and the request goes to another. A provider's limits are per key."
|
||||||
|
},
|
||||||
"peer": {
|
"peer": {
|
||||||
"type": "string",
|
"type": "string",
|
||||||
"description": "Peer names the federated llmgw cluster this model is offered by, for a model that federation added (`llmgw ctl federation add`) rather than one written by hand. Such a model is named PEER/MODEL, comes and goes with the peer's own list, and is sent only while the peer is up."
|
"description": "Peer names the federated llmgw cluster this model is offered by, for a model that federation added (`llmgw ctl federation add`) rather than one written by hand. Such a model is named PEER/MODEL, comes and goes with the peer's own list, and is sent only while the peer is up."
|
||||||
@@ -9072,6 +9079,20 @@
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"catalog.ExternalKey": {
|
||||||
|
"type": "object",
|
||||||
|
"description": "ExternalKey is one of a provider's keys.",
|
||||||
|
"properties": {
|
||||||
|
"file": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "File holds the key, on the control plane, as APIKeyFile does."
|
||||||
|
},
|
||||||
|
"weight": {
|
||||||
|
"type": "number",
|
||||||
|
"description": "Weight is its share of the requests against the others'; zero is 1."
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
"catalog.GPUNeed": {
|
"catalog.GPUNeed": {
|
||||||
"type": "object",
|
"type": "object",
|
||||||
"description": "GPUNeed is what card a profile needs.",
|
"description": "GPUNeed is what card a profile needs.",
|
||||||
|
|||||||
@@ -1323,6 +1323,26 @@ catalog, and a key rotated in the file is used within seconds. A model whose key
|
|||||||
file cannot be read is not offered. Everything else applies as to any model:
|
file cannot be read is not offered. Everything else applies as to any model:
|
||||||
prices and budgets, rate limits, usage records, and key allowlists.
|
prices and budgets, rate limits, usage records, and key allowlists.
|
||||||
|
|
||||||
|
A provider's limits are per key. With several keys, one answering 429 need not
|
||||||
|
hold up the model: give them all, each with an optional weight (its share
|
||||||
|
against the others'), in place of `api_key_file`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
external:
|
||||||
|
url: https://api.openai.com
|
||||||
|
model: gpt-4o-mini
|
||||||
|
api_key_files:
|
||||||
|
- {file: /etc/llmgw/secrets/openai-1.key, weight: 2}
|
||||||
|
- {file: /etc/llmgw/secrets/openai-2.key}
|
||||||
|
```
|
||||||
|
|
||||||
|
Each key is an endpoint of its own, `external:cloud-mini#1`, `#2`. A key that
|
||||||
|
answers 429 is passed over while its breaker is open, and the request goes to
|
||||||
|
another. One that answers 401 (revoked, expired, mistyped) is passed over too,
|
||||||
|
and `external_key_refused` goes to the notification webhooks, naming the
|
||||||
|
endpoint and never the key. The usage records say which key answered by the
|
||||||
|
same name. A key whose file cannot be read is left out, and the others stay.
|
||||||
|
|
||||||
**Its prompts leave the cluster.** Decide who may send them: a key or project
|
**Its prompts leave the cluster.** Decide who may send them: a key or project
|
||||||
whose allowlist leaves the external model out cannot reach it, as a fallback or
|
whose allowlist leaves the external model out cannot reach it, as a fallback or
|
||||||
by name. The cluster page and `ctl models` mark it `external`.
|
by name. The cluster page and `ctl models` mark it `external`.
|
||||||
@@ -2521,6 +2541,82 @@ and point it at `https://<gateway>/scim/v2` with that token as the bearer.
|
|||||||
join and leave them. Membership in a group is removed when someone leaves
|
join and leave them. Membership in a group is removed when someone leaves
|
||||||
it.
|
it.
|
||||||
|
|
||||||
|
### Entra ID
|
||||||
|
|
||||||
|
Entra ID is an OpenID Connect provider like the others, with two things to
|
||||||
|
set in the app registration:
|
||||||
|
|
||||||
|
1. **Redirect URI:** under Authentication, add a Web platform with
|
||||||
|
`https://<gateway>/login/oidc/callback`. The issuer is
|
||||||
|
`https://login.microsoftonline.com/<tenant-id>/v2.0`.
|
||||||
|
2. **Groups in the token:** under Token configuration, add a groups claim.
|
||||||
|
Choose **Groups assigned to the application**, and assign the groups you
|
||||||
|
map in `roles` and `memberships` to the enterprise application. Entra
|
||||||
|
sends groups by object id unless you choose to emit names
|
||||||
|
(`sAMAccountName` for groups synced from AD), so write them the same way
|
||||||
|
in the mapping.
|
||||||
|
|
||||||
|
Entra leaves the groups out of the token for someone in more than 200 of
|
||||||
|
them and sends a pointer to Graph instead. llmgw then keeps that person's
|
||||||
|
role and memberships as they were, rather than reading it as no groups, and
|
||||||
|
logs an error saying so. "Groups assigned to the application" avoids this.
|
||||||
|
So does SCIM below, which Entra also speaks: in the enterprise application
|
||||||
|
under Provisioning, set the tenant URL to `https://<gateway>/scim/v2` and
|
||||||
|
the secret token to the contents of `auth.scim.token_file`.
|
||||||
|
|
||||||
|
## Active Directory
|
||||||
|
|
||||||
|
With an on-premises Active Directory, people sign in on the gateway's own
|
||||||
|
form with their AD username (`ada`, `CORP\ada`, `ada@corp.example`) and
|
||||||
|
password. The gateway checks them with the domain controllers over LDAPS:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
auth:
|
||||||
|
enabled: true
|
||||||
|
ldap:
|
||||||
|
urls: [ldaps://dc1.corp.example, ldaps://dc2.corp.example] # tried in order
|
||||||
|
ca_file: /etc/llmgw/corp-ca.pem # the domain's CA, when not in the system's
|
||||||
|
bind_dn: CN=svc-llmgw,OU=Service,DC=corp,DC=example
|
||||||
|
bind_password_file: /etc/llmgw/secrets/ldap-bind # root-only
|
||||||
|
base_dn: OU=Staff,DC=corp,DC=example # where people are
|
||||||
|
user_filter: (memberOf:1.2.840.113556.1.4.1941:=CN=llmgw-users,OU=Groups,DC=corp,DC=example)
|
||||||
|
label: Acme # "Sign in with your Acme account"
|
||||||
|
create_accounts: true
|
||||||
|
roles: # by group CN or full DN; nested groups count
|
||||||
|
admin: [llmgw-admins]
|
||||||
|
operator: [llmgw-ops]
|
||||||
|
memberships:
|
||||||
|
ml-engineers: "team:Acme/ml"
|
||||||
|
sync_interval: 15m # negative turns the sync off
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Service account:** an ordinary domain user is enough: it only reads.
|
||||||
|
It finds the person and reads their groups. The person's password is
|
||||||
|
checked by binding as them, and is never stored.
|
||||||
|
- **Encryption:** `ldaps://`, or `ldap://` with `start_tls: true`.
|
||||||
|
Plaintext LDAP is refused, since passwords would cross the network.
|
||||||
|
- **Who may sign in:** enabled user objects under `base_dn` that match
|
||||||
|
`user_filter`. The example admits members of `llmgw-users`, nested groups
|
||||||
|
included.
|
||||||
|
- **Accounts:** an account is tied to the person's `objectGUID`, so a changed
|
||||||
|
mail or name is still the same account. An account that already exists
|
||||||
|
with the person's mail is taken over. Without `create_accounts`, only
|
||||||
|
people who already have an account can sign in. Local accounts, such as
|
||||||
|
the administrator `server init` made, keep signing in with their own
|
||||||
|
password.
|
||||||
|
- **Sync:** every `sync_interval` the leader reads the directory.
|
||||||
|
- Someone disabled or deleted in AD is suspended and their sessions
|
||||||
|
end.
|
||||||
|
- When they are enabled again they are reinstated. An account an
|
||||||
|
administrator suspended by hand stays suspended.
|
||||||
|
- Roles follow groups, and memberships the directory gave are taken away
|
||||||
|
when the person leaves the group.
|
||||||
|
- A sync that would suspend more than half of the directory's accounts is
|
||||||
|
refused and logged, rather than locking everyone out over a wrong
|
||||||
|
`base_dn` or filter.
|
||||||
|
- **Passwords:** a person's password is changed in AD. The gateway sends no
|
||||||
|
reset link for these accounts.
|
||||||
|
|
||||||
## Secrets in Vault
|
## Secrets in Vault
|
||||||
|
|
||||||
Every setting that names a file holding a secret — the store's `@address`,
|
Every setting that names a file holding a secret — the store's `@address`,
|
||||||
|
|||||||
@@ -37,6 +37,7 @@ rather than repeated.
|
|||||||
| `default_tpm` | `int` | [Auth](#auth) |
|
| `default_tpm` | `int` | [Auth](#auth) |
|
||||||
| `signup_open` | `bool` | [Auth](#auth) |
|
| `signup_open` | `bool` | [Auth](#auth) |
|
||||||
| `oidc` | `OIDC` | [Auth](#auth) |
|
| `oidc` | `OIDC` | [Auth](#auth) |
|
||||||
|
| `ldap` | `LDAP` | [Auth](#auth) |
|
||||||
| `scim` | `SCIM` | [Auth](#auth) |
|
| `scim` | `SCIM` | [Auth](#auth) |
|
||||||
| `max_key_ttl` | `time.Duration` | [Auth](#auth) |
|
| `max_key_ttl` | `time.Duration` | [Auth](#auth) |
|
||||||
| `personal` | `Personal` | [Auth](#auth) |
|
| `personal` | `Personal` | [Auth](#auth) |
|
||||||
@@ -105,10 +106,13 @@ rather than repeated.
|
|||||||
| `url` | `string` | [External](#external) |
|
| `url` | `string` | [External](#external) |
|
||||||
| `model` | `string` | [External](#external) |
|
| `model` | `string` | [External](#external) |
|
||||||
| `api_key_file` | `string` | [External](#external) |
|
| `api_key_file` | `string` | [External](#external) |
|
||||||
|
| `api_key_files` | `[]ExternalKey` | [External](#external) |
|
||||||
| `peer` | `string` | [External](#external) |
|
| `peer` | `string` | [External](#external) |
|
||||||
| `api` | `string` | [External](#external) |
|
| `api` | `string` | [External](#external) |
|
||||||
| `workflow` | `string` | [External](#external) |
|
| `workflow` | `string` | [External](#external) |
|
||||||
| `output` | `string` | [External](#external) |
|
| `output` | `string` | [External](#external) |
|
||||||
|
| `file` | `string` | [ExternalKey](#externalkey) |
|
||||||
|
| `weight` | `float64` | [ExternalKey](#externalkey) |
|
||||||
| `budgets` | `[]FacilityBudget` | [Facility](#facility) |
|
| `budgets` | `[]FacilityBudget` | [Facility](#facility) |
|
||||||
| `margin` | `float64` | [Facility](#facility) |
|
| `margin` | `float64` | [Facility](#facility) |
|
||||||
| `gpu_watts` | `float64` | [Facility](#facility) |
|
| `gpu_watts` | `float64` | [Facility](#facility) |
|
||||||
@@ -144,6 +148,20 @@ rather than repeated.
|
|||||||
| `tokenize_timeout` | `time.Duration` | [KVIndex](#kvindex) |
|
| `tokenize_timeout` | `time.Duration` | [KVIndex](#kvindex) |
|
||||||
| `min_blocks` | `int` | [KVIndex](#kvindex) |
|
| `min_blocks` | `int` | [KVIndex](#kvindex) |
|
||||||
| `max_blocks` | `int` | [KVIndex](#kvindex) |
|
| `max_blocks` | `int` | [KVIndex](#kvindex) |
|
||||||
|
| `urls` | `[]string` | [LDAP](#ldap) |
|
||||||
|
| `start_tls` | `bool` | [LDAP](#ldap) |
|
||||||
|
| `ca_file` | `string` | [LDAP](#ldap) |
|
||||||
|
| `bind_dn` | `string` | [LDAP](#ldap) |
|
||||||
|
| `bind_password_file` | `string` | [LDAP](#ldap) |
|
||||||
|
| `base_dn` | `string` | [LDAP](#ldap) |
|
||||||
|
| `group_base_dn` | `string` | [LDAP](#ldap) |
|
||||||
|
| `user_filter` | `string` | [LDAP](#ldap) |
|
||||||
|
| `label` | `string` | [LDAP](#ldap) |
|
||||||
|
| `create_accounts` | `bool` | [LDAP](#ldap) |
|
||||||
|
| `roles` | `map[string][]string` | [LDAP](#ldap) |
|
||||||
|
| `memberships` | `map[string]string` | [LDAP](#ldap) |
|
||||||
|
| `sync_interval` | `time.Duration` | [LDAP](#ldap) |
|
||||||
|
| `timeout` | `time.Duration` | [LDAP](#ldap) |
|
||||||
| `contact` | `string` | [Landing](#landing) |
|
| `contact` | `string` | [Landing](#landing) |
|
||||||
| `notice` | `string` | [Landing](#landing) |
|
| `notice` | `string` | [Landing](#landing) |
|
||||||
| `language` | `string` | [Landing](#landing) |
|
| `language` | `string` | [Landing](#landing) |
|
||||||
@@ -616,6 +634,15 @@ OIDC is sign-in through an identity provider.
|
|||||||
|
|
||||||
Takes a [OIDC](#oidc) block.
|
Takes a [OIDC](#oidc) block.
|
||||||
|
|
||||||
|
### `ldap`
|
||||||
|
|
||||||
|
`LDAP`
|
||||||
|
|
||||||
|
LDAP is sign-in against a directory, Active Directory above all, and
|
||||||
|
the accounts kept in step with it.
|
||||||
|
|
||||||
|
Takes a [LDAP](#ldap) block.
|
||||||
|
|
||||||
### `scim`
|
### `scim`
|
||||||
|
|
||||||
`SCIM`
|
`SCIM`
|
||||||
@@ -1103,6 +1130,15 @@ APIKeyFile holds the provider's key, on the control plane. A file
|
|||||||
rather than the key itself, so the key is in neither the catalog, the
|
rather than the key itself, so the key is in neither the catalog, the
|
||||||
store nor the audit log; empty for a provider that asks for none.
|
store nor the audit log; empty for a provider that asks for none.
|
||||||
|
|
||||||
|
### `api_key_files`
|
||||||
|
|
||||||
|
`[]ExternalKey`
|
||||||
|
|
||||||
|
APIKeyFiles are several keys for the provider, in place of
|
||||||
|
APIKeyFile: one endpoint each (external:MODEL#1, #2…), so a key that
|
||||||
|
answers 429 or 401 is passed over while its breaker is open and the
|
||||||
|
request goes to another. A provider's limits are per key.
|
||||||
|
|
||||||
### `peer`
|
### `peer`
|
||||||
|
|
||||||
`string`
|
`string`
|
||||||
@@ -1135,6 +1171,20 @@ plane.
|
|||||||
|
|
||||||
Output is what it makes: pictures, or meshes (see ComfyUI.Output).
|
Output is what it makes: pictures, or meshes (see ComfyUI.Output).
|
||||||
|
|
||||||
|
## ExternalKey
|
||||||
|
|
||||||
|
### `file`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
File holds the key, on the control plane, as APIKeyFile does.
|
||||||
|
|
||||||
|
### `weight`
|
||||||
|
|
||||||
|
`float64`
|
||||||
|
|
||||||
|
Weight is its share of the requests against the others'; zero is 1.
|
||||||
|
|
||||||
## Facility
|
## Facility
|
||||||
|
|
||||||
### `budgets`
|
### `budgets`
|
||||||
@@ -1401,6 +1451,106 @@ MaxBlocks bounds how many of a prompt's blocks are hashed and looked
|
|||||||
up. Beyond it a longer match still helps, but the difference no longer
|
up. Beyond it a longer match still helps, but the difference no longer
|
||||||
decides anything. Zero takes 4096.
|
decides anything. Zero takes 4096.
|
||||||
|
|
||||||
|
## LDAP
|
||||||
|
|
||||||
|
### `urls`
|
||||||
|
|
||||||
|
`[]string`
|
||||||
|
|
||||||
|
URLs are the directory's servers, tried in turn: ldaps:// (636), or
|
||||||
|
ldap:// (389) with StartTLS.
|
||||||
|
|
||||||
|
### `start_tls`
|
||||||
|
|
||||||
|
`bool`
|
||||||
|
|
||||||
|
StartTLS upgrades an ldap:// connection before anything is sent. A
|
||||||
|
password never crosses a connection without TLS: ldap:// without it
|
||||||
|
is refused.
|
||||||
|
|
||||||
|
### `ca_file`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
CAFile is the authority the servers' certificates are checked against,
|
||||||
|
when it is not the system's: an AD Certificate Services root.
|
||||||
|
|
||||||
|
### `bind_dn`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
BindDN and the file holding its password: a service account that may
|
||||||
|
read users and groups. Nothing is written to the directory.
|
||||||
|
|
||||||
|
### `bind_password_file`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
### `base_dn`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
BaseDN is where people are searched for, subtree included.
|
||||||
|
|
||||||
|
### `group_base_dn`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
GroupBaseDN is where their groups are; empty is BaseDN's domain root.
|
||||||
|
|
||||||
|
### `user_filter`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
UserFilter narrows who may sign in, ANDed with the login match: a
|
||||||
|
group (memberOf=...) or an OU's people only. Empty is every enabled
|
||||||
|
user object.
|
||||||
|
|
||||||
|
### `label`
|
||||||
|
|
||||||
|
`string`
|
||||||
|
|
||||||
|
Label is what the form says to sign in with: "Sign in with your
|
||||||
|
<label> account". Empty is "Active Directory".
|
||||||
|
|
||||||
|
### `create_accounts`
|
||||||
|
|
||||||
|
`bool`
|
||||||
|
|
||||||
|
CreateAccounts makes an account, active, for a person the directory
|
||||||
|
vouches for and the gateway does not know. Off, only existing accounts
|
||||||
|
sign in this way.
|
||||||
|
|
||||||
|
### `roles`
|
||||||
|
|
||||||
|
`map[string][]string`
|
||||||
|
|
||||||
|
Roles maps a staff role (viewer, operator, billing, admin) to the
|
||||||
|
groups that give it, by name (CN) or whole DN. Set at every sign-in
|
||||||
|
and sync: someone taken out of the group loses the role.
|
||||||
|
|
||||||
|
### `memberships`
|
||||||
|
|
||||||
|
`map[string]string`
|
||||||
|
|
||||||
|
Memberships maps a group, by CN or DN, to a place in the tree —
|
||||||
|
org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT — joined at sign-in
|
||||||
|
and kept by the sync, which also takes away a place whose group the
|
||||||
|
person has left.
|
||||||
|
|
||||||
|
### `sync_interval`
|
||||||
|
|
||||||
|
`time.Duration`
|
||||||
|
|
||||||
|
SyncInterval is how often the leader reads the directory. Zero is
|
||||||
|
15m; negative turns the sync off, leaving only sign-in.
|
||||||
|
|
||||||
|
### `timeout`
|
||||||
|
|
||||||
|
`time.Duration`
|
||||||
|
|
||||||
|
Timeout bounds each round trip to a server. Zero is 10s.
|
||||||
|
|
||||||
## Landing
|
## Landing
|
||||||
|
|
||||||
### `contact`
|
### `contact`
|
||||||
|
|||||||
@@ -12,6 +12,7 @@ require (
|
|||||||
github.com/charmbracelet/x/ansi v0.11.6
|
github.com/charmbracelet/x/ansi v0.11.6
|
||||||
github.com/coreos/go-oidc/v3 v3.21.0
|
github.com/coreos/go-oidc/v3 v3.21.0
|
||||||
github.com/go-jose/go-jose/v4 v4.1.4
|
github.com/go-jose/go-jose/v4 v4.1.4
|
||||||
|
github.com/go-ldap/ldap/v3 v3.4.15
|
||||||
github.com/go-zeromq/zmq4 v0.17.0
|
github.com/go-zeromq/zmq4 v0.17.0
|
||||||
github.com/jackc/pgx/v5 v5.11.0
|
github.com/jackc/pgx/v5 v5.11.0
|
||||||
github.com/knadh/koanf/parsers/yaml v1.1.1
|
github.com/knadh/koanf/parsers/yaml v1.1.1
|
||||||
@@ -45,6 +46,7 @@ require (
|
|||||||
require (
|
require (
|
||||||
filippo.io/edwards25519 v1.2.0 // indirect
|
filippo.io/edwards25519 v1.2.0 // indirect
|
||||||
filippo.io/hpke v0.4.0 // indirect
|
filippo.io/hpke v0.4.0 // indirect
|
||||||
|
github.com/Azure/go-ntlmssp v0.1.1 // indirect
|
||||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
|
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
|
||||||
github.com/beorn7/perks v1.0.1 // indirect
|
github.com/beorn7/perks v1.0.1 // indirect
|
||||||
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
|
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
|
||||||
@@ -57,6 +59,7 @@ require (
|
|||||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||||
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
|
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f // indirect
|
||||||
github.com/fsnotify/fsnotify v1.9.0 // indirect
|
github.com/fsnotify/fsnotify v1.9.0 // indirect
|
||||||
|
github.com/go-asn1-ber/asn1-ber v1.5.8 // indirect
|
||||||
github.com/go-logr/logr v1.4.4 // indirect
|
github.com/go-logr/logr v1.4.4 // indirect
|
||||||
github.com/go-logr/stdr v1.2.2 // indirect
|
github.com/go-logr/stdr v1.2.2 // indirect
|
||||||
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
|
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
|
||||||
|
|||||||
@@ -8,8 +8,12 @@ filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
|
|||||||
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
|
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
|
||||||
filippo.io/hpke v0.4.0 h1:p575VVQ6ted4pL+it6M00V/f2qTZITO0zgmdKCkd5+A=
|
filippo.io/hpke v0.4.0 h1:p575VVQ6ted4pL+it6M00V/f2qTZITO0zgmdKCkd5+A=
|
||||||
filippo.io/hpke v0.4.0/go.mod h1:EmAN849/P3qdeK+PCMkDpDm83vRHM5cDipBJ8xbQLVY=
|
filippo.io/hpke v0.4.0/go.mod h1:EmAN849/P3qdeK+PCMkDpDm83vRHM5cDipBJ8xbQLVY=
|
||||||
|
github.com/Azure/go-ntlmssp v0.1.1 h1:l+FM/EEMb0U9QZE7mKNEDw5Mu3mFiaa2GKOoTSsNDPw=
|
||||||
|
github.com/Azure/go-ntlmssp v0.1.1/go.mod h1:NYqdhxd/8aAct/s4qSYZEerdPuH1liG2/X9DiVTbhpk=
|
||||||
github.com/NVIDIA/go-nvml v0.13.4-0 h1:o3jp9u2x1R9ShFE3v+Aesp55XOSIQFMJz/VGNUcJaNE=
|
github.com/NVIDIA/go-nvml v0.13.4-0 h1:o3jp9u2x1R9ShFE3v+Aesp55XOSIQFMJz/VGNUcJaNE=
|
||||||
github.com/NVIDIA/go-nvml v0.13.4-0/go.mod h1:id63qwpoDWpFXwnwM6psDCSqW4BmNu6mWpr4YeQtPGo=
|
github.com/NVIDIA/go-nvml v0.13.4-0/go.mod h1:id63qwpoDWpFXwnwM6psDCSqW4BmNu6mWpr4YeQtPGo=
|
||||||
|
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e h1:4dAU9FXIyQktpoUAgOJK3OTFc/xug0PCXYCqU0FgDKI=
|
||||||
|
github.com/alexbrainman/sspi v0.0.0-20250919150558-7d374ff0d59e/go.mod h1:cEWa1LVoE5KvSD9ONXsZrj0z6KqySlCCNKHlLzbqAt4=
|
||||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
|
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
|
||||||
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
|
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
|
||||||
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
|
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
|
||||||
@@ -46,8 +50,12 @@ github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f h1:Y/CXytFA4m6
|
|||||||
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
|
github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97MGsxSvLiUE2X8qFplwetxpGLQrlU1Q9AUEIzCaM=
|
||||||
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
|
||||||
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
|
||||||
|
github.com/go-asn1-ber/asn1-ber v1.5.8 h1:H9AZkK22UOmfX8J84ubyaZxKJZ3FMHVwn8swoMML7iQ=
|
||||||
|
github.com/go-asn1-ber/asn1-ber v1.5.8/go.mod h1:hEBeB/ic+5LoWskz+yKT7vGhhPYkProFKoKdwZRWMe0=
|
||||||
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
|
github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
|
||||||
github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
|
github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
|
||||||
|
github.com/go-ldap/ldap/v3 v3.4.15 h1:eRB4f4Ecn1POL1mO86+uUDlecmyWsdiJwh60bAUS4UY=
|
||||||
|
github.com/go-ldap/ldap/v3 v3.4.15/go.mod h1:jeD0c98EdZQCKKKKe4kTWs5aCf6y+j8KuYsYLznZSQk=
|
||||||
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||||
github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8=
|
github.com/go-logr/logr v1.4.4 h1:tG4xh9yMsRCAiodLVTxyrkzSZ9+o0L1Kg/+cPVcbP/8=
|
||||||
github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
github.com/go-logr/logr v1.4.4/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||||
@@ -69,6 +77,8 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
|||||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.30.0 h1:/Tnpcb2E0Pz/tN9s3bfEY2Q8ePCEX9iuS+cneUwncnw=
|
github.com/grpc-ecosystem/grpc-gateway/v2 v2.30.0 h1:/Tnpcb2E0Pz/tN9s3bfEY2Q8ePCEX9iuS+cneUwncnw=
|
||||||
github.com/grpc-ecosystem/grpc-gateway/v2 v2.30.0/go.mod h1:zOBXOsUaBSjKgmH4OGzV1esUpR3oUSCPYVd2cUBjKYY=
|
github.com/grpc-ecosystem/grpc-gateway/v2 v2.30.0/go.mod h1:zOBXOsUaBSjKgmH4OGzV1esUpR3oUSCPYVd2cUBjKYY=
|
||||||
|
github.com/hashicorp/go-uuid v1.0.3 h1:2gKiV6YVmrJ1i2CKKa9obLvRieoRGviZFL26PcT/Co8=
|
||||||
|
github.com/hashicorp/go-uuid v1.0.3/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro=
|
||||||
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
|
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
|
||||||
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
|
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
|
||||||
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
|
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
|
||||||
@@ -81,6 +91,18 @@ github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg=
|
|||||||
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
|
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
|
||||||
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
|
||||||
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
|
||||||
|
github.com/jcmturner/aescts/v2 v2.0.0 h1:9YKLH6ey7H4eDBXW8khjYslgyqG2xZikXP0EQFKrle8=
|
||||||
|
github.com/jcmturner/aescts/v2 v2.0.0/go.mod h1:AiaICIRyfYg35RUkr8yESTqvSy7csK90qZ5xfvvsoNs=
|
||||||
|
github.com/jcmturner/dnsutils/v2 v2.0.0 h1:lltnkeZGL0wILNvrNiVCR6Ro5PGU/SeBvVO/8c/iPbo=
|
||||||
|
github.com/jcmturner/dnsutils/v2 v2.0.0/go.mod h1:b0TnjGOvI/n42bZa+hmXL+kFJZsFT7G4t3HTlQ184QM=
|
||||||
|
github.com/jcmturner/gofork v1.7.6 h1:QH0l3hzAU1tfT3rZCnW5zXl+orbkNMMRGJfdJjHVETg=
|
||||||
|
github.com/jcmturner/gofork v1.7.6/go.mod h1:1622LH6i/EZqLloHfE7IeZ0uEJwMSUyQ/nDd82IeqRo=
|
||||||
|
github.com/jcmturner/goidentity/v6 v6.0.1 h1:VKnZd2oEIMorCTsFBnJWbExfNN7yZr3EhJAxwOkZg6o=
|
||||||
|
github.com/jcmturner/goidentity/v6 v6.0.1/go.mod h1:X1YW3bgtvwAXju7V3LCIMpY0Gbxyjn/mY9zx4tFonSg=
|
||||||
|
github.com/jcmturner/gokrb5/v8 v8.4.4 h1:x1Sv4HaTpepFkXbt2IkL29DXRf8sOfZXo8eRKh687T8=
|
||||||
|
github.com/jcmturner/gokrb5/v8 v8.4.4/go.mod h1:1btQEpgT6k+unzCwX1KdWMEwPPkkgBtP+F6aCACiMrs=
|
||||||
|
github.com/jcmturner/rpc/v2 v2.0.3 h1:7FXXj8Ti1IaVFpSAziCZWNzbNuZmnvw/i6CqLNdWfZY=
|
||||||
|
github.com/jcmturner/rpc/v2 v2.0.3/go.mod h1:VUJYCIDm3PVOEHw8sgt091/20OJjskO/YJki3ELg/Hc=
|
||||||
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
||||||
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||||
github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo=
|
github.com/knadh/koanf/maps v0.1.2 h1:RBfmAW5CnZT+PJ1CVc1QSJKf4Xu9kxfQgYVQSu8hpbo=
|
||||||
|
|||||||
@@ -0,0 +1,63 @@
|
|||||||
|
package account
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
type fakeDirectory struct {
|
||||||
|
asked []string
|
||||||
|
u store.User
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeDirectory) SignIn(_ context.Context, login, _ string) (store.User, error) {
|
||||||
|
f.asked = append(f.asked, login)
|
||||||
|
return f.u, f.err
|
||||||
|
}
|
||||||
|
|
||||||
|
// With a directory, a name the store has no password for is the
|
||||||
|
// directory's to check; an account with a password here — the
|
||||||
|
// administrator init made — is still checked here, so a directory that is
|
||||||
|
// down locks nobody out.
|
||||||
|
func TestPasswordsKeptElsewhereAreTheDirectorysToCheck(t *testing.T) {
|
||||||
|
svc, st := newService(t)
|
||||||
|
ctx := t.Context()
|
||||||
|
hash, _ := HashPassword("local-secret")
|
||||||
|
admin := store.User{ID: "usr-admin", Email: "admin@example.com", Name: "a", PasswordHash: hash,
|
||||||
|
State: store.UserActive, CreatedAt: time.Now()}
|
||||||
|
ada := store.User{ID: "usr-ada", Email: "ada@corp.example", Name: "Ada", PasswordHash: "!ldap",
|
||||||
|
State: store.UserActive, CreatedAt: time.Now()}
|
||||||
|
for _, u := range []store.User{admin, ada} {
|
||||||
|
if err := st.CreateUser(ctx, u); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
dir := &fakeDirectory{u: ada}
|
||||||
|
svc.SetDirectory(dir)
|
||||||
|
|
||||||
|
if _, _, err := svc.SignIn(ctx, "admin@example.com", "local-secret"); err != nil || len(dir.asked) != 0 {
|
||||||
|
t.Fatalf("the local administrator: %v, directory asked %v", err, dir.asked)
|
||||||
|
}
|
||||||
|
// A name, not an address, and an account kept by the directory.
|
||||||
|
for _, login := range []string{`CORP\ada`, "ada@corp.example"} {
|
||||||
|
if tok, _, err := svc.SignIn(ctx, login, "ad-secret"); err != nil || tok == "" {
|
||||||
|
t.Errorf("%s: %v", login, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(dir.asked) != 2 || dir.asked[0] != `CORP\ada` {
|
||||||
|
t.Errorf("the directory was asked %v", dir.asked)
|
||||||
|
}
|
||||||
|
dir.err = ErrBadCredentials
|
||||||
|
if _, _, err := svc.SignIn(ctx, "ada", "wrong"); !errors.Is(err, ErrBadCredentials) {
|
||||||
|
t.Errorf("a password the directory refused: %v", err)
|
||||||
|
}
|
||||||
|
// A reset link would give a directory's account a password here.
|
||||||
|
if _, _, err := svc.RequestReset(ctx, st, "ada@corp.example"); !errors.Is(err, ErrNoReset) {
|
||||||
|
t.Errorf("a directory's account was sent a reset link: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -80,8 +80,26 @@ type Service struct {
|
|||||||
|
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
attempts map[string]*attemptWindow
|
attempts map[string]*attemptWindow
|
||||||
|
|
||||||
|
// dir checks the passwords of accounts this store keeps none for.
|
||||||
|
dir Directory
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Directory checks a password against an outside directory — Active
|
||||||
|
// Directory over LDAP — and answers the account to sign in: made or kept in
|
||||||
|
// step with the directory. Its errors are this package's ErrBadCredentials
|
||||||
|
// and ErrSuspended for the person's own, and anything else for the
|
||||||
|
// directory's.
|
||||||
|
type Directory interface {
|
||||||
|
SignIn(ctx context.Context, login, password string) (store.User, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetDirectory has passwords for accounts with none here, and names that are
|
||||||
|
// not addresses, checked by the directory. An account with a password here
|
||||||
|
// — the administrator init made — is still checked here, so a directory that
|
||||||
|
// is down locks nobody out.
|
||||||
|
func (s *Service) SetDirectory(d Directory) { s.dir = d }
|
||||||
|
|
||||||
type attemptWindow struct {
|
type attemptWindow struct {
|
||||||
count int
|
count int
|
||||||
since time.Time
|
since time.Time
|
||||||
@@ -137,6 +155,7 @@ func (s *Service) SignUp(ctx context.Context, email, name, password string) erro
|
|||||||
// SignIn checks a password and opens a session, returning the session token and
|
// SignIn checks a password and opens a session, returning the session token and
|
||||||
// the CSRF token that must accompany the forms in it.
|
// the CSRF token that must accompany the forms in it.
|
||||||
func (s *Service) SignIn(ctx context.Context, email, password string) (token, csrf string, err error) {
|
func (s *Service) SignIn(ctx context.Context, email, password string) (token, csrf string, err error) {
|
||||||
|
login := strings.TrimSpace(email)
|
||||||
email = store.NormalizeEmail(email)
|
email = store.NormalizeEmail(email)
|
||||||
// Counted per address and the client trying it, so somebody guessing
|
// Counted per address and the client trying it, so somebody guessing
|
||||||
// from elsewhere cannot lock the owner out of their own account; and per
|
// from elsewhere cannot lock the owner out of their own account; and per
|
||||||
@@ -151,6 +170,15 @@ func (s *Service) SignIn(ctx context.Context, email, password string) (token, cs
|
|||||||
}
|
}
|
||||||
|
|
||||||
u, err := s.store.UserByEmail(ctx, email)
|
u, err := s.store.UserByEmail(ctx, email)
|
||||||
|
// No password kept here — the directory's, or nobody's — and a
|
||||||
|
// directory to ask: it decides.
|
||||||
|
if s.dir != nil && (errors.Is(err, store.ErrNotFound) || (err == nil && strings.HasPrefix(u.PasswordHash, "!"))) {
|
||||||
|
if u, err = s.dir.SignIn(ctx, login, password); err != nil {
|
||||||
|
return "", "", err
|
||||||
|
}
|
||||||
|
s.clearAttempts(byClient)
|
||||||
|
return s.open(ctx, u)
|
||||||
|
}
|
||||||
if errors.Is(err, store.ErrNotFound) {
|
if errors.Is(err, store.ErrNotFound) {
|
||||||
// Verify anyway, against a hash that cannot match. Returning early here
|
// Verify anyway, against a hash that cannot match. Returning early here
|
||||||
// would let a stopwatch enumerate every account on the system.
|
// would let a stopwatch enumerate every account on the system.
|
||||||
@@ -180,7 +208,11 @@ func (s *Service) SignIn(ctx context.Context, email, password string) (token, cs
|
|||||||
case store.UserSuspended:
|
case store.UserSuspended:
|
||||||
return "", "", ErrSuspended
|
return "", "", ErrSuspended
|
||||||
}
|
}
|
||||||
|
return s.open(ctx, u)
|
||||||
|
}
|
||||||
|
|
||||||
|
// open makes a session for a person whose password was right.
|
||||||
|
func (s *Service) open(ctx context.Context, u store.User) (token, csrf string, err error) {
|
||||||
if token, err = store.NewSessionToken(); err != nil {
|
if token, err = store.NewSessionToken(); err != nil {
|
||||||
return "", "", err
|
return "", "", err
|
||||||
}
|
}
|
||||||
@@ -402,7 +434,10 @@ func (s *Service) RequestReset(ctx context.Context, rs ResetStore, email string)
|
|||||||
return "", store.User{}, ErrTooManyTries
|
return "", store.User{}, ErrTooManyTries
|
||||||
}
|
}
|
||||||
u, err := rs.UserByEmail(ctx, email)
|
u, err := rs.UserByEmail(ctx, email)
|
||||||
if err != nil || u.PasswordHash == "" || u.State != store.UserActive {
|
// An account whose password is kept elsewhere — "!oidc", "!scim",
|
||||||
|
// "!ldap" — gets no link: setting one here would give it a second way
|
||||||
|
// in that its directory knows nothing of, and outlives its leaving.
|
||||||
|
if err != nil || u.PasswordHash == "" || strings.HasPrefix(u.PasswordHash, "!") || u.State != store.UserActive {
|
||||||
return "", store.User{}, ErrNoReset
|
return "", store.User{}, ErrNoReset
|
||||||
}
|
}
|
||||||
token, err := rs.CreatePasswordReset(ctx, u.ID, ResetLinkTTL)
|
token, err := rs.CreatePasswordReset(ctx, u.ID, ResetLinkTTL)
|
||||||
|
|||||||
@@ -150,6 +150,7 @@ func newBenchCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
|
|||||||
f.DurationVar(&cfg.SLO.TPOTp99, "tpot-p99", cfg.SLO.TPOTp99, "per-output-token target")
|
f.DurationVar(&cfg.SLO.TPOTp99, "tpot-p99", cfg.SLO.TPOTp99, "per-output-token target")
|
||||||
f.BoolVar(&save, "save", false,
|
f.BoolVar(&save, "save", false,
|
||||||
"record the capacity against the model and profile, so the number outlives this terminal")
|
"record the capacity against the model and profile, so the number outlives this terminal")
|
||||||
|
cmd.AddCommand(newBenchOverheadCmd(asJSON))
|
||||||
return cmd
|
return cmd
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,421 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime"
|
||||||
|
"slices"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"syscall"
|
||||||
|
"text/tabwriter"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/spf13/cobra"
|
||||||
|
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/guard"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/router"
|
||||||
|
storepkg "gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// What llmgw adds to a request, measured: the same load straight at an
|
||||||
|
// engine that answers at once, and through the gateway in front of it —
|
||||||
|
// bare, with authentication and usage accounting, and with the guardrails'
|
||||||
|
// pattern checks besides. The engine's own time is the same on both paths,
|
||||||
|
// so what differs is the gateway's.
|
||||||
|
//
|
||||||
|
// Everything runs in this process, on loopback: no cluster, no GPU, and the
|
||||||
|
// numbers are this machine's. On a machine shared with other work (a VM on
|
||||||
|
// an oversubscribed host) they move with it; compare modes run together,
|
||||||
|
// not figures from different runs.
|
||||||
|
|
||||||
|
// overheadMode is one way a request reaches the engine.
|
||||||
|
type overheadMode struct {
|
||||||
|
name string
|
||||||
|
gateway bool
|
||||||
|
auth, guard bool
|
||||||
|
}
|
||||||
|
|
||||||
|
var overheadModes = []overheadMode{
|
||||||
|
{name: "direct"},
|
||||||
|
{name: "gateway", gateway: true},
|
||||||
|
{name: "gateway+auth", gateway: true, auth: true},
|
||||||
|
{name: "gateway+auth+guard", gateway: true, auth: true, guard: true},
|
||||||
|
}
|
||||||
|
|
||||||
|
// overheadStep is one mode at one request rate.
|
||||||
|
type overheadStep struct {
|
||||||
|
Mode string `json:"mode"`
|
||||||
|
Rate int `json:"rate"`
|
||||||
|
Sent int `json:"sent"`
|
||||||
|
Failed int `json:"failed"`
|
||||||
|
P50 time.Duration `json:"p50"`
|
||||||
|
P99 time.Duration `json:"p99"`
|
||||||
|
AddedP50 time.Duration `json:"added_p50"`
|
||||||
|
AddedP99 time.Duration `json:"added_p99"`
|
||||||
|
// CPUPer1k is the process's CPU time per thousand requests beyond what
|
||||||
|
// the direct mode spent at the same rate: the gateway's.
|
||||||
|
CPUPer1k time.Duration `json:"cpu_per_1k"`
|
||||||
|
cpu time.Duration
|
||||||
|
}
|
||||||
|
|
||||||
|
func newBenchOverheadCmd(asJSON *bool) *cobra.Command {
|
||||||
|
var (
|
||||||
|
rates []int
|
||||||
|
each time.Duration
|
||||||
|
)
|
||||||
|
cmd := &cobra.Command{
|
||||||
|
Use: "overhead",
|
||||||
|
Short: "Measure what the gateway adds to a request, against an engine that answers at once",
|
||||||
|
Long: "Runs an engine that answers at once and a gateway in front of it, in this\n" +
|
||||||
|
"process on loopback, and sends the same load at each rate straight to the\n" +
|
||||||
|
"engine and through the gateway: bare, with authentication and usage\n" +
|
||||||
|
"accounting, and with the guardrails' pattern checks too. Reports each\n" +
|
||||||
|
"request's latency at p50 and p99, what the gateway added to it, and the\n" +
|
||||||
|
"gateway's CPU time per thousand requests.\n\n" +
|
||||||
|
"Needs no cluster. The figures are this machine's: on one shared with other\n" +
|
||||||
|
"work they move with it, so compare the modes of one run, and note the\n" +
|
||||||
|
"machine's load beside any figure you keep.",
|
||||||
|
Args: cobra.NoArgs,
|
||||||
|
RunE: func(cmd *cobra.Command, _ []string) error {
|
||||||
|
steps, err := runOverhead(cmd.Context(), rates, each)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if *asJSON {
|
||||||
|
return json.NewEncoder(cmd.OutOrStdout()).Encode(steps)
|
||||||
|
}
|
||||||
|
printOverhead(cmd.OutOrStdout(), steps)
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
f := cmd.Flags()
|
||||||
|
f.IntSliceVar(&rates, "rates", []int{100, 500, 1000, 2000}, "requests a second to send, one step each")
|
||||||
|
f.DurationVar(&each, "each", 3*time.Second, "how long each step sends for")
|
||||||
|
return cmd
|
||||||
|
}
|
||||||
|
|
||||||
|
func printOverhead(w io.Writer, steps []overheadStep) {
|
||||||
|
tw := tabwriter.NewWriter(w, 0, 0, 2, ' ', 0)
|
||||||
|
fmt.Fprintln(tw, "MODE\tRATE/S\tSENT\tFAILED\tP50\tP99\t+P50\t+P99\tCPU/1K REQ")
|
||||||
|
for _, s := range steps {
|
||||||
|
added := func(d time.Duration) string {
|
||||||
|
if s.Mode == "direct" {
|
||||||
|
return "-"
|
||||||
|
}
|
||||||
|
return msOf(d)
|
||||||
|
}
|
||||||
|
cpu := "-"
|
||||||
|
if s.Mode != "direct" {
|
||||||
|
cpu = msOf(s.CPUPer1k)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(tw, "%s\t%d\t%d\t%d\t%s\t%s\t%s\t%s\t%s\n", s.Mode, s.Rate, s.Sent, s.Failed,
|
||||||
|
msOf(s.P50), msOf(s.P99), added(s.AddedP50), added(s.AddedP99), cpu)
|
||||||
|
}
|
||||||
|
_ = tw.Flush()
|
||||||
|
fmt.Fprintf(w, "\n%d CPUs, %s. Loopback, in one process: the figures are this machine's.\n",
|
||||||
|
runtime.NumCPU(), loadAverage())
|
||||||
|
}
|
||||||
|
|
||||||
|
func msOf(d time.Duration) string { return fmt.Sprintf("%.2fms", float64(d)/float64(time.Millisecond)) }
|
||||||
|
|
||||||
|
// loadAverage is the machine's, where it can be read, to keep beside the
|
||||||
|
// figures.
|
||||||
|
func loadAverage() string {
|
||||||
|
if b, err := os.ReadFile("/proc/loadavg"); err == nil {
|
||||||
|
if f := strings.Fields(string(b)); len(f) >= 3 {
|
||||||
|
return "load " + strings.Join(f[:3], " ")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// macOS: "{ 1.23 1.10 0.98 }".
|
||||||
|
if b, err := exec.Command("sysctl", "-n", "vm.loadavg").Output(); err == nil {
|
||||||
|
if f := strings.Fields(strings.Trim(strings.TrimSpace(string(b)), "{}")); len(f) >= 3 {
|
||||||
|
return "load " + strings.Join(f[:3], " ")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "load not read"
|
||||||
|
}
|
||||||
|
|
||||||
|
// runOverhead measures every mode at every rate.
|
||||||
|
func runOverhead(ctx context.Context, rates []int, each time.Duration) ([]overheadStep, error) {
|
||||||
|
engine, err := instantEngine()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer engine.Close()
|
||||||
|
dir, err := os.MkdirTemp("", "llmgw-overhead-")
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer os.RemoveAll(dir)
|
||||||
|
|
||||||
|
type target struct {
|
||||||
|
url, key string
|
||||||
|
stop func()
|
||||||
|
}
|
||||||
|
targets := map[string]target{}
|
||||||
|
defer func() {
|
||||||
|
for _, t := range targets {
|
||||||
|
if t.stop != nil {
|
||||||
|
t.stop()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
for _, m := range overheadModes {
|
||||||
|
if !m.gateway {
|
||||||
|
targets[m.name] = target{url: engine.URL}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
url, key, stop, err := overheadGateway(ctx, engine.URL, filepath.Join(dir, m.name+".db"), m)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", m.name, err)
|
||||||
|
}
|
||||||
|
targets[m.name] = target{url: url, key: key, stop: stop}
|
||||||
|
}
|
||||||
|
|
||||||
|
client := &http.Client{Transport: &http.Transport{MaxIdleConns: 1024, MaxIdleConnsPerHost: 1024}, Timeout: 30 * time.Second}
|
||||||
|
var out []overheadStep
|
||||||
|
for _, rate := range rates {
|
||||||
|
var direct overheadStep
|
||||||
|
for _, m := range overheadModes {
|
||||||
|
t := targets[m.name]
|
||||||
|
s := loadStep(ctx, client, t.url, t.key, rate, each)
|
||||||
|
s.Mode = m.name
|
||||||
|
if m.name == "direct" {
|
||||||
|
direct = s
|
||||||
|
} else {
|
||||||
|
s.AddedP50, s.AddedP99 = max(0, s.P50-direct.P50), max(0, s.P99-direct.P99)
|
||||||
|
if s.Sent > 0 {
|
||||||
|
s.CPUPer1k = max(0, s.cpu-direct.cpu) * 1000 / time.Duration(s.Sent)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out = append(out, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// loadStep sends rate requests a second for d, each in its own goroutine,
|
||||||
|
// and measures each one's latency and the process's CPU meanwhile.
|
||||||
|
func loadStep(ctx context.Context, client *http.Client, url, key string, rate int, d time.Duration) overheadStep {
|
||||||
|
s := overheadStep{Rate: rate}
|
||||||
|
body := []byte(`{"model":"bench","max_tokens":8,"messages":[{"role":"user","content":"What is the capital of France?"}]}`)
|
||||||
|
var (
|
||||||
|
mu sync.Mutex
|
||||||
|
lat []time.Duration
|
||||||
|
wg sync.WaitGroup
|
||||||
|
total = int(float64(rate) * d.Seconds())
|
||||||
|
)
|
||||||
|
interval := time.Second / time.Duration(max(rate, 1))
|
||||||
|
cpu0 := processCPU()
|
||||||
|
start := time.Now()
|
||||||
|
for i := range total {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if wait := time.Until(start.Add(time.Duration(i) * interval)); wait > 0 {
|
||||||
|
time.Sleep(wait)
|
||||||
|
}
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, url+"/v1/chat/completions", bytes.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
if key != "" {
|
||||||
|
req.Header.Set("Authorization", "Bearer "+key)
|
||||||
|
}
|
||||||
|
t := time.Now()
|
||||||
|
resp, err := client.Do(req)
|
||||||
|
ok := err == nil && resp.StatusCode == http.StatusOK
|
||||||
|
if resp != nil {
|
||||||
|
_, _ = io.Copy(io.Discard, resp.Body)
|
||||||
|
resp.Body.Close()
|
||||||
|
}
|
||||||
|
el := time.Since(t)
|
||||||
|
mu.Lock()
|
||||||
|
s.Sent++
|
||||||
|
if ok {
|
||||||
|
lat = append(lat, el)
|
||||||
|
} else {
|
||||||
|
s.Failed++
|
||||||
|
}
|
||||||
|
mu.Unlock()
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
s.cpu = processCPU() - cpu0
|
||||||
|
s.P50, s.P99 = percentile(lat, 0.50), percentile(lat, 0.99)
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
func percentile(ds []time.Duration, p float64) time.Duration {
|
||||||
|
if len(ds) == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
ds = slices.Clone(ds)
|
||||||
|
sort.Slice(ds, func(i, j int) bool { return ds[i] < ds[j] })
|
||||||
|
i := int(p * float64(len(ds)-1))
|
||||||
|
return ds[i]
|
||||||
|
}
|
||||||
|
|
||||||
|
// processCPU is the CPU time this process has used, user and system.
|
||||||
|
func processCPU() time.Duration {
|
||||||
|
var ru syscall.Rusage
|
||||||
|
if err := syscall.Getrusage(syscall.RUSAGE_SELF, &ru); err != nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return time.Duration(ru.Utime.Nano() + ru.Stime.Nano())
|
||||||
|
}
|
||||||
|
|
||||||
|
// instantEngine is an OpenAI-compatible engine that answers at once, with
|
||||||
|
// the metrics the router scrapes.
|
||||||
|
func instantEngine() (*httpServer, error) {
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.HandleFunc("GET /metrics", func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
fmt.Fprint(w, "# TYPE vllm:num_requests_running gauge\nvllm:num_requests_running 0\n"+
|
||||||
|
"# TYPE vllm:num_requests_waiting gauge\nvllm:num_requests_waiting 0\n"+
|
||||||
|
"# TYPE vllm:kv_cache_usage_perc gauge\nvllm:kv_cache_usage_perc 0.1\n")
|
||||||
|
})
|
||||||
|
mux.HandleFunc("GET /health", func(w http.ResponseWriter, _ *http.Request) { w.WriteHeader(http.StatusOK) })
|
||||||
|
mux.HandleFunc("POST /v1/chat/completions", func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_, _ = io.Copy(io.Discard, r.Body)
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
fmt.Fprint(w, `{"id":"b","object":"chat.completion","model":"bench","choices":[{"index":0,"message":{"role":"assistant","content":"Paris."},"finish_reason":"stop"}],"usage":{"prompt_tokens":14,"completion_tokens":2,"total_tokens":16}}`)
|
||||||
|
})
|
||||||
|
return serveLoopback(mux)
|
||||||
|
}
|
||||||
|
|
||||||
|
// httpServer is a handler served on a loopback port.
|
||||||
|
type httpServer struct {
|
||||||
|
URL string
|
||||||
|
srv *http.Server
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *httpServer) Close() { _ = s.srv.Close() }
|
||||||
|
|
||||||
|
func serveLoopback(h http.Handler) (*httpServer, error) {
|
||||||
|
ln, err := net.Listen("tcp", "127.0.0.1:0")
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
srv := &http.Server{Handler: h, ReadHeaderTimeout: 10 * time.Second}
|
||||||
|
go func() { _ = srv.Serve(ln) }()
|
||||||
|
return &httpServer{URL: "http://" + ln.Addr().String(), srv: srv}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// overheadGateway starts a gateway in front of the engine as the mode asks,
|
||||||
|
// and returns its address and, with auth, a key.
|
||||||
|
func overheadGateway(ctx context.Context, engineURL, storePath string, m overheadMode) (string, string, func(), error) {
|
||||||
|
cfg := config.Default()
|
||||||
|
cfg.Listen = "127.0.0.1:0"
|
||||||
|
cfg.Router.ScrapeInterval = 200 * time.Millisecond
|
||||||
|
cfg.Endpoints = []config.Endpoint{{ID: "bench", URL: engineURL, Models: []string{"bench"}}}
|
||||||
|
key := ""
|
||||||
|
if m.auth {
|
||||||
|
var err error
|
||||||
|
if key, err = overheadKey(ctx, storePath); err != nil {
|
||||||
|
return "", "", nil, err
|
||||||
|
}
|
||||||
|
cfg.Auth.Enabled, cfg.Auth.Store = true, storePath
|
||||||
|
// Limits would measure themselves refusing: off.
|
||||||
|
cfg.Auth.DefaultRPM, cfg.Auth.DefaultTPM = 0, 0
|
||||||
|
}
|
||||||
|
rt, err := router.New(cfg, slog.New(slog.DiscardHandler))
|
||||||
|
if err != nil {
|
||||||
|
return "", "", nil, err
|
||||||
|
}
|
||||||
|
if m.guard {
|
||||||
|
g, err := baselineGuard()
|
||||||
|
if err != nil {
|
||||||
|
return "", "", nil, err
|
||||||
|
}
|
||||||
|
rt.SetGuard(g)
|
||||||
|
}
|
||||||
|
rctx, cancel := context.WithCancel(ctx)
|
||||||
|
rt.Start(rctx)
|
||||||
|
srv, err := serveLoopback(rt.Handler())
|
||||||
|
if err != nil {
|
||||||
|
cancel()
|
||||||
|
return "", "", nil, err
|
||||||
|
}
|
||||||
|
stop := func() { srv.Close(); cancel(); rt.Stop() }
|
||||||
|
// Ready once a request goes through: the endpoint is scraped first.
|
||||||
|
deadline := time.Now().Add(10 * time.Second)
|
||||||
|
for {
|
||||||
|
body := `{"model":"bench","messages":[{"role":"user","content":"hi"}]}`
|
||||||
|
req, _ := http.NewRequestWithContext(ctx, http.MethodPost, srv.URL+"/v1/chat/completions", strings.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
if key != "" {
|
||||||
|
req.Header.Set("Authorization", "Bearer "+key)
|
||||||
|
}
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err == nil {
|
||||||
|
_, _ = io.Copy(io.Discard, resp.Body)
|
||||||
|
resp.Body.Close()
|
||||||
|
if resp.StatusCode == http.StatusOK {
|
||||||
|
return srv.URL, key, stop, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
stop()
|
||||||
|
status := "no answer"
|
||||||
|
if err == nil {
|
||||||
|
status = resp.Status
|
||||||
|
}
|
||||||
|
return "", "", nil, fmt.Errorf("the gateway did not serve within 10s (%s)", status)
|
||||||
|
}
|
||||||
|
time.Sleep(100 * time.Millisecond)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// overheadKey makes a store with one project and a key in it.
|
||||||
|
func overheadKey(ctx context.Context, path string) (string, error) {
|
||||||
|
st, err := storepkg.Open(ctx, path)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
defer st.Close()
|
||||||
|
org, err := st.CreateOrg(ctx, "bench")
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
team, err := st.CreateTeam(ctx, org, "bench")
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
project, err := st.CreateProject(ctx, team, "bench")
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
secret, k, err := storepkg.NewKey("key-bench", "bench", project)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
k.CreatedAt = time.Now()
|
||||||
|
return secret, st.CreateKey(ctx, k)
|
||||||
|
}
|
||||||
|
|
||||||
|
// baselineGuard is the policy init installs, without its judge: the pattern
|
||||||
|
// checks every request passes.
|
||||||
|
func baselineGuard() (guard.Guard, error) {
|
||||||
|
m, err := config.Parse([]byte(guardSection(false)))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
cfg, err := config.FromMap(m)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return guard.NewGate(cfg.Guard, nil, slog.New(slog.DiscardHandler))
|
||||||
|
}
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
package cli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The short version, so CI sees the gateway's overhead on every change: every
|
||||||
|
// mode answers every request, and what the gateway adds is measured, not
|
||||||
|
// guessed. No bound is asserted on it, as CI's machines vary; the table is
|
||||||
|
// in the test's log.
|
||||||
|
func TestTheGatewaysOverheadIsMeasured(t *testing.T) {
|
||||||
|
steps, err := runOverhead(t.Context(), []int{50}, time.Second)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(steps) != len(overheadModes) {
|
||||||
|
t.Fatalf("%d steps for %d modes", len(steps), len(overheadModes))
|
||||||
|
}
|
||||||
|
for _, s := range steps {
|
||||||
|
if s.Sent == 0 || s.Failed > 0 || s.P50 <= 0 || s.P99 < s.P50 {
|
||||||
|
t.Errorf("%s: sent %d failed %d p50 %s p99 %s", s.Mode, s.Sent, s.Failed, s.P50, s.P99)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var b bytes.Buffer
|
||||||
|
printOverhead(&b, steps)
|
||||||
|
if !strings.Contains(b.String(), "gateway+auth+guard") {
|
||||||
|
t.Errorf("table:\n%s", b.String())
|
||||||
|
}
|
||||||
|
t.Logf("\n%s", b.String())
|
||||||
|
}
|
||||||
@@ -32,6 +32,19 @@ type judgeKeeper struct {
|
|||||||
// tried is when each judge was last asked for, so one that cannot be
|
// tried is when each judge was last asked for, so one that cannot be
|
||||||
// fetched is not asked for again on every look.
|
// fetched is not asked for again on every look.
|
||||||
tried map[string]time.Time
|
tried map[string]time.Time
|
||||||
|
|
||||||
|
// What refitting an earlier install's judge needs: the catalog to change
|
||||||
|
// its profile in, the replicas to restart, and the nodes' CPUs. Any nil
|
||||||
|
// leaves judges as they are.
|
||||||
|
profiles interface {
|
||||||
|
SetProfiles(ctx context.Context, name string, profiles []catalog.Profile, by string) (catalog.Model, error)
|
||||||
|
}
|
||||||
|
restart func(ids ...string) error
|
||||||
|
replicas func(model string) []string
|
||||||
|
nodeCPUs func() []int
|
||||||
|
// looked is each judge already refitted or found changed by hand, so
|
||||||
|
// each is said once.
|
||||||
|
looked map[string]bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// judgeRetry is how long a judge whose serving failed waits before it is
|
// judgeRetry is how long a judge whose serving failed waits before it is
|
||||||
@@ -69,6 +82,7 @@ func (k *judgeKeeper) check() {
|
|||||||
have := map[string]bool{}
|
have := map[string]bool{}
|
||||||
for _, m := range k.models() {
|
for _, m := range k.models() {
|
||||||
have[m.Name] = true
|
have[m.Name] = true
|
||||||
|
k.refit(m)
|
||||||
}
|
}
|
||||||
busy := map[string]bool{}
|
busy := map[string]bool{}
|
||||||
for _, j := range k.serve.Jobs() {
|
for _, j := range k.serve.Jobs() {
|
||||||
@@ -114,3 +128,88 @@ func (k *judgeKeeper) check() {
|
|||||||
"judge", name, "repo", kj.Repo, "file", kj.File, "job", job.ID)
|
"judge", name, "repo", kj.Repo, "file", kj.File, "job", job.ID)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// refit moves a judge an earlier `server init` installed to what init
|
||||||
|
// installs now, and restarts its replicas onto it. Those judges reserved no
|
||||||
|
// cores or one, 2 or 4 GB, and ran llama.cpp's prompt cache: on fewer than
|
||||||
|
// eight cores every judgment missed its 5 s budget and fail: closed refused
|
||||||
|
// every request, until fixed by hand. A judge whose profile differs in any
|
||||||
|
// other way was set by somebody, and is left as it is, said once.
|
||||||
|
func (k *judgeKeeper) refit(m catalog.Model) {
|
||||||
|
if k.profiles == nil || k.restart == nil || k.replicas == nil || m.Name != defaultJudge || !m.IsGuard() {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if k.looked == nil {
|
||||||
|
k.looked = map[string]bool{}
|
||||||
|
}
|
||||||
|
if k.looked[m.Name] {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
k.looked[m.Name] = true
|
||||||
|
if !earlierInitJudge(m) {
|
||||||
|
if len(m.Profiles) != 1 || !slices.Contains(m.Profiles[0].Args, "--cache-ram") {
|
||||||
|
k.log.Info("the judge's profile is not one init wrote; left as it is",
|
||||||
|
"judge", m.Name, "see", "llmgw ctl model resources "+m.Name)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var cpus []int
|
||||||
|
if k.nodeCPUs != nil {
|
||||||
|
cpus = k.nodeCPUs()
|
||||||
|
}
|
||||||
|
p := m.Profiles[0]
|
||||||
|
p.MinMemoryGB = judgeMemoryGB
|
||||||
|
p.CPUCores = refitCores(m.Scaling.Min, cpus)
|
||||||
|
p.Args = []string{"--ctx-size", "8192", "--cache-ram", "0"}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if _, err := k.profiles.SetProfiles(ctx, m.Name, []catalog.Profile{p}, "llmgw (the judge an earlier init installed)"); err != nil {
|
||||||
|
k.log.Warn("could not refit the judge an earlier init installed", "judge", m.Name, "err", err)
|
||||||
|
k.looked[m.Name] = false
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ids := k.replicas(m.Name)
|
||||||
|
if len(ids) > 0 {
|
||||||
|
if err := k.restart(ids...); err != nil {
|
||||||
|
k.log.Warn("refitted the judge, but could not restart it", "judge", m.Name, "err", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
k.log.Info("refitted the judge an earlier init installed", "judge", m.Name,
|
||||||
|
"memory_gb", p.MinMemoryGB, "cpu_cores", p.CPUCores, "replicas_restarted", len(ids))
|
||||||
|
}
|
||||||
|
|
||||||
|
// earlierInitJudge is whether a judge's entry is exactly what `server init`
|
||||||
|
// before v0.14.7 wrote: one llama.cpp profile on CPUs, an 8192 context and
|
||||||
|
// no other args, 2 or 4 GB, none to two cores.
|
||||||
|
func earlierInitJudge(m catalog.Model) bool {
|
||||||
|
if len(m.Profiles) != 1 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
p := m.Profiles[0]
|
||||||
|
return p.Name == "cpu" && p.Engine == "llamacpp" && p.GPUClass == "" && p.GPU == nil &&
|
||||||
|
len(p.NodeSelector) == 0 && slices.Equal(p.Args, []string{"--ctx-size", "8192"}) &&
|
||||||
|
(p.MinMemoryGB == 2 || p.MinMemoryGB == 4) && p.CPUCores >= 0 && p.CPUCores <= 2
|
||||||
|
}
|
||||||
|
|
||||||
|
// refitCores is a judge replica's cores by init's rule: up to four for the
|
||||||
|
// one judge of a cluster, two each where there are more, fewer where the
|
||||||
|
// smallest node cannot hold them all.
|
||||||
|
func refitCores(replicas int, cpus []int) int {
|
||||||
|
smallest := 0
|
||||||
|
for _, c := range cpus {
|
||||||
|
if c > 0 && (smallest == 0 || c < smallest) {
|
||||||
|
smallest = c
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if replicas <= 1 {
|
||||||
|
if smallest == 0 {
|
||||||
|
return judgeCores
|
||||||
|
}
|
||||||
|
return min(smallest, judgeCoresAlone)
|
||||||
|
}
|
||||||
|
cores := judgeCores
|
||||||
|
if len(cpus) == 1 && smallest > 0 && replicas*cores > smallest {
|
||||||
|
cores = max(1, smallest/replicas)
|
||||||
|
}
|
||||||
|
return cores
|
||||||
|
}
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
package cli
|
package cli
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"slices"
|
"slices"
|
||||||
@@ -124,3 +125,81 @@ func TestCategoriesFollowTheJudges(t *testing.T) {
|
|||||||
t.Errorf("no categories (every one counts) became %v", got)
|
t.Errorf("no categories (every one counts) became %v", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
type fakeProfiles struct {
|
||||||
|
set map[string][]catalog.Profile
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeProfiles) SetProfiles(_ context.Context, name string, ps []catalog.Profile, _ string) (catalog.Model, error) {
|
||||||
|
if f.set == nil {
|
||||||
|
f.set = map[string][]catalog.Profile{}
|
||||||
|
}
|
||||||
|
f.set[name] = ps
|
||||||
|
return catalog.Model{Name: name, Profiles: ps}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// A judge that v0.14.6 or earlier's init installed is moved to today's
|
||||||
|
// figures, once, and its replicas restarted onto them; one somebody changed
|
||||||
|
// is left alone.
|
||||||
|
func TestAnEarlierInstallsJudgeIsRefittedOnce(t *testing.T) {
|
||||||
|
lib := guard.Config{Policies: map[string]guard.Policy{
|
||||||
|
"baseline": {Judge: &guard.JudgePolicy{Model: "qwen3guard-0.6b", Input: "block"}},
|
||||||
|
}}
|
||||||
|
earlier := guardModel("http://10.0.0.5:8081/qwen3guard-0.6b/guard/g.gguf", 2, 1)
|
||||||
|
earlier.Profiles[0].MinMemoryGB, earlier.Profiles[0].Args = 4, []string{"--ctx-size", "8192"}
|
||||||
|
inCatalog := []catalog.Model{earlier}
|
||||||
|
fp := &fakeProfiles{}
|
||||||
|
var restarted []string
|
||||||
|
k := &judgeKeeper{
|
||||||
|
library: func() guard.Config { return lib },
|
||||||
|
models: func() []catalog.Model { return inCatalog },
|
||||||
|
serve: &fakeServe{},
|
||||||
|
log: slog.New(slog.DiscardHandler),
|
||||||
|
profiles: fp,
|
||||||
|
restart: func(ids ...string) error { restarted = append(restarted, ids...); return nil },
|
||||||
|
replicas: func(string) []string { return []string{"j-1", "j-2"} },
|
||||||
|
nodeCPUs: func() []int { return []int{4} },
|
||||||
|
}
|
||||||
|
k.check()
|
||||||
|
got := fp.set["qwen3guard-0.6b"]
|
||||||
|
if len(got) != 1 || got[0].MinMemoryGB != 2 || got[0].CPUCores != 2 || !slices.Contains(got[0].Args, "--cache-ram") {
|
||||||
|
t.Fatalf("refitted to %+v", got)
|
||||||
|
}
|
||||||
|
if !slices.Equal(restarted, []string{"j-1", "j-2"}) {
|
||||||
|
t.Errorf("restarted %v", restarted)
|
||||||
|
}
|
||||||
|
k.check()
|
||||||
|
if len(restarted) != 2 {
|
||||||
|
t.Error("refitted twice")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Changed by somebody: an extra arg. Left as it is.
|
||||||
|
mine := guardModel("http://x/g.gguf", 2, 3)
|
||||||
|
mine.Profiles[0].Args = []string{"--ctx-size", "16384"}
|
||||||
|
inCatalog = []catalog.Model{mine}
|
||||||
|
fp.set, k.looked = nil, nil
|
||||||
|
k.check()
|
||||||
|
if fp.set != nil {
|
||||||
|
t.Errorf("a judge set by hand was changed: %+v", fp.set)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAJudgeIsRefittedToTheCoresInitWouldGive(t *testing.T) {
|
||||||
|
for _, c := range []struct {
|
||||||
|
replicas int
|
||||||
|
cpus []int
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{1, []int{4}, 4},
|
||||||
|
{1, []int{2}, 2},
|
||||||
|
{1, []int{16}, 4},
|
||||||
|
{2, []int{4}, 2},
|
||||||
|
{2, []int{2}, 1},
|
||||||
|
{2, []int{4, 32}, 2},
|
||||||
|
{2, nil, 2},
|
||||||
|
} {
|
||||||
|
if got := refitCores(c.replicas, c.cpus); got != c.want {
|
||||||
|
t.Errorf("%d judges on %v: %d cores, want %d", c.replicas, c.cpus, got, c.want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -593,6 +593,25 @@ func newServerCmd() *cobra.Command {
|
|||||||
},
|
},
|
||||||
gpus: func() []adminapi.ClassUse { return adminAPI.Summary().GPUs },
|
gpus: func() []adminapi.ClassUse { return adminAPI.Summary().GPUs },
|
||||||
log: log,
|
log: log,
|
||||||
|
// An earlier install's judge, moved to today's figures.
|
||||||
|
profiles: catalogSvc,
|
||||||
|
restart: admin.Reconciler().Restart,
|
||||||
|
replicas: func(model string) []string {
|
||||||
|
var ids []string
|
||||||
|
for _, r := range admin.Reconciler().ReadyReplicas() {
|
||||||
|
if r.Model == model {
|
||||||
|
ids = append(ids, r.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ids
|
||||||
|
},
|
||||||
|
nodeCPUs: func() []int {
|
||||||
|
var out []int
|
||||||
|
for _, n := range admin.Nodes() {
|
||||||
|
out = append(out, n.CPUCount)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
},
|
||||||
}
|
}
|
||||||
go keeper.run(ctx, 30*time.Second)
|
go keeper.run(ctx, 30*time.Second)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -442,6 +442,9 @@ type Auth struct {
|
|||||||
|
|
||||||
// OIDC is sign-in through an identity provider.
|
// OIDC is sign-in through an identity provider.
|
||||||
OIDC OIDC `koanf:"oidc"`
|
OIDC OIDC `koanf:"oidc"`
|
||||||
|
// LDAP is sign-in against a directory, Active Directory above all, and
|
||||||
|
// the accounts kept in step with it.
|
||||||
|
LDAP LDAP `koanf:"ldap"`
|
||||||
// SCIM is account provisioning by the identity provider.
|
// SCIM is account provisioning by the identity provider.
|
||||||
SCIM SCIM `koanf:"scim"`
|
SCIM SCIM `koanf:"scim"`
|
||||||
|
|
||||||
@@ -968,6 +971,12 @@ func (s Server) Validate() error {
|
|||||||
if err := s.Auth.OIDC.Validate(); err != nil {
|
if err := s.Auth.OIDC.Validate(); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
if err := s.Auth.LDAP.Validate(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if s.Auth.LDAP.Enabled() && !s.Auth.Enabled {
|
||||||
|
return fmt.Errorf("auth.ldap needs auth.enabled: it signs people in to the accounts the store keeps")
|
||||||
|
}
|
||||||
if err := s.AgentTLS.Validate(); err != nil {
|
if err := s.AgentTLS.Validate(); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,137 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// LDAP is sign-in against a directory — Active Directory above all — with
|
||||||
|
// the directory's own passwords, and the accounts kept in step with it.
|
||||||
|
//
|
||||||
|
// auth:
|
||||||
|
// ldap:
|
||||||
|
// urls: [ldaps://dc1.corp.example.com, ldaps://dc2.corp.example.com]
|
||||||
|
// bind_dn: CN=llmgw,OU=Service Accounts,DC=corp,DC=example,DC=com
|
||||||
|
// bind_password_file: /etc/llmgw/secrets/ldap-bind
|
||||||
|
// base_dn: OU=Staff,DC=corp,DC=example,DC=com
|
||||||
|
// create_accounts: true
|
||||||
|
// roles: {admin: [llmgw-admins]}
|
||||||
|
// memberships: {ml-team: "team:Acme/ml"}
|
||||||
|
//
|
||||||
|
// A person signs in on the portal's form with their sAMAccountName, their
|
||||||
|
// userPrincipalName or their mail, and their directory password. The
|
||||||
|
// gateway finds them with the bind account, checks the password by binding
|
||||||
|
// as them, and reads their groups, nested ones included. Every sync_interval
|
||||||
|
// the leader reads the directory again: an account disabled or gone there
|
||||||
|
// is suspended here, its sessions ended and its API keys refused.
|
||||||
|
type LDAP struct {
|
||||||
|
// URLs are the directory's servers, tried in turn: ldaps:// (636), or
|
||||||
|
// ldap:// (389) with StartTLS.
|
||||||
|
URLs []string `koanf:"urls"`
|
||||||
|
// StartTLS upgrades an ldap:// connection before anything is sent. A
|
||||||
|
// password never crosses a connection without TLS: ldap:// without it
|
||||||
|
// is refused.
|
||||||
|
StartTLS bool `koanf:"start_tls"`
|
||||||
|
// CAFile is the authority the servers' certificates are checked against,
|
||||||
|
// when it is not the system's: an AD Certificate Services root.
|
||||||
|
CAFile string `koanf:"ca_file"`
|
||||||
|
// BindDN and the file holding its password: a service account that may
|
||||||
|
// read users and groups. Nothing is written to the directory.
|
||||||
|
BindDN string `koanf:"bind_dn"`
|
||||||
|
BindPasswordFile string `koanf:"bind_password_file"`
|
||||||
|
// BaseDN is where people are searched for, subtree included.
|
||||||
|
BaseDN string `koanf:"base_dn"`
|
||||||
|
// GroupBaseDN is where their groups are; empty is BaseDN's domain root.
|
||||||
|
GroupBaseDN string `koanf:"group_base_dn"`
|
||||||
|
// UserFilter narrows who may sign in, ANDed with the login match: a
|
||||||
|
// group (memberOf=...) or an OU's people only. Empty is every enabled
|
||||||
|
// user object.
|
||||||
|
UserFilter string `koanf:"user_filter"`
|
||||||
|
// Label is what the form says to sign in with: "Sign in with your
|
||||||
|
// <label> account". Empty is "Active Directory".
|
||||||
|
Label string `koanf:"label"`
|
||||||
|
// CreateAccounts makes an account, active, for a person the directory
|
||||||
|
// vouches for and the gateway does not know. Off, only existing accounts
|
||||||
|
// sign in this way.
|
||||||
|
CreateAccounts bool `koanf:"create_accounts"`
|
||||||
|
// Roles maps a staff role (viewer, operator, billing, admin) to the
|
||||||
|
// groups that give it, by name (CN) or whole DN. Set at every sign-in
|
||||||
|
// and sync: someone taken out of the group loses the role.
|
||||||
|
Roles map[string][]string `koanf:"roles"`
|
||||||
|
// Memberships maps a group, by CN or DN, to a place in the tree —
|
||||||
|
// org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT — joined at sign-in
|
||||||
|
// and kept by the sync, which also takes away a place whose group the
|
||||||
|
// person has left.
|
||||||
|
Memberships map[string]string `koanf:"memberships"`
|
||||||
|
// SyncInterval is how often the leader reads the directory. Zero is
|
||||||
|
// 15m; negative turns the sync off, leaving only sign-in.
|
||||||
|
SyncInterval time.Duration `koanf:"sync_interval"`
|
||||||
|
// Timeout bounds each round trip to a server. Zero is 10s.
|
||||||
|
Timeout time.Duration `koanf:"timeout"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Enabled reports whether directory sign-in is configured.
|
||||||
|
func (l LDAP) Enabled() bool { return len(l.URLs) > 0 }
|
||||||
|
|
||||||
|
// Validate says what in the configuration cannot work.
|
||||||
|
func (l LDAP) Validate() error {
|
||||||
|
if !l.Enabled() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for _, raw := range l.URLs {
|
||||||
|
u, err := url.Parse(raw)
|
||||||
|
if err != nil || u.Host == "" {
|
||||||
|
return fmt.Errorf("auth.ldap.urls: %q is not an ldap:// or ldaps:// address", raw)
|
||||||
|
}
|
||||||
|
switch u.Scheme {
|
||||||
|
case "ldaps":
|
||||||
|
case "ldap":
|
||||||
|
if !l.StartTLS {
|
||||||
|
return fmt.Errorf("auth.ldap.urls: %q is plaintext: use ldaps://, or set start_tls; passwords would cross it", raw)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("auth.ldap.urls: %q is not an ldap:// or ldaps:// address", raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case strings.TrimSpace(l.BindDN) == "" || l.BindPasswordFile == "":
|
||||||
|
return errors.New("auth.ldap: bind_dn and bind_password_file are needed, for the account that finds people")
|
||||||
|
case strings.TrimSpace(l.BaseDN) == "":
|
||||||
|
return errors.New("auth.ldap: base_dn is needed, where people are searched for")
|
||||||
|
case l.UserFilter != "" && !(strings.HasPrefix(l.UserFilter, "(") && strings.HasSuffix(l.UserFilter, ")")):
|
||||||
|
return fmt.Errorf("auth.ldap.user_filter: %q is not an LDAP filter in parentheses", l.UserFilter)
|
||||||
|
}
|
||||||
|
for role := range l.Roles {
|
||||||
|
if !validStaffRole(role) {
|
||||||
|
return fmt.Errorf("auth.ldap.roles: %q is not viewer, operator, billing or admin", role)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for g, place := range l.Memberships {
|
||||||
|
if !validPlace(place) {
|
||||||
|
return fmt.Errorf("auth.ldap.memberships[%s]: %q is not org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT", g, place)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Interval is SyncInterval with its default; zero means off.
|
||||||
|
func (l LDAP) Interval() time.Duration {
|
||||||
|
switch {
|
||||||
|
case l.SyncInterval < 0:
|
||||||
|
return 0
|
||||||
|
case l.SyncInterval == 0:
|
||||||
|
return 15 * time.Minute
|
||||||
|
}
|
||||||
|
return l.SyncInterval
|
||||||
|
}
|
||||||
|
|
||||||
|
// RoundTrip is Timeout with its default.
|
||||||
|
func (l LDAP) RoundTrip() time.Duration {
|
||||||
|
if l.Timeout <= 0 {
|
||||||
|
return 10 * time.Second
|
||||||
|
}
|
||||||
|
return l.Timeout
|
||||||
|
}
|
||||||
+18
-5
@@ -72,21 +72,34 @@ func (o OIDC) Validate() error {
|
|||||||
return errors.New("auth.oidc.client_id is required")
|
return errors.New("auth.oidc.client_id is required")
|
||||||
}
|
}
|
||||||
for role := range o.Roles {
|
for role := range o.Roles {
|
||||||
switch role {
|
if !validStaffRole(role) {
|
||||||
case "viewer", "operator", "billing", "admin":
|
|
||||||
default:
|
|
||||||
return fmt.Errorf("auth.oidc.roles: %q is not viewer, operator, billing or admin", role)
|
return fmt.Errorf("auth.oidc.roles: %q is not viewer, operator, billing or admin", role)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
for g, where := range o.Memberships {
|
for g, where := range o.Memberships {
|
||||||
kind, path, ok := strings.Cut(where, ":")
|
if !validPlace(where) {
|
||||||
if !ok || path == "" || (kind != "org" && kind != "team" && kind != "project") {
|
|
||||||
return fmt.Errorf("auth.oidc.memberships.%s is %q: org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT", g, where)
|
return fmt.Errorf("auth.oidc.memberships.%s is %q: org:NAME, team:ORG/TEAM or project:ORG/TEAM/PROJECT", g, where)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// validStaffRole is a role groups may give: viewer, operator, billing, admin.
|
||||||
|
func validStaffRole(role string) bool {
|
||||||
|
switch role {
|
||||||
|
case "viewer", "operator", "billing", "admin":
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// validPlace is a place in the tree a group may give: org:NAME,
|
||||||
|
// team:ORG/TEAM or project:ORG/TEAM/PROJECT.
|
||||||
|
func validPlace(where string) bool {
|
||||||
|
kind, path, ok := strings.Cut(where, ":")
|
||||||
|
return ok && path != "" && (kind == "org" || kind == "team" || kind == "project")
|
||||||
|
}
|
||||||
|
|
||||||
// SCIM lets an identity provider provision accounts (/scim/v2): made,
|
// SCIM lets an identity provider provision accounts (/scim/v2): made,
|
||||||
// suspended and reinstated as they are there, with groups mapped by
|
// suspended and reinstated as they are there, with groups mapped by
|
||||||
// auth.oidc's roles and memberships.
|
// auth.oidc's roles and memberships.
|
||||||
|
|||||||
@@ -1533,6 +1533,11 @@ type External struct {
|
|||||||
// rather than the key itself, so the key is in neither the catalog, the
|
// rather than the key itself, so the key is in neither the catalog, the
|
||||||
// store nor the audit log; empty for a provider that asks for none.
|
// store nor the audit log; empty for a provider that asks for none.
|
||||||
APIKeyFile string `json:"api_key_file,omitempty" koanf:"api_key_file"`
|
APIKeyFile string `json:"api_key_file,omitempty" koanf:"api_key_file"`
|
||||||
|
// APIKeyFiles are several keys for the provider, in place of
|
||||||
|
// APIKeyFile: one endpoint each (external:MODEL#1, #2…), so a key that
|
||||||
|
// answers 429 or 401 is passed over while its breaker is open and the
|
||||||
|
// request goes to another. A provider's limits are per key.
|
||||||
|
APIKeyFiles []ExternalKey `json:"api_key_files,omitempty" koanf:"api_key_files"`
|
||||||
// Peer names the federated llmgw cluster this model is offered by, for
|
// Peer names the federated llmgw cluster this model is offered by, for
|
||||||
// a model that federation added (`llmgw ctl federation add`) rather
|
// a model that federation added (`llmgw ctl federation add`) rather
|
||||||
// than one written by hand. Such a model is named PEER/MODEL, comes and
|
// than one written by hand. Such a model is named PEER/MODEL, comes and
|
||||||
@@ -1551,6 +1556,14 @@ type External struct {
|
|||||||
Output string `json:"output,omitempty" koanf:"output"`
|
Output string `json:"output,omitempty" koanf:"output"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ExternalKey is one of a provider's keys.
|
||||||
|
type ExternalKey struct {
|
||||||
|
// File holds the key, on the control plane, as APIKeyFile does.
|
||||||
|
File string `json:"file" koanf:"file"`
|
||||||
|
// Weight is its share of the requests against the others'; zero is 1.
|
||||||
|
Weight float64 `json:"weight,omitempty" koanf:"weight"`
|
||||||
|
}
|
||||||
|
|
||||||
// APIComfyUI is External.API for a ComfyUI server.
|
// APIComfyUI is External.API for a ComfyUI server.
|
||||||
const APIComfyUI = "comfyui"
|
const APIComfyUI = "comfyui"
|
||||||
|
|
||||||
@@ -1568,6 +1581,20 @@ func (e *External) validate(model string, profiles int) error {
|
|||||||
if strings.TrimSpace(e.Model) == "" {
|
if strings.TrimSpace(e.Model) == "" {
|
||||||
return fmt.Errorf("model %q: external model needs the name the provider knows it by", model)
|
return fmt.Errorf("model %q: external model needs the name the provider knows it by", model)
|
||||||
}
|
}
|
||||||
|
if e.APIKeyFile != "" && len(e.APIKeyFiles) > 0 {
|
||||||
|
return fmt.Errorf("model %q: api_key_file or api_key_files, not both", model)
|
||||||
|
}
|
||||||
|
for i, k := range e.APIKeyFiles {
|
||||||
|
if strings.TrimSpace(k.File) == "" {
|
||||||
|
return fmt.Errorf("model %q: api_key_files[%d] names no file", model, i)
|
||||||
|
}
|
||||||
|
if k.Weight < 0 {
|
||||||
|
return fmt.Errorf("model %q: api_key_files[%d] has a negative weight", model, i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(e.APIKeyFiles) > 0 && e.Peer != "" {
|
||||||
|
return fmt.Errorf("model %q: a federated model is reached with its peer's key, not api_key_files", model)
|
||||||
|
}
|
||||||
switch e.API {
|
switch e.API {
|
||||||
case "", "openai":
|
case "", "openai":
|
||||||
if e.Workflow != "" || e.Output != "" {
|
if e.Workflow != "" || e.Output != "" {
|
||||||
|
|||||||
@@ -429,3 +429,23 @@ func TestAGuardModelSaysHowItIsReadAndAlwaysRuns(t *testing.T) {
|
|||||||
t.Error("a judge is not critical by default")
|
t.Error("a judge is not critical by default")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestAProvidersKeysAreOneListOrOneFile(t *testing.T) {
|
||||||
|
ext := func(e External) Model {
|
||||||
|
e.URL, e.Model = "https://api.example.com", "m"
|
||||||
|
return Model{Name: "cloud", Source: "external", External: &e}
|
||||||
|
}
|
||||||
|
for _, bad := range []External{
|
||||||
|
{APIKeyFile: "/k", APIKeyFiles: []ExternalKey{{File: "/k2"}}},
|
||||||
|
{APIKeyFiles: []ExternalKey{{File: " "}}},
|
||||||
|
{APIKeyFiles: []ExternalKey{{File: "/k", Weight: -1}}},
|
||||||
|
{Peer: "p", APIKeyFiles: []ExternalKey{{File: "/k"}}},
|
||||||
|
} {
|
||||||
|
if _, err := New([]Model{ext(bad)}); err == nil {
|
||||||
|
t.Errorf("taken: %+v", bad)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := New([]Model{ext(External{APIKeyFiles: []ExternalKey{{File: "/a", Weight: 2}, {File: "/b"}}})}); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
+28
-12
@@ -438,15 +438,6 @@ func (c *Controller) externalEndpoints() []endpoint.Info {
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
key := ""
|
|
||||||
if f := m.External.APIKeyFile; f != "" {
|
|
||||||
b, err := secrets.Read(context.Background(), f)
|
|
||||||
if err != nil {
|
|
||||||
c.log.Warn("an external model's key could not be read; not offered", "model", m.Name, "err", err)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
key = strings.TrimSpace(string(b))
|
|
||||||
}
|
|
||||||
// A peer's endpoint says so in its id, which is what usage records
|
// A peer's endpoint says so in its id, which is what usage records
|
||||||
// as the replica that answered.
|
// as the replica that answered.
|
||||||
id := "external:" + m.Name
|
id := "external:" + m.Name
|
||||||
@@ -462,14 +453,39 @@ func (c *Controller) externalEndpoints() []endpoint.Info {
|
|||||||
}
|
}
|
||||||
engine, workflow = endpoint.EngineComfyUI, w
|
engine, workflow = endpoint.EngineComfyUI, w
|
||||||
}
|
}
|
||||||
out = append(out, endpoint.Info{
|
base := endpoint.Info{
|
||||||
ID: id, URL: strings.TrimRight(m.External.URL, "/"),
|
ID: id, URL: strings.TrimRight(m.External.URL, "/"),
|
||||||
Models: append([]string{m.Name}, m.Aliases...), APIKey: key,
|
Models: append([]string{m.Name}, m.Aliases...),
|
||||||
Engine: engine, Pool: "external", Weight: 1,
|
Engine: engine, Pool: "external", Weight: 1,
|
||||||
UpstreamModel: m.External.Model,
|
UpstreamModel: m.External.Model,
|
||||||
Peer: m.External.Peer, PeerCluster: cluster, PeerTags: tags,
|
Peer: m.External.Peer, PeerCluster: cluster, PeerTags: tags,
|
||||||
Workflow: workflow,
|
Workflow: workflow,
|
||||||
})
|
}
|
||||||
|
keys := m.External.APIKeyFiles
|
||||||
|
if len(keys) == 0 {
|
||||||
|
keys = []catalog.ExternalKey{{File: m.External.APIKeyFile}}
|
||||||
|
}
|
||||||
|
for i, k := range keys {
|
||||||
|
e := base
|
||||||
|
// One endpoint a key, numbered, so usage says which answered
|
||||||
|
// and a breaker stands for one key — never the key itself.
|
||||||
|
if len(m.External.APIKeyFiles) > 0 {
|
||||||
|
e.ID = fmt.Sprintf("%s#%d", id, i+1)
|
||||||
|
if k.Weight > 0 {
|
||||||
|
e.Weight = k.Weight
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if k.File != "" {
|
||||||
|
b, err := secrets.Read(context.Background(), k.File)
|
||||||
|
if err != nil {
|
||||||
|
c.log.Warn("an external model's key could not be read; not offered", "model", m.Name,
|
||||||
|
"endpoint", e.ID, "err", err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
e.APIKey = strings.TrimSpace(string(b))
|
||||||
|
}
|
||||||
|
out = append(out, e)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -73,3 +73,28 @@ func TestAnExternalModelIsPublishedWithItsKeyAndNothingIsPlacedForIt(t *testing.
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Several keys for one provider: an endpoint each, numbered, with its
|
||||||
|
// weight; one whose file cannot be read is left out and the others stay.
|
||||||
|
func TestEachOfAProvidersKeysIsAnEndpointOfItsOwn(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
a, c := filepath.Join(dir, "a.key"), filepath.Join(dir, "c.key")
|
||||||
|
_ = os.WriteFile(a, []byte("sk-a\n"), 0o600)
|
||||||
|
_ = os.WriteFile(c, []byte("sk-c"), 0o600)
|
||||||
|
cat, err := catalog.New([]catalog.Model{{Name: "cloud", Source: "external",
|
||||||
|
External: &catalog.External{URL: "https://api.example.com", Model: "gpt-4o-mini",
|
||||||
|
APIKeyFiles: []catalog.ExternalKey{{File: a, Weight: 3}, {File: filepath.Join(dir, "gone")}, {File: c}}}}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
sink := &recordSink{}
|
||||||
|
New(DefaultConfig(), catalog.NewLive(cat), noDispatch{}, sink, slog.New(slog.DiscardHandler)).Publish(t.Context())
|
||||||
|
got := map[string]endpoint.Info{}
|
||||||
|
for _, e := range sink.infos {
|
||||||
|
got[e.ID] = e
|
||||||
|
}
|
||||||
|
if len(got) != 2 || got["external:cloud#1"].APIKey != "sk-a" || got["external:cloud#1"].Weight != 3 ||
|
||||||
|
got["external:cloud#3"].APIKey != "sk-c" || got["external:cloud#3"].Weight != 1 {
|
||||||
|
t.Errorf("published %+v", sink.infos)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -504,7 +504,11 @@ func (s *Server) run(j *Job) {
|
|||||||
req = fitted
|
req = fitted
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// Under the lock: a job is read by Get while it runs, and the race
|
||||||
|
// detector caught this write beside it.
|
||||||
|
s.mu.Lock()
|
||||||
j.Request = req
|
j.Request = req
|
||||||
|
s.mu.Unlock()
|
||||||
memGB := int((header.MemoryBytes(set.Bytes, req.CtxPool) + 1<<30 - 1) >> 30)
|
memGB := int((header.MemoryBytes(set.Bytes, req.CtxPool) + 1<<30 - 1) >> 30)
|
||||||
basis := fmt.Sprintf("weights %.1f GB + KV cache %.1f GB for %d tokens + engine",
|
basis := fmt.Sprintf("weights %.1f GB + KV cache %.1f GB for %d tokens + engine",
|
||||||
gb(set.Bytes), gb(int64(req.CtxPool)*header.KVBytesPerToken()), req.CtxPool)
|
gb(set.Bytes), gb(int64(req.CtxPool)*header.KVBytesPerToken()), req.CtxPool)
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
|
"slices"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -248,6 +249,19 @@ func (s *Service) Resources(ctx context.Context, name, profile string, memoryGB,
|
|||||||
return final, err
|
return final, err
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// SetProfiles replaces a model's profiles, as a model added with them
|
||||||
|
// would have: for a model whose engines are to run otherwise, its args among
|
||||||
|
// it. Running replicas keep what they were started with until restarted.
|
||||||
|
func (s *Service) SetProfiles(ctx context.Context, name string, profiles []catalog.Profile, by string) (catalog.Model, error) {
|
||||||
|
if len(profiles) == 0 {
|
||||||
|
return catalog.Model{}, errors.New("a model needs a profile")
|
||||||
|
}
|
||||||
|
return s.update(ctx, name, by, "replace its profiles", func(m *catalog.Model) error {
|
||||||
|
m.Profiles = slices.Clone(profiles)
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// SetClass sets a model's capacity class: critical, standard or
|
// SetClass sets a model's capacity class: critical, standard or
|
||||||
// opportunistic, or empty to go back to what its scaling implies.
|
// opportunistic, or empty to go back to what its scaling implies.
|
||||||
func (s *Service) SetClass(ctx context.Context, name, class, by string) (catalog.Model, error) {
|
func (s *Service) SetClass(ctx context.Context, name, class, by string) (catalog.Model, error) {
|
||||||
|
|||||||
@@ -915,7 +915,20 @@ func (r *Reconciler) reconcileModel(ctx context.Context, m catalog.Model) {
|
|||||||
// A drained replica is stopped once its replacement is serving, or once the
|
// A drained replica is stopped once its replacement is serving, or once the
|
||||||
// grace period expires. Stopping it the moment it was drained would cut
|
// grace period expires. Stopping it the moment it was drained would cut
|
||||||
// in-flight requests, which is the thing draining exists to avoid.
|
// in-flight requests, which is the thing draining exists to avoid.
|
||||||
replaced := len(liveIDs) >= desired
|
//
|
||||||
|
// Serving counts a replica being restarted that still answers: with
|
||||||
|
// replicas restarted one after another, the second's retiring kept the
|
||||||
|
// first's drained one waiting out its grace — and with it the second's
|
||||||
|
// replacement, on a node with no room until the first had gone.
|
||||||
|
serving := len(liveIDs)
|
||||||
|
r.mu.Lock()
|
||||||
|
for _, id := range retiringIDs {
|
||||||
|
if rep, ok := r.replicas[id]; ok && rep.State == Ready {
|
||||||
|
serving++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
r.mu.Unlock()
|
||||||
|
replaced := serving >= desired
|
||||||
for _, id := range drainingIDs {
|
for _, id := range drainingIDs {
|
||||||
r.mu.Lock()
|
r.mu.Lock()
|
||||||
rep, ok := r.replicas[id]
|
rep, ok := r.replicas[id]
|
||||||
|
|||||||
@@ -2436,3 +2436,51 @@ func TestTheRoomAnInPlaceRestartFreesIsItsReplacements(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Two models restarted together on a node whose cores are all reserved,
|
||||||
|
// one to need more and one less: the judge fits only once the starter
|
||||||
|
// has given back its cores. The judge's restart waits for that, rather
|
||||||
|
// than being refused as if nothing would ever make room. Met on a 4-core
|
||||||
|
// single-machine install after `ctl model resources` on both.
|
||||||
|
func TestARestartWaitsForRoomAnotherRestartWillGiveBack(t *testing.T) {
|
||||||
|
cpu := inventory.Node{ID: "a", MgmtAddr: "10.0.0.1", State: inventory.Ready, MemoryTotalMB: 16 * 1024, CPUCount: 4}
|
||||||
|
judge := catalog.Model{Name: "judge", Source: "/m", Scaling: catalog.Scaling{Min: 2, Max: 2},
|
||||||
|
Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 2, CPUCores: 1}}}
|
||||||
|
starter := catalog.Model{Name: "starter", Source: "/m", Scaling: catalog.Scaling{Min: 1, Max: 1},
|
||||||
|
Profiles: []catalog.Profile{{Name: "cpu", Engine: "llamacpp", MinMemoryGB: 2, CPUCores: 2}}}
|
||||||
|
h := newFleetHarness(t, []catalog.Model{judge, starter}, cpu)
|
||||||
|
h.settle(t, 3)
|
||||||
|
var old []string
|
||||||
|
for _, rep := range h.r.ReadyReplicas() {
|
||||||
|
old = append(old, rep.ID)
|
||||||
|
}
|
||||||
|
if len(old) != 3 {
|
||||||
|
t.Fatalf("%d replicas ready, want 3", len(old))
|
||||||
|
}
|
||||||
|
|
||||||
|
judge.Profiles[0].CPUCores, starter.Profiles[0].CPUCores = 2, 0
|
||||||
|
cat, err := catalog.New([]catalog.Model{judge, starter})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
h.r.cat.Store(cat)
|
||||||
|
if err := h.r.Restart(old...); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for range 12 {
|
||||||
|
h.r.Reconcile(t.Context())
|
||||||
|
h.markReady()
|
||||||
|
h.advance(10 * time.Second)
|
||||||
|
}
|
||||||
|
for _, id := range old {
|
||||||
|
if st := h.r.RestartState(id); strings.HasPrefix(st, "refused") {
|
||||||
|
t.Errorf("%s: %s", id, st)
|
||||||
|
}
|
||||||
|
if rep, ok := replicaByID(h, id); ok && rep.State == Ready {
|
||||||
|
t.Errorf("%s still serves on its old figures", id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if j, s := replicasOf(h, "judge"), replicasOf(h, "starter"); j != 2 || s != 1 {
|
||||||
|
t.Errorf("judge %d, starter %d after the restarts", j, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -22,6 +22,10 @@ import (
|
|||||||
// stopped now, and its replacement takes its place. Waiting would change
|
// stopped now, and its replacement takes its place. Waiting would change
|
||||||
// nothing; it used to wait DrainGrace anyway, five minutes reading only
|
// nothing; it used to wait DrainGrace anyway, five minutes reading only
|
||||||
// "restart replacing", which looked like a restart that never would.
|
// "restart replacing", which looked like a restart that never would.
|
||||||
|
// - other replicas whose own restarts are under way: the replacement fits
|
||||||
|
// once they have restarted, as a starter edited to reserve fewer cores
|
||||||
|
// gives back what a judge edited to reserve more needs. Waited for, up
|
||||||
|
// to DrainGrace, so that two restarts each waiting on the other end.
|
||||||
// - more than it: the replacement fits nowhere even with it stopped — its
|
// - more than it: the replacement fits nowhere even with it stopped — its
|
||||||
// profile was edited to need more than any node has free. Stopping it
|
// profile was edited to need more than any node has free. Stopping it
|
||||||
// would leave the model with nothing, so the restart is given up, and
|
// would leave the model with nothing, so the restart is given up, and
|
||||||
@@ -62,6 +66,8 @@ func (r *Reconciler) retire(ctx context.Context, m catalog.Model, ids []string,
|
|||||||
r.log.Warn("restarting a replica in place: it is the only thing in its replacement's way",
|
r.log.Warn("restarting a replica in place: it is the only thing in its replacement's way",
|
||||||
"replica", id, "model", m.Name)
|
"replica", id, "model", m.Name)
|
||||||
r.stopForRestart(ctx, id, "restarted in place: it was the only thing in its replacement's way")
|
r.stopForRestart(ctx, id, "restarted in place: it was the only thing in its replacement's way")
|
||||||
|
case outcome == noRoom && r.now().Sub(since) <= r.cfg.DrainGrace && r.fitsAfterRestarts(m, id):
|
||||||
|
// Another restart under way gives the room back; waited for.
|
||||||
case outcome == noRoom || outcome == overLimit:
|
case outcome == noRoom || outcome == overLimit:
|
||||||
r.refuseRestart(ctx, id, "its replacement fits on no node even with this replica stopped "+
|
r.refuseRestart(ctx, id, "its replacement fits on no node even with this replica stopped "+
|
||||||
"(llmgw ctl explain-placement "+m.Name+")")
|
"(llmgw ctl explain-placement "+m.Name+")")
|
||||||
@@ -73,6 +79,28 @@ func (r *Reconciler) retire(ctx context.Context, m catalog.Model, ids []string,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// fitsAfterRestarts is whether a model's replacement for a replica fits with
|
||||||
|
// that replica gone and every other on its way out: queued to restart, being
|
||||||
|
// replaced, or draining. Their replacements take room too, so this is
|
||||||
|
// whether waiting could help, not a promise that it will.
|
||||||
|
func (r *Reconciler) fitsAfterRestarts(m catalog.Model, id string) bool {
|
||||||
|
gone := []victim{{ID: id, Model: m.Name}}
|
||||||
|
r.mu.Lock()
|
||||||
|
for oid, o := range r.replicas {
|
||||||
|
if oid == id {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if o.State == Draining || !o.RetiringAt.IsZero() || slices.Contains(r.restartQueue[o.Model], oid) {
|
||||||
|
gone = append(gone, victim{ID: oid, Model: o.Model})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
r.mu.Unlock()
|
||||||
|
if len(gone) == 1 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return r.fitsWithout(m, nil, gone)
|
||||||
|
}
|
||||||
|
|
||||||
// stopForRestart drains a replica and stops it at once, and places its
|
// stopForRestart drains a replica and stops it at once, and places its
|
||||||
// replacement in what it freed.
|
// replacement in what it freed.
|
||||||
//
|
//
|
||||||
|
|||||||
@@ -0,0 +1,395 @@
|
|||||||
|
// Package directory keeps accounts in step with an outside directory —
|
||||||
|
// Active Directory over LDAP — at sign-in and by a periodic sync: made for
|
||||||
|
// people the directory vouches for, given the staff role and the places in
|
||||||
|
// the tree their groups map to, and suspended when the directory disables or
|
||||||
|
// loses them.
|
||||||
|
package directory
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Identity is a person as the directory has them.
|
||||||
|
type Identity struct {
|
||||||
|
// ID never changes for the person: an AD objectGUID.
|
||||||
|
ID string
|
||||||
|
DN string
|
||||||
|
Email string
|
||||||
|
Name string
|
||||||
|
// Groups are the person's groups, nested ones included, each by its CN
|
||||||
|
// and by its DN, so a mapping may name either.
|
||||||
|
Groups []string
|
||||||
|
// Disabled is an account the directory has turned off.
|
||||||
|
Disabled bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// Mapping is what a person's groups give them: a staff role and places in
|
||||||
|
// the tree. The same shape as auth.oidc's.
|
||||||
|
type Mapping struct {
|
||||||
|
Roles map[string][]string
|
||||||
|
Memberships map[string]string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Role is the highest staff role the groups give, "" when none.
|
||||||
|
func (m Mapping) Role(groups []string) string {
|
||||||
|
for _, r := range []string{store.RoleAdmin, store.RoleOperator, store.RoleBilling, store.RoleViewer} {
|
||||||
|
if slices.ContainsFunc(m.Roles[r], func(g string) bool { return hasGroup(groups, g) }) {
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// Places are the places in the tree the groups give, each once.
|
||||||
|
func (m Mapping) Places(groups []string) []string {
|
||||||
|
var out []string
|
||||||
|
for g, where := range m.Memberships {
|
||||||
|
if hasGroup(groups, g) && !slices.Contains(out, where) {
|
||||||
|
out = append(out, where)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
slices.Sort(out)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// Groups are every group the mapping names.
|
||||||
|
func (m Mapping) Groups() []string {
|
||||||
|
var out []string
|
||||||
|
add := func(g string) {
|
||||||
|
if !slices.ContainsFunc(out, func(o string) bool { return strings.EqualFold(o, g) }) {
|
||||||
|
out = append(out, g)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, gs := range m.Roles {
|
||||||
|
for _, g := range gs {
|
||||||
|
add(g)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for g := range m.Memberships {
|
||||||
|
add(g)
|
||||||
|
}
|
||||||
|
slices.Sort(out)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// hasGroup matches a group's CN or DN as AD does: case aside.
|
||||||
|
func hasGroup(groups []string, g string) bool {
|
||||||
|
return slices.ContainsFunc(groups, func(have string) bool { return strings.EqualFold(have, g) })
|
||||||
|
}
|
||||||
|
|
||||||
|
// Store is what keeping accounts needs of the store.
|
||||||
|
type Store interface {
|
||||||
|
UserByEmail(ctx context.Context, email string) (store.User, error)
|
||||||
|
UserByID(ctx context.Context, id string) (store.User, error)
|
||||||
|
CreateUser(ctx context.Context, u store.User) error
|
||||||
|
SetStaffRole(ctx context.Context, id, role string) error
|
||||||
|
SetUserState(ctx context.Context, id string, state store.UserState) error
|
||||||
|
DeleteUserSessions(ctx context.Context, userID string) error
|
||||||
|
AddMembership(ctx context.Context, user string, scope store.Scope, id, role, by string, at time.Time) error
|
||||||
|
RemoveMembership(ctx context.Context, userID string, scope store.Scope, scopeID string, at time.Time) (int, error)
|
||||||
|
MembershipsOf(ctx context.Context, userID string) ([]store.Membership, error)
|
||||||
|
AllOrgs(ctx context.Context) ([]store.Org, error)
|
||||||
|
AllTeams(ctx context.Context) ([]store.TeamPath, error)
|
||||||
|
AllProjects(ctx context.Context) ([]store.ProjectPath, error)
|
||||||
|
DirectoryUser(ctx context.Context, directory, externalID string) (store.DirectoryLink, error)
|
||||||
|
LinkDirectory(ctx context.Context, userID, directory, externalID string, at time.Time) error
|
||||||
|
DirectoryLinks(ctx context.Context, directory string) ([]store.DirectoryLink, error)
|
||||||
|
SetSyncSuspended(ctx context.Context, userID string, suspended bool, at time.Time) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Keeper keeps one directory's accounts.
|
||||||
|
type Keeper struct {
|
||||||
|
St Store
|
||||||
|
// Name is the directory's, as links and memberships record it: "ldap".
|
||||||
|
Name string
|
||||||
|
Mapping Mapping
|
||||||
|
CreateAccounts bool
|
||||||
|
Now func() time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// The ways a person the directory vouches for is still refused.
|
||||||
|
var (
|
||||||
|
ErrNoAccount = errors.New("no account here, and accounts are not made for the directory's people")
|
||||||
|
ErrSuspended = errors.New("the account is suspended here")
|
||||||
|
ErrNotActive = errors.New("the account is not active here")
|
||||||
|
ErrNoEmail = errors.New("the directory has no mail for this person")
|
||||||
|
// ErrImplausible is a sync that would suspend too many at once: done
|
||||||
|
// nothing, and said.
|
||||||
|
ErrImplausible = errors.New("the directory's answer is implausible; nobody was suspended")
|
||||||
|
)
|
||||||
|
|
||||||
|
func (k *Keeper) now() time.Time {
|
||||||
|
if k.Now != nil {
|
||||||
|
return k.Now()
|
||||||
|
}
|
||||||
|
return time.Now()
|
||||||
|
}
|
||||||
|
|
||||||
|
// SignIn is the account for a person the directory has just vouched for:
|
||||||
|
// found by their directory id, else their mail, else made; given what their
|
||||||
|
// groups map to; refused when it may not sign in.
|
||||||
|
func (k *Keeper) SignIn(ctx context.Context, id Identity) (store.User, error) {
|
||||||
|
u, link, err := k.account(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return store.User{}, err
|
||||||
|
}
|
||||||
|
// The directory has them enabled, or the bind would have failed: an
|
||||||
|
// account the sync suspended for being disabled there is theirs again.
|
||||||
|
if u.State == store.UserSuspended && link.SuspendedBySync {
|
||||||
|
if u, err = k.reinstate(ctx, u); err != nil {
|
||||||
|
return store.User{}, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case u.State == store.UserSuspended:
|
||||||
|
return store.User{}, ErrSuspended
|
||||||
|
case !u.CanSignIn():
|
||||||
|
return store.User{}, fmt.Errorf("%w: %s", ErrNotActive, u.State)
|
||||||
|
}
|
||||||
|
if err := k.apply(ctx, &u, id, false); err != nil {
|
||||||
|
return store.User{}, err
|
||||||
|
}
|
||||||
|
return u, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// account finds or makes the account and links it to the directory.
|
||||||
|
func (k *Keeper) account(ctx context.Context, id Identity) (store.User, store.DirectoryLink, error) {
|
||||||
|
link, err := k.St.DirectoryUser(ctx, k.Name, id.ID)
|
||||||
|
switch {
|
||||||
|
case err == nil:
|
||||||
|
u, err := k.St.UserByID(ctx, link.UserID)
|
||||||
|
return u, link, err
|
||||||
|
case !errors.Is(err, store.ErrNotFound):
|
||||||
|
return store.User{}, link, err
|
||||||
|
}
|
||||||
|
email := store.NormalizeEmail(id.Email)
|
||||||
|
if email == "" {
|
||||||
|
return store.User{}, link, ErrNoEmail
|
||||||
|
}
|
||||||
|
u, err := k.St.UserByEmail(ctx, email)
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, store.ErrNotFound) && k.CreateAccounts:
|
||||||
|
name := id.Name
|
||||||
|
if name == "" {
|
||||||
|
name = email
|
||||||
|
}
|
||||||
|
u = store.User{ID: "usr-" + randomID(), Email: email, Name: name,
|
||||||
|
// No password works here: the directory's is checked.
|
||||||
|
PasswordHash: "!" + k.Name, State: store.UserActive, CreatedAt: k.now()}
|
||||||
|
if err := k.St.CreateUser(ctx, u); err != nil {
|
||||||
|
return store.User{}, link, err
|
||||||
|
}
|
||||||
|
case errors.Is(err, store.ErrNotFound):
|
||||||
|
return store.User{}, link, fmt.Errorf("%w: %s", ErrNoAccount, email)
|
||||||
|
case err != nil:
|
||||||
|
return store.User{}, link, err
|
||||||
|
}
|
||||||
|
if err := k.St.LinkDirectory(ctx, u.ID, k.Name, id.ID, k.now()); err != nil {
|
||||||
|
return store.User{}, link, err
|
||||||
|
}
|
||||||
|
return u, store.DirectoryLink{UserID: u.ID, Directory: k.Name, ExternalID: id.ID}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// apply gives the account the role and places its groups map to. With
|
||||||
|
// remove, a place this directory gave that the groups no longer do is taken
|
||||||
|
// away; one given otherwise is never touched.
|
||||||
|
func (k *Keeper) apply(ctx context.Context, u *store.User, id Identity, remove bool) error {
|
||||||
|
if len(k.Mapping.Roles) > 0 {
|
||||||
|
if role := k.Mapping.Role(id.Groups); role != u.StaffRole() {
|
||||||
|
if err := k.St.SetStaffRole(ctx, u.ID, role); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
u.IsAdmin, u.Role = role == store.RoleAdmin, role
|
||||||
|
if role == store.RoleAdmin {
|
||||||
|
u.Role = ""
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
want := map[string]store.Scope{}
|
||||||
|
for _, where := range k.Mapping.Places(id.Groups) {
|
||||||
|
scope, sid, err := k.place(ctx, where)
|
||||||
|
if err != nil {
|
||||||
|
continue // a mapping to a place the tree lacks: said by Sync
|
||||||
|
}
|
||||||
|
want[string(scope)+"/"+sid] = scope
|
||||||
|
if err := k.St.AddMembership(ctx, u.ID, scope, sid, store.RoleMember, k.Name, k.now()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !remove {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
have, err := k.St.MembershipsOf(ctx, u.ID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for _, m := range have {
|
||||||
|
if m.AddedBy != k.Name {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := want[string(m.Scope)+"/"+m.ScopeID]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, err := k.St.RemoveMembership(ctx, u.ID, m.Scope, m.ScopeID, k.now()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (k *Keeper) reinstate(ctx context.Context, u store.User) (store.User, error) {
|
||||||
|
if err := k.St.SetUserState(ctx, u.ID, store.UserActive); err != nil {
|
||||||
|
return u, err
|
||||||
|
}
|
||||||
|
if err := k.St.SetSyncSuspended(ctx, u.ID, false, k.now()); err != nil {
|
||||||
|
return u, err
|
||||||
|
}
|
||||||
|
u.State = store.UserActive
|
||||||
|
return u, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SyncReport is what a sync did.
|
||||||
|
type SyncReport struct {
|
||||||
|
Seen, Updated, Suspended, Reinstated int
|
||||||
|
// Unmapped are mappings to places the tree does not have.
|
||||||
|
Unmapped []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sync brings every account the directory keeps in step with what it says
|
||||||
|
// now: people disabled or gone there are suspended here (their sessions
|
||||||
|
// ended; their keys refuse, as a suspended owner's do), those enabled again
|
||||||
|
// that the sync had suspended are reinstated, and everyone's role and places
|
||||||
|
// follow their groups. People the directory has that never signed in here
|
||||||
|
// are left until they do.
|
||||||
|
func (k *Keeper) Sync(ctx context.Context, people []Identity) (SyncReport, error) {
|
||||||
|
var rep SyncReport
|
||||||
|
rep.Unmapped = k.unmapped(ctx)
|
||||||
|
byID := make(map[string]Identity, len(people))
|
||||||
|
for _, p := range people {
|
||||||
|
byID[p.ID] = p
|
||||||
|
}
|
||||||
|
links, err := k.St.DirectoryLinks(ctx, k.Name)
|
||||||
|
if err != nil {
|
||||||
|
return rep, err
|
||||||
|
}
|
||||||
|
// A directory that answers with nobody, or with half the people missing,
|
||||||
|
// is far likelier a wrong base, a filter or a bind account's lost rights
|
||||||
|
// than half the company leaving at once. Nothing is suspended for it.
|
||||||
|
gone := 0
|
||||||
|
for _, l := range links {
|
||||||
|
if p, there := byID[l.ExternalID]; !there || p.Disabled {
|
||||||
|
gone++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(links) > 0 && (len(people) == 0 || (gone >= 3 && gone*2 > len(links))) {
|
||||||
|
return rep, fmt.Errorf("%w: %d of the %d accounts it keeps would be suspended", ErrImplausible, gone, len(links))
|
||||||
|
}
|
||||||
|
for _, l := range links {
|
||||||
|
rep.Seen++
|
||||||
|
u, err := k.St.UserByID(ctx, l.UserID)
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
p, there := byID[l.ExternalID]
|
||||||
|
switch {
|
||||||
|
case !there || p.Disabled:
|
||||||
|
if u.State == store.UserActive {
|
||||||
|
if err := k.St.SetUserState(ctx, u.ID, store.UserSuspended); err != nil {
|
||||||
|
return rep, err
|
||||||
|
}
|
||||||
|
if err := k.St.DeleteUserSessions(ctx, u.ID); err != nil {
|
||||||
|
return rep, err
|
||||||
|
}
|
||||||
|
if err := k.St.SetSyncSuspended(ctx, u.ID, true, k.now()); err != nil {
|
||||||
|
return rep, err
|
||||||
|
}
|
||||||
|
rep.Suspended++
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
if u.State == store.UserSuspended && l.SuspendedBySync {
|
||||||
|
if u, err = k.reinstate(ctx, u); err != nil {
|
||||||
|
return rep, err
|
||||||
|
}
|
||||||
|
rep.Reinstated++
|
||||||
|
}
|
||||||
|
if u.State != store.UserActive {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
before := u.StaffRole()
|
||||||
|
if err := k.apply(ctx, &u, p, true); err != nil {
|
||||||
|
return rep, err
|
||||||
|
}
|
||||||
|
if u.StaffRole() != before {
|
||||||
|
rep.Updated++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return rep, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// unmapped are the mapping's places the tree does not have.
|
||||||
|
func (k *Keeper) unmapped(ctx context.Context) []string {
|
||||||
|
var out []string
|
||||||
|
for _, where := range k.Mapping.Memberships {
|
||||||
|
if _, _, err := k.place(ctx, where); err != nil && !slices.Contains(out, where) {
|
||||||
|
out = append(out, where)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
slices.Sort(out)
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// place finds org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT, as
|
||||||
|
// auth.oidc's mapping does.
|
||||||
|
func (k *Keeper) place(ctx context.Context, where string) (store.Scope, string, error) {
|
||||||
|
kind, path, _ := strings.Cut(where, ":")
|
||||||
|
norm := func(s string) string {
|
||||||
|
parts := strings.Split(s, "/")
|
||||||
|
for i := range parts {
|
||||||
|
parts[i] = strings.TrimSpace(parts[i])
|
||||||
|
}
|
||||||
|
return strings.Join(parts, "/")
|
||||||
|
}
|
||||||
|
same := func(a string) bool { return strings.EqualFold(norm(a), norm(path)) }
|
||||||
|
switch kind {
|
||||||
|
case "org":
|
||||||
|
orgs, err := k.St.AllOrgs(ctx)
|
||||||
|
for _, o := range orgs {
|
||||||
|
if same(o.Name) {
|
||||||
|
return store.ScopeOrg, o.ID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", "", errors.Join(err, errors.New("no such organization"))
|
||||||
|
case "team":
|
||||||
|
teams, err := k.St.AllTeams(ctx)
|
||||||
|
for _, t := range teams {
|
||||||
|
if same(t.OrgName + "/" + t.Name) {
|
||||||
|
return store.ScopeTeam, t.ID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", "", errors.Join(err, errors.New("no such team"))
|
||||||
|
default:
|
||||||
|
projects, err := k.St.AllProjects(ctx)
|
||||||
|
for _, p := range projects {
|
||||||
|
if same(p.Path) {
|
||||||
|
return store.ScopeProject, p.ID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", "", errors.Join(err, errors.New("no such project"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func randomID() string {
|
||||||
|
var b [8]byte
|
||||||
|
_, _ = rand.Read(b[:])
|
||||||
|
return hex.EncodeToString(b[:])
|
||||||
|
}
|
||||||
@@ -0,0 +1,142 @@
|
|||||||
|
package directory
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"path/filepath"
|
||||||
|
"slices"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func newKeeper(t *testing.T) (*Keeper, *store.DB, string) {
|
||||||
|
t.Helper()
|
||||||
|
st, err := store.Open(t.Context(), filepath.Join(t.TempDir(), "s.db"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { st.Close() })
|
||||||
|
org, _ := st.CreateOrg(t.Context(), "Acme")
|
||||||
|
team, _ := st.CreateTeam(t.Context(), org, "ml")
|
||||||
|
k := &Keeper{St: st, Name: "ldap", CreateAccounts: true, Mapping: Mapping{
|
||||||
|
Roles: map[string][]string{"admin": {"llmgw-admins"}, "viewer": {"CN=Staff,OU=Groups,DC=corp,DC=example"}},
|
||||||
|
Memberships: map[string]string{"ml-team": "team:Acme/ml", "ghosts": "team:Acme/nowhere"},
|
||||||
|
}}
|
||||||
|
return k, st, team
|
||||||
|
}
|
||||||
|
|
||||||
|
var ada = Identity{ID: "guid-ada", DN: "CN=Ada,OU=Staff,DC=corp,DC=example", Email: "Ada@Corp.example", Name: "Ada L",
|
||||||
|
Groups: []string{"ml-team", "CN=ml-team,OU=Groups,DC=corp,DC=example", "Staff", "CN=Staff,OU=Groups,DC=corp,DC=example"}}
|
||||||
|
|
||||||
|
func TestAPersonTheDirectoryVouchesForIsMadeAnAccountWithWhatTheirGroupsGive(t *testing.T) {
|
||||||
|
k, st, team := newKeeper(t)
|
||||||
|
u, err := k.SignIn(t.Context(), ada)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if u.Email != "ada@corp.example" || u.Name != "Ada L" || u.PasswordHash != "!ldap" || u.StaffRole() != "viewer" {
|
||||||
|
t.Errorf("account %+v role %q", u, u.StaffRole())
|
||||||
|
}
|
||||||
|
ms, _ := st.MembershipsOf(t.Context(), u.ID)
|
||||||
|
if len(ms) != 1 || ms[0].ScopeID != team || ms[0].AddedBy != "ldap" {
|
||||||
|
t.Errorf("memberships %+v", ms)
|
||||||
|
}
|
||||||
|
// Their mail changes there: still the same account, by its directory id.
|
||||||
|
moved := ada
|
||||||
|
moved.Email = "ada.lovelace@corp.example"
|
||||||
|
again, err := k.SignIn(t.Context(), moved)
|
||||||
|
if err != nil || again.ID != u.ID {
|
||||||
|
t.Errorf("a new mail made a new account: %v %+v", err, again)
|
||||||
|
}
|
||||||
|
// The higher role of two wins.
|
||||||
|
boss := ada
|
||||||
|
boss.Groups = append(slices.Clone(ada.Groups), "llmgw-admins")
|
||||||
|
if u, _ := k.SignIn(t.Context(), boss); u.StaffRole() != "admin" {
|
||||||
|
t.Errorf("role %q", u.StaffRole())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWithoutAccountsMadeOnlyThoseThatExistSignIn(t *testing.T) {
|
||||||
|
k, _, _ := newKeeper(t)
|
||||||
|
k.CreateAccounts = false
|
||||||
|
if _, err := k.SignIn(t.Context(), ada); !errors.Is(err, ErrNoAccount) {
|
||||||
|
t.Errorf("err %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTheSyncSuspendsWhomTheDirectoryDisabledAndReinstatesThemAfter(t *testing.T) {
|
||||||
|
k, st, _ := newKeeper(t)
|
||||||
|
ctx := t.Context()
|
||||||
|
people := []Identity{ada}
|
||||||
|
for _, n := range []string{"b", "c", "d"} {
|
||||||
|
people = append(people, Identity{ID: "guid-" + n, Email: n + "@corp.example"})
|
||||||
|
}
|
||||||
|
ids := map[string]string{}
|
||||||
|
for _, p := range people {
|
||||||
|
u, err := k.SignIn(ctx, p)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ids[p.ID] = u.ID
|
||||||
|
}
|
||||||
|
if err := st.CreateSession(ctx, ids["guid-ada"], "tok", "csrf", time.Hour); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// An administrator suspends c here, by hand.
|
||||||
|
_ = st.SetUserState(ctx, ids["guid-c"], store.UserSuspended)
|
||||||
|
|
||||||
|
// Ada is disabled there, and has left ml-team.
|
||||||
|
off := ada
|
||||||
|
off.Disabled = true
|
||||||
|
rep, err := k.Sync(ctx, []Identity{off, people[1], people[2], people[3]})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
u, _ := st.UserByID(ctx, ids["guid-ada"])
|
||||||
|
if u.State != store.UserSuspended || rep.Suspended != 1 {
|
||||||
|
t.Errorf("ada %s, report %+v", u.State, rep)
|
||||||
|
}
|
||||||
|
if _, _, err := st.SessionByToken(ctx, "tok"); err == nil {
|
||||||
|
t.Error("a suspended account's session still works")
|
||||||
|
}
|
||||||
|
if !slices.Contains(rep.Unmapped, "team:Acme/nowhere") {
|
||||||
|
t.Errorf("an unmapped place was not said: %+v", rep)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Enabled again, and out of ml-team: reinstated, and its place taken
|
||||||
|
// away. c, suspended by hand, stays so.
|
||||||
|
back := ada
|
||||||
|
back.Groups = []string{"Staff", "CN=Staff,OU=Groups,DC=corp,DC=example"}
|
||||||
|
rep, err = k.Sync(ctx, []Identity{back, people[1], people[2], people[3]})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
u, _ = st.UserByID(ctx, ids["guid-ada"])
|
||||||
|
ms, _ := st.MembershipsOf(ctx, u.ID)
|
||||||
|
if u.State != store.UserActive || rep.Reinstated != 1 || len(ms) != 0 {
|
||||||
|
t.Errorf("ada %s, memberships %+v, report %+v", u.State, ms, rep)
|
||||||
|
}
|
||||||
|
if c, _ := st.UserByID(ctx, ids["guid-c"]); c.State != store.UserSuspended {
|
||||||
|
t.Errorf("an account suspended by hand was reinstated: %s", c.State)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestASyncThatWouldSuspendMostIsRefused(t *testing.T) {
|
||||||
|
k, st, _ := newKeeper(t)
|
||||||
|
ctx := t.Context()
|
||||||
|
for _, n := range []string{"a", "b", "c", "d"} {
|
||||||
|
if _, err := k.SignIn(ctx, Identity{ID: "guid-" + n, Email: n + "@corp.example"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, people := range [][]Identity{nil, {{ID: "guid-a", Email: "a@corp.example"}}} {
|
||||||
|
if _, err := k.Sync(ctx, people); !errors.Is(err, ErrImplausible) {
|
||||||
|
t.Errorf("%d people: %v", len(people), err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
users, _ := st.Users(ctx, store.UserSuspended)
|
||||||
|
if len(users) != 0 {
|
||||||
|
t.Errorf("suspended %d", len(users))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
package directory
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/tls"
|
||||||
|
"crypto/x509"
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/go-ldap/ldap/v3"
|
||||||
|
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/secrets"
|
||||||
|
)
|
||||||
|
|
||||||
|
// LDAP is a directory reached over LDAP, written for Active Directory: its
|
||||||
|
// attribute names, its disabled flag, and its matching rule for nested
|
||||||
|
// groups (LDAP_MATCHING_RULE_IN_CHAIN).
|
||||||
|
type LDAP struct {
|
||||||
|
cfg config.LDAP
|
||||||
|
tls *tls.Config
|
||||||
|
dial func(url string) (ldapConn, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ldapConn is the part of a connection used here, so tests need no server.
|
||||||
|
type ldapConn interface {
|
||||||
|
Bind(user, password string) error
|
||||||
|
Search(*ldap.SearchRequest) (*ldap.SearchResult, error)
|
||||||
|
SearchWithPaging(*ldap.SearchRequest, uint32) (*ldap.SearchResult, error)
|
||||||
|
Close() error
|
||||||
|
}
|
||||||
|
|
||||||
|
// inChain is AD's matching rule that follows nested group membership.
|
||||||
|
const inChain = "1.2.840.113556.1.4.1941"
|
||||||
|
|
||||||
|
// uacDisabled is userAccountControl's ACCOUNTDISABLE bit.
|
||||||
|
const uacDisabled = 0x2
|
||||||
|
|
||||||
|
var personAttrs = []string{"objectGUID", "sAMAccountName", "userPrincipalName", "mail", "displayName", "userAccountControl"}
|
||||||
|
|
||||||
|
// ErrBadCredentials is a person the directory does not know, or a password
|
||||||
|
// it does not take: said alike, so a form cannot tell which.
|
||||||
|
var ErrBadCredentials = errors.New("the directory does not take that name and password")
|
||||||
|
|
||||||
|
// NewLDAP prepares a directory from its configuration.
|
||||||
|
func NewLDAP(cfg config.LDAP) (*LDAP, error) {
|
||||||
|
tc := &tls.Config{MinVersion: tls.VersionTLS12}
|
||||||
|
if cfg.CAFile != "" {
|
||||||
|
pem, err := os.ReadFile(cfg.CAFile)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("auth.ldap.ca_file: %w", err)
|
||||||
|
}
|
||||||
|
pool := x509.NewCertPool()
|
||||||
|
if !pool.AppendCertsFromPEM(pem) {
|
||||||
|
return nil, fmt.Errorf("auth.ldap.ca_file %s holds no certificate", cfg.CAFile)
|
||||||
|
}
|
||||||
|
tc.RootCAs = pool
|
||||||
|
}
|
||||||
|
d := &LDAP{cfg: cfg, tls: tc}
|
||||||
|
d.dial = d.dialURL
|
||||||
|
return d, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *LDAP) dialURL(u string) (ldapConn, error) {
|
||||||
|
c, err := ldap.DialURL(u, ldap.DialWithTLSConfig(d.tls))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
c.SetTimeout(d.cfg.RoundTrip())
|
||||||
|
if strings.HasPrefix(u, "ldap://") {
|
||||||
|
// Validate refuses ldap:// without start_tls: nothing is sent first.
|
||||||
|
if err := c.StartTLS(d.tls); err != nil {
|
||||||
|
c.Close()
|
||||||
|
return nil, fmt.Errorf("StartTLS: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// service is a connection bound as the account that finds people, to the
|
||||||
|
// first server that answers.
|
||||||
|
func (d *LDAP) service(ctx context.Context) (ldapConn, error) {
|
||||||
|
pw, err := secrets.Read(ctx, d.cfg.BindPasswordFile)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("auth.ldap.bind_password_file: %w", err)
|
||||||
|
}
|
||||||
|
var errs []error
|
||||||
|
for _, u := range d.cfg.URLs {
|
||||||
|
c, err := d.dial(u)
|
||||||
|
if err != nil {
|
||||||
|
errs = append(errs, fmt.Errorf("%s: %w", u, err))
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := c.Bind(d.cfg.BindDN, strings.TrimSpace(string(pw))); err != nil {
|
||||||
|
c.Close()
|
||||||
|
// A server that refuses the service account refuses it
|
||||||
|
// everywhere: not tried on the next.
|
||||||
|
return nil, fmt.Errorf("binding as %s on %s: %w", d.cfg.BindDN, u, err)
|
||||||
|
}
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("no directory server answered: %w", errors.Join(errs...))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Authenticate checks a person's name and password against the directory
|
||||||
|
// and says who they are there, groups and all.
|
||||||
|
func (d *LDAP) Authenticate(ctx context.Context, login, password string) (Identity, error) {
|
||||||
|
login = strings.TrimSpace(login)
|
||||||
|
// An empty password is an unauthenticated bind, which succeeds: never
|
||||||
|
// one for a person.
|
||||||
|
if login == "" || password == "" {
|
||||||
|
return Identity{}, ErrBadCredentials
|
||||||
|
}
|
||||||
|
// DOMAIN\user, as Windows writes it.
|
||||||
|
if i := strings.LastIndexByte(login, '\\'); i >= 0 {
|
||||||
|
login = login[i+1:]
|
||||||
|
}
|
||||||
|
c, err := d.service(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return Identity{}, err
|
||||||
|
}
|
||||||
|
defer c.Close()
|
||||||
|
q := ldap.EscapeFilter(login)
|
||||||
|
filter := "(&" + d.people(false) + "(|(sAMAccountName=" + q + ")(userPrincipalName=" + q + ")(mail=" + q + ")))"
|
||||||
|
res, err := c.Search(ldap.NewSearchRequest(d.cfg.BaseDN, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases,
|
||||||
|
2, 0, false, filter, personAttrs, nil))
|
||||||
|
if err != nil {
|
||||||
|
return Identity{}, fmt.Errorf("finding %s: %w", login, err)
|
||||||
|
}
|
||||||
|
if len(res.Entries) != 1 {
|
||||||
|
// None, or two people one name matches: neither is let in.
|
||||||
|
return Identity{}, ErrBadCredentials
|
||||||
|
}
|
||||||
|
id := identity(res.Entries[0])
|
||||||
|
if err := c.Bind(id.DN, password); err != nil {
|
||||||
|
var le *ldap.Error
|
||||||
|
if errors.As(err, &le) && le.ResultCode == ldap.LDAPResultInvalidCredentials {
|
||||||
|
return Identity{}, ErrBadCredentials
|
||||||
|
}
|
||||||
|
return Identity{}, fmt.Errorf("checking %s's password: %w", login, err)
|
||||||
|
}
|
||||||
|
// Back to the service account to read groups, which a person may not.
|
||||||
|
pw, err := secrets.Read(ctx, d.cfg.BindPasswordFile)
|
||||||
|
if err != nil {
|
||||||
|
return Identity{}, err
|
||||||
|
}
|
||||||
|
if err := c.Bind(d.cfg.BindDN, strings.TrimSpace(string(pw))); err != nil {
|
||||||
|
return Identity{}, err
|
||||||
|
}
|
||||||
|
if id.Groups, err = d.groupsOf(c, id.DN); err != nil {
|
||||||
|
return Identity{}, err
|
||||||
|
}
|
||||||
|
return id, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// groupsOf are a person's groups, nested ones included, by CN and DN.
|
||||||
|
func (d *LDAP) groupsOf(c ldapConn, dn string) ([]string, error) {
|
||||||
|
filter := "(&(objectClass=group)(member:" + inChain + ":=" + ldap.EscapeFilter(dn) + "))"
|
||||||
|
res, err := c.SearchWithPaging(ldap.NewSearchRequest(d.groupBase(), ldap.ScopeWholeSubtree, ldap.NeverDerefAliases,
|
||||||
|
0, 0, false, filter, []string{"cn"}, nil), 500)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("reading the groups of %s: %w", dn, err)
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, e := range res.Entries {
|
||||||
|
out = append(out, e.GetAttributeValue("cn"), e.DN)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// People are everyone the directory has under the base, enabled or not,
|
||||||
|
// each with the mapped groups they are in: what a sync compares against.
|
||||||
|
func (d *LDAP) People(ctx context.Context, m Mapping) ([]Identity, error) {
|
||||||
|
c, err := d.service(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer c.Close()
|
||||||
|
res, err := c.SearchWithPaging(ldap.NewSearchRequest(d.cfg.BaseDN, ldap.ScopeWholeSubtree, ldap.NeverDerefAliases,
|
||||||
|
0, 0, false, d.people(true), personAttrs, nil), 500)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("listing people: %w", err)
|
||||||
|
}
|
||||||
|
people := make([]Identity, 0, len(res.Entries))
|
||||||
|
at := map[string]int{}
|
||||||
|
for _, e := range res.Entries {
|
||||||
|
id := identity(e)
|
||||||
|
at[id.ID] = len(people)
|
||||||
|
people = append(people, id)
|
||||||
|
}
|
||||||
|
// Only the groups the mapping names matter: a query each, for its
|
||||||
|
// members nested ones included, rather than one per person.
|
||||||
|
for _, g := range m.Groups() {
|
||||||
|
dns, err := d.groupDNs(c, g)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, gdn := range dns {
|
||||||
|
filter := "(&" + d.people(true) + "(memberOf:" + inChain + ":=" + ldap.EscapeFilter(gdn) + "))"
|
||||||
|
r, err := c.SearchWithPaging(ldap.NewSearchRequest(d.cfg.BaseDN, ldap.ScopeWholeSubtree,
|
||||||
|
ldap.NeverDerefAliases, 0, 0, false, filter, []string{"objectGUID"}, nil), 500)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("reading the members of %s: %w", gdn, err)
|
||||||
|
}
|
||||||
|
for _, e := range r.Entries {
|
||||||
|
if i, ok := at[guid(e.GetRawAttributeValue("objectGUID"))]; ok {
|
||||||
|
people[i].Groups = append(people[i].Groups, g, gdn)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return people, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// groupDNs are the groups a mapping's name means: a DN as it is, a CN
|
||||||
|
// looked up.
|
||||||
|
func (d *LDAP) groupDNs(c ldapConn, g string) ([]string, error) {
|
||||||
|
if strings.Contains(g, "=") {
|
||||||
|
return []string{g}, nil
|
||||||
|
}
|
||||||
|
res, err := c.Search(ldap.NewSearchRequest(d.groupBase(), ldap.ScopeWholeSubtree, ldap.NeverDerefAliases,
|
||||||
|
0, 0, false, "(&(objectClass=group)(cn="+ldap.EscapeFilter(g)+"))", []string{"cn"}, nil))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("finding the group %s: %w", g, err)
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, e := range res.Entries {
|
||||||
|
out = append(out, e.DN)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// people is the filter for person accounts under the user filter; enabled
|
||||||
|
// ones only, unless all.
|
||||||
|
func (d *LDAP) people(all bool) string {
|
||||||
|
f := "(objectCategory=person)(objectClass=user)"
|
||||||
|
if !all {
|
||||||
|
f += "(!(userAccountControl:1.2.840.113556.1.4.803:=" + strconv.Itoa(uacDisabled) + "))"
|
||||||
|
}
|
||||||
|
return "(&" + f + d.cfg.UserFilter + ")"
|
||||||
|
}
|
||||||
|
|
||||||
|
// groupBase is where groups are searched: GroupBaseDN, else the domain root
|
||||||
|
// of BaseDN (its DC= parts).
|
||||||
|
func (d *LDAP) groupBase() string {
|
||||||
|
if d.cfg.GroupBaseDN != "" {
|
||||||
|
return d.cfg.GroupBaseDN
|
||||||
|
}
|
||||||
|
var dc []string
|
||||||
|
for _, rdn := range strings.Split(d.cfg.BaseDN, ",") {
|
||||||
|
if strings.HasPrefix(strings.ToUpper(strings.TrimSpace(rdn)), "DC=") {
|
||||||
|
dc = append(dc, strings.TrimSpace(rdn))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(dc) == 0 {
|
||||||
|
return d.cfg.BaseDN
|
||||||
|
}
|
||||||
|
return strings.Join(dc, ",")
|
||||||
|
}
|
||||||
|
|
||||||
|
func identity(e *ldap.Entry) Identity {
|
||||||
|
email := e.GetAttributeValue("mail")
|
||||||
|
if email == "" {
|
||||||
|
email = e.GetAttributeValue("userPrincipalName")
|
||||||
|
}
|
||||||
|
uac, _ := strconv.Atoi(e.GetAttributeValue("userAccountControl"))
|
||||||
|
return Identity{
|
||||||
|
ID: guid(e.GetRawAttributeValue("objectGUID")), DN: e.DN,
|
||||||
|
Email: email, Name: e.GetAttributeValue("displayName"),
|
||||||
|
Disabled: uac&uacDisabled != 0,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// guid is an objectGUID as Windows writes it: its first three parts
|
||||||
|
// little-endian. Another directory's id is kept as hex.
|
||||||
|
func guid(b []byte) string {
|
||||||
|
if len(b) != 16 {
|
||||||
|
return fmt.Sprintf("%x", b)
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%08x-%04x-%04x-%x-%x", binary.LittleEndian.Uint32(b[0:4]),
|
||||||
|
binary.LittleEndian.Uint16(b[4:6]), binary.LittleEndian.Uint16(b[6:8]), b[8:10], b[10:16])
|
||||||
|
}
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
package directory
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/go-ldap/ldap/v3"
|
||||||
|
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
// fakeConn is a directory with one person in it.
|
||||||
|
type fakeConn struct {
|
||||||
|
binds []string
|
||||||
|
filters []string
|
||||||
|
password string // the person's
|
||||||
|
people int // how many the person search finds
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeConn) Bind(user, password string) error {
|
||||||
|
f.binds = append(f.binds, user)
|
||||||
|
if strings.HasPrefix(user, "CN=Ada") && password != f.password {
|
||||||
|
return &ldap.Error{ResultCode: ldap.LDAPResultInvalidCredentials}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeConn) Search(r *ldap.SearchRequest) (*ldap.SearchResult, error) {
|
||||||
|
f.filters = append(f.filters, r.Filter)
|
||||||
|
var out []*ldap.Entry
|
||||||
|
for i := range f.people {
|
||||||
|
e := ldap.NewEntry("CN=Ada,OU=Staff,DC=corp,DC=example", map[string][]string{
|
||||||
|
"mail": {"ada@corp.example"}, "displayName": {"Ada"}, "userAccountControl": {"512"},
|
||||||
|
})
|
||||||
|
e.Attributes = append(e.Attributes, &ldap.EntryAttribute{Name: "objectGUID", ByteValues: [][]byte{guidBytes(byte(i))}})
|
||||||
|
out = append(out, e)
|
||||||
|
}
|
||||||
|
return &ldap.SearchResult{Entries: out}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeConn) SearchWithPaging(r *ldap.SearchRequest, _ uint32) (*ldap.SearchResult, error) {
|
||||||
|
f.filters = append(f.filters, r.Filter)
|
||||||
|
return &ldap.SearchResult{Entries: []*ldap.Entry{
|
||||||
|
ldap.NewEntry("CN=ml-team,OU=Groups,DC=corp,DC=example", map[string][]string{"cn": {"ml-team"}}),
|
||||||
|
}}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *fakeConn) Close() error { return nil }
|
||||||
|
|
||||||
|
func guidBytes(last byte) []byte {
|
||||||
|
return []byte{0x78, 0x56, 0x34, 0x12, 0x34, 0x12, 0x78, 0x56, 0x9a, 0xbc, 0xde, 0xf0, 0x12, 0x34, 0x56, last}
|
||||||
|
}
|
||||||
|
|
||||||
|
func fakeLDAP(t *testing.T, c *fakeConn) *LDAP {
|
||||||
|
t.Helper()
|
||||||
|
pw := filepath.Join(t.TempDir(), "bind")
|
||||||
|
_ = os.WriteFile(pw, []byte("svc-secret\n"), 0o600)
|
||||||
|
d, err := NewLDAP(config.LDAP{URLs: []string{"ldaps://dc1"}, BindDN: "CN=llmgw,OU=Svc,DC=corp,DC=example",
|
||||||
|
BindPasswordFile: pw, BaseDN: "OU=Staff,DC=corp,DC=example", UserFilter: "(memberOf=CN=llm,DC=corp,DC=example)"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
d.dial = func(string) (ldapConn, error) { return c, nil }
|
||||||
|
return d
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPersonIsCheckedByBindingAsThemAndTheirGroupsReadNested(t *testing.T) {
|
||||||
|
c := &fakeConn{password: "ad-secret", people: 1}
|
||||||
|
d := fakeLDAP(t, c)
|
||||||
|
id, err := d.Authenticate(t.Context(), `CORP\ada*)(x=`, "ad-secret")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if id.Email != "ada@corp.example" || id.ID != "12345678-1234-5678-9abc-def012345600" ||
|
||||||
|
!slicesHas(id.Groups, "ml-team") || !slicesHas(id.Groups, "CN=ml-team,OU=Groups,DC=corp,DC=example") {
|
||||||
|
t.Errorf("identity %+v", id)
|
||||||
|
}
|
||||||
|
// The name is escaped into the filter, the domain part dropped; the
|
||||||
|
// user filter is ANDed in; disabled accounts are not found; groups are
|
||||||
|
// read with the in-chain rule.
|
||||||
|
find, groups := c.filters[0], c.filters[1]
|
||||||
|
if !strings.Contains(find, `(sAMAccountName=ada\2a\29\28x=)`) || !strings.Contains(find, "(memberOf=CN=llm,DC=corp,DC=example)") ||
|
||||||
|
!strings.Contains(find, "1.2.840.113556.1.4.803:=2") || !strings.Contains(groups, inChain) {
|
||||||
|
t.Errorf("filters %q", c.filters)
|
||||||
|
}
|
||||||
|
// Service account, then the person, then the service account again.
|
||||||
|
if len(c.binds) != 3 || c.binds[1] != "CN=Ada,OU=Staff,DC=corp,DC=example" {
|
||||||
|
t.Errorf("binds %v", c.binds)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWrongPasswordsEmptyOnesAndAmbiguousNamesAreRefusedAlike(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
people int
|
||||||
|
password string
|
||||||
|
}{{1, "wrong"}, {1, ""}, {0, "ad-secret"}, {2, "ad-secret"}} {
|
||||||
|
d := fakeLDAP(t, &fakeConn{password: "ad-secret", people: tc.people})
|
||||||
|
if _, err := d.Authenticate(t.Context(), "ada", tc.password); !errors.Is(err, ErrBadCredentials) {
|
||||||
|
t.Errorf("%d found, password %q: %v", tc.people, tc.password, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPlaintextLDAPIsRefusedWithoutStartTLS(t *testing.T) {
|
||||||
|
base := config.LDAP{BindDN: "CN=x", BindPasswordFile: "/f", BaseDN: "DC=corp"}
|
||||||
|
for _, c := range []struct {
|
||||||
|
urls []string
|
||||||
|
start bool
|
||||||
|
ok bool
|
||||||
|
}{{[]string{"ldaps://dc1"}, false, true}, {[]string{"ldap://dc1"}, false, false},
|
||||||
|
{[]string{"ldap://dc1"}, true, true}, {[]string{"http://dc1"}, true, false}} {
|
||||||
|
l := base
|
||||||
|
l.URLs, l.StartTLS = c.urls, c.start
|
||||||
|
if err := l.Validate(); (err == nil) != c.ok {
|
||||||
|
t.Errorf("%v start_tls=%v: %v", c.urls, c.start, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func slicesHas(xs []string, x string) bool {
|
||||||
|
for _, v := range xs {
|
||||||
|
if v == x {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -33,6 +33,7 @@ var Reference = []Block{
|
|||||||
{Key: "default_tpm", Type: "int", Doc: "", Nested: ""},
|
{Key: "default_tpm", Type: "int", Doc: "", Nested: ""},
|
||||||
{Key: "signup_open", Type: "bool", Doc: "SignupOpen serves the signup form. Accounts it creates are pending until\nan operator approves them, so this is not a way in — but turning it off\nmeans accounts are made with `llmgw users create` instead.", Nested: ""},
|
{Key: "signup_open", Type: "bool", Doc: "SignupOpen serves the signup form. Accounts it creates are pending until\nan operator approves them, so this is not a way in — but turning it off\nmeans accounts are made with `llmgw users create` instead.", Nested: ""},
|
||||||
{Key: "oidc", Type: "OIDC", Doc: "OIDC is sign-in through an identity provider.", Nested: "config.OIDC"},
|
{Key: "oidc", Type: "OIDC", Doc: "OIDC is sign-in through an identity provider.", Nested: "config.OIDC"},
|
||||||
|
{Key: "ldap", Type: "LDAP", Doc: "LDAP is sign-in against a directory, Active Directory above all, and\nthe accounts kept in step with it.", Nested: "config.LDAP"},
|
||||||
{Key: "scim", Type: "SCIM", Doc: "SCIM is account provisioning by the identity provider.", Nested: "config.SCIM"},
|
{Key: "scim", Type: "SCIM", Doc: "SCIM is account provisioning by the identity provider.", Nested: "config.SCIM"},
|
||||||
{Key: "max_key_ttl", Type: "time.Duration", Doc: "MaxKeyTTL is how long any API key may live; a key asked for with no\nexpiry is given this. Organizations, teams and projects can set\nshorter ones (llmgw keys ttl). Zero is no limit.", Nested: ""},
|
{Key: "max_key_ttl", Type: "time.Duration", Doc: "MaxKeyTTL is how long any API key may live; a key asked for with no\nexpiry is given this. Organizations, teams and projects can set\nshorter ones (llmgw keys ttl). Zero is no limit.", Nested: ""},
|
||||||
{Key: "personal", Type: "Personal", Doc: "Personal is what a person may do on their own, outside any project: make\npersonal keys and use the chat page personally, within these limits.", Nested: "config.Personal"},
|
{Key: "personal", Type: "Personal", Doc: "Personal is what a person may do on their own, outside any project: make\npersonal keys and use the chat page personally, within these limits.", Nested: "config.Personal"},
|
||||||
@@ -125,11 +126,16 @@ var Reference = []Block{
|
|||||||
{Key: "url", Type: "string", Doc: "URL is the provider's base address, without /v1: https://api.openai.com.", Nested: ""},
|
{Key: "url", Type: "string", Doc: "URL is the provider's base address, without /v1: https://api.openai.com.", Nested: ""},
|
||||||
{Key: "model", Type: "string", Doc: "Model is what the provider calls it; requests are rewritten to it.", Nested: ""},
|
{Key: "model", Type: "string", Doc: "Model is what the provider calls it; requests are rewritten to it.", Nested: ""},
|
||||||
{Key: "api_key_file", Type: "string", Doc: "APIKeyFile holds the provider's key, on the control plane. A file\nrather than the key itself, so the key is in neither the catalog, the\nstore nor the audit log; empty for a provider that asks for none.", Nested: ""},
|
{Key: "api_key_file", Type: "string", Doc: "APIKeyFile holds the provider's key, on the control plane. A file\nrather than the key itself, so the key is in neither the catalog, the\nstore nor the audit log; empty for a provider that asks for none.", Nested: ""},
|
||||||
|
{Key: "api_key_files", Type: "[]ExternalKey", Doc: "APIKeyFiles are several keys for the provider, in place of\nAPIKeyFile: one endpoint each (external:MODEL#1, #2…), so a key that\nanswers 429 or 401 is passed over while its breaker is open and the\nrequest goes to another. A provider's limits are per key.", Nested: ""},
|
||||||
{Key: "peer", Type: "string", Doc: "Peer names the federated llmgw cluster this model is offered by, for\na model that federation added (`llmgw ctl federation add`) rather\nthan one written by hand. Such a model is named PEER/MODEL, comes and\ngoes with the peer's own list, and is sent only while the peer is up.", Nested: ""},
|
{Key: "peer", Type: "string", Doc: "Peer names the federated llmgw cluster this model is offered by, for\na model that federation added (`llmgw ctl federation add`) rather\nthan one written by hand. Such a model is named PEER/MODEL, comes and\ngoes with the peer's own list, and is sent only while the peer is up.", Nested: ""},
|
||||||
{Key: "api", Type: "string", Doc: "API is what the provider speaks: OpenAI's, unless this says comfyui —\na ComfyUI server, asked for pictures at /v1/images/generations and\nsent the workflow below with the request's prompt in it. Model is then\nthe checkpoint file the workflow loads, as ComfyUI lists it.", Nested: ""},
|
{Key: "api", Type: "string", Doc: "API is what the provider speaks: OpenAI's, unless this says comfyui —\na ComfyUI server, asked for pictures at /v1/images/generations and\nsent the workflow below with the request's prompt in it. Model is then\nthe checkpoint file the workflow loads, as ComfyUI lists it.", Nested: ""},
|
||||||
{Key: "workflow", Type: "string", Doc: "Workflow is the ComfyUI workflow, in its API format: a built-in's name\n(flux-schnell, the default, or flux-dev) or a JSON file on the control\nplane.", Nested: ""},
|
{Key: "workflow", Type: "string", Doc: "Workflow is the ComfyUI workflow, in its API format: a built-in's name\n(flux-schnell, the default, or flux-dev) or a JSON file on the control\nplane.", Nested: ""},
|
||||||
{Key: "output", Type: "string", Doc: "Output is what it makes: pictures, or meshes (see ComfyUI.Output).", Nested: ""},
|
{Key: "output", Type: "string", Doc: "Output is what it makes: pictures, or meshes (see ComfyUI.Output).", Nested: ""},
|
||||||
}},
|
}},
|
||||||
|
{Package: "catalog", Name: "ExternalKey", Title: "ExternalKey", Doc: "", Fields: []Field{
|
||||||
|
{Key: "file", Type: "string", Doc: "File holds the key, on the control plane, as APIKeyFile does.", Nested: ""},
|
||||||
|
{Key: "weight", Type: "float64", Doc: "Weight is its share of the requests against the others'; zero is 1.", Nested: ""},
|
||||||
|
}},
|
||||||
{Package: "config", Name: "Facility", Title: "Facility", Doc: "", Fields: []Field{
|
{Package: "config", Name: "Facility", Title: "Facility", Doc: "", Fields: []Field{
|
||||||
{Key: "budgets", Type: "[]FacilityBudget", Doc: "Budgets are the domains with a power cap.", Nested: ""},
|
{Key: "budgets", Type: "[]FacilityBudget", Doc: "Budgets are the domains with a power cap.", Nested: ""},
|
||||||
{Key: "margin", Type: "float64", Doc: "Margin is the share of every cap kept free, 0.1 for ten percent. Zero\nkeeps none, for caps already given derated.", Nested: ""},
|
{Key: "margin", Type: "float64", Doc: "Margin is the share of every cap kept free, 0.1 for ten percent. Zero\nkeeps none, for caps already given derated.", Nested: ""},
|
||||||
@@ -183,6 +189,22 @@ var Reference = []Block{
|
|||||||
{Key: "min_blocks", Type: "int", Doc: "MinBlocks is the fewest leading blocks a replica must hold to be\npreferred for them; fewer is coincidence — a chat template's opening\ntokens — rather than a prefix worth following. Zero takes 4, which is\n64 tokens at vLLM's usual block size of 16.", Nested: ""},
|
{Key: "min_blocks", Type: "int", Doc: "MinBlocks is the fewest leading blocks a replica must hold to be\npreferred for them; fewer is coincidence — a chat template's opening\ntokens — rather than a prefix worth following. Zero takes 4, which is\n64 tokens at vLLM's usual block size of 16.", Nested: ""},
|
||||||
{Key: "max_blocks", Type: "int", Doc: "MaxBlocks bounds how many of a prompt's blocks are hashed and looked\nup. Beyond it a longer match still helps, but the difference no longer\ndecides anything. Zero takes 4096.", Nested: ""},
|
{Key: "max_blocks", Type: "int", Doc: "MaxBlocks bounds how many of a prompt's blocks are hashed and looked\nup. Beyond it a longer match still helps, but the difference no longer\ndecides anything. Zero takes 4096.", Nested: ""},
|
||||||
}},
|
}},
|
||||||
|
{Package: "config", Name: "LDAP", Title: "LDAP", Doc: "", Fields: []Field{
|
||||||
|
{Key: "urls", Type: "[]string", Doc: "URLs are the directory's servers, tried in turn: ldaps:// (636), or\nldap:// (389) with StartTLS.", Nested: ""},
|
||||||
|
{Key: "start_tls", Type: "bool", Doc: "StartTLS upgrades an ldap:// connection before anything is sent. A\npassword never crosses a connection without TLS: ldap:// without it\nis refused.", Nested: ""},
|
||||||
|
{Key: "ca_file", Type: "string", Doc: "CAFile is the authority the servers' certificates are checked against,\nwhen it is not the system's: an AD Certificate Services root.", Nested: ""},
|
||||||
|
{Key: "bind_dn", Type: "string", Doc: "BindDN and the file holding its password: a service account that may\nread users and groups. Nothing is written to the directory.", Nested: ""},
|
||||||
|
{Key: "bind_password_file", Type: "string", Doc: "", Nested: ""},
|
||||||
|
{Key: "base_dn", Type: "string", Doc: "BaseDN is where people are searched for, subtree included.", Nested: ""},
|
||||||
|
{Key: "group_base_dn", Type: "string", Doc: "GroupBaseDN is where their groups are; empty is BaseDN's domain root.", Nested: ""},
|
||||||
|
{Key: "user_filter", Type: "string", Doc: "UserFilter narrows who may sign in, ANDed with the login match: a\ngroup (memberOf=...) or an OU's people only. Empty is every enabled\nuser object.", Nested: ""},
|
||||||
|
{Key: "label", Type: "string", Doc: "Label is what the form says to sign in with: \"Sign in with your\n<label> account\". Empty is \"Active Directory\".", Nested: ""},
|
||||||
|
{Key: "create_accounts", Type: "bool", Doc: "CreateAccounts makes an account, active, for a person the directory\nvouches for and the gateway does not know. Off, only existing accounts\nsign in this way.", Nested: ""},
|
||||||
|
{Key: "roles", Type: "map[string][]string", Doc: "Roles maps a staff role (viewer, operator, billing, admin) to the\ngroups that give it, by name (CN) or whole DN. Set at every sign-in\nand sync: someone taken out of the group loses the role.", Nested: ""},
|
||||||
|
{Key: "memberships", Type: "map[string]string", Doc: "Memberships maps a group, by CN or DN, to a place in the tree —\norg:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT — joined at sign-in\nand kept by the sync, which also takes away a place whose group the\nperson has left.", Nested: ""},
|
||||||
|
{Key: "sync_interval", Type: "time.Duration", Doc: "SyncInterval is how often the leader reads the directory. Zero is\n15m; negative turns the sync off, leaving only sign-in.", Nested: ""},
|
||||||
|
{Key: "timeout", Type: "time.Duration", Doc: "Timeout bounds each round trip to a server. Zero is 10s.", Nested: ""},
|
||||||
|
}},
|
||||||
{Package: "config", Name: "Landing", Title: "Landing", Doc: "", Fields: []Field{
|
{Package: "config", Name: "Landing", Title: "Landing", Doc: "", Fields: []Field{
|
||||||
{Key: "contact", Type: "string", Doc: "Contact is where someone without a key should go — an address, a URL, a\nchannel name. Rendered as a link when it is one, as text otherwise.", Nested: ""},
|
{Key: "contact", Type: "string", Doc: "Contact is where someone without a key should go — an address, a URL, a\nchannel name. Rendered as a link when it is one, as text otherwise.", Nested: ""},
|
||||||
{Key: "notice", Type: "string", Doc: "Notice is an optional line of your own, for whatever the reader needs to\nknow that this page cannot guess.", Nested: ""},
|
{Key: "notice", Type: "string", Doc: "Notice is an optional line of your own, for whatever the reader needs to\nknow that this page cannot guess.", Nested: ""},
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ var Sources = []string{
|
|||||||
"internal/guard/policy.go",
|
"internal/guard/policy.go",
|
||||||
"internal/guard/judge.go",
|
"internal/guard/judge.go",
|
||||||
"internal/config/oidc.go",
|
"internal/config/oidc.go",
|
||||||
|
"internal/config/ldap.go",
|
||||||
"internal/config/firewall.go",
|
"internal/config/firewall.go",
|
||||||
"internal/config/serving.go",
|
"internal/config/serving.go",
|
||||||
"internal/config/presets.go",
|
"internal/config/presets.go",
|
||||||
|
|||||||
@@ -43,10 +43,15 @@ const (
|
|||||||
// CanaryRolledBack is a model's canary taken out because it served
|
// CanaryRolledBack is a model's canary taken out because it served
|
||||||
// worse than the model it was a candidate to replace.
|
// worse than the model it was a candidate to replace.
|
||||||
CanaryRolledBack = "canary_rolled_back"
|
CanaryRolledBack = "canary_rolled_back"
|
||||||
|
// ExternalKeyRefused is an outside provider answering 401 to one of a
|
||||||
|
// model's keys: revoked, expired or mistyped. Requests go to its other
|
||||||
|
// keys meanwhile, where it has any.
|
||||||
|
ExternalKeyRefused = "external_key_refused"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Kinds are every event there is.
|
// Kinds are every event there is.
|
||||||
var Kinds = []string{ModelParked, NodeLost, BudgetSpent, UpdateAvailable, JudgeUnavailable, CanaryRolledBack}
|
var Kinds = []string{ModelParked, NodeLost, BudgetSpent, UpdateAvailable, JudgeUnavailable, CanaryRolledBack,
|
||||||
|
ExternalKeyRefused}
|
||||||
|
|
||||||
// Event is one thing that happened.
|
// Event is one thing that happened.
|
||||||
type Event struct {
|
type Event struct {
|
||||||
|
|||||||
@@ -131,7 +131,18 @@ const defaultActivationWait = 45 * time.Second
|
|||||||
// SetNotifier tells the accountant where to say a budget is spent. Nothing
|
// SetNotifier tells the accountant where to say a budget is spent. Nothing
|
||||||
// without a store, where there are no budgets.
|
// without a store, where there are no budgets.
|
||||||
func (r *Router) SetNotifier(n *notify.Notifier) {
|
func (r *Router) SetNotifier(n *notify.Notifier) {
|
||||||
|
r.notifier = n
|
||||||
if r.acct != nil {
|
if r.acct != nil {
|
||||||
r.acct.SetNotifier(n)
|
r.acct.SetNotifier(n)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// keyRefused says that an outside provider answered 401 to one of a model's
|
||||||
|
// keys, by its endpoint, which numbers it: never the key.
|
||||||
|
func (r *Router) keyRefused(model, endpointID string) {
|
||||||
|
r.log.Warn("an outside provider refused a key", "model", model, "endpoint", endpointID)
|
||||||
|
r.notifier.Notify(notify.ExternalKeyRefused, endpointID,
|
||||||
|
"the provider of "+model+" answered 401 to the key of "+endpointID+
|
||||||
|
": revoked, expired or mistyped; replace its file on the control plane",
|
||||||
|
map[string]any{"model": model, "endpoint": endpointID})
|
||||||
|
}
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
package router
|
||||||
|
|
||||||
|
import (
|
||||||
|
"cmp"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"log/slog"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/account"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/directory"
|
||||||
|
storepkg "gitea.mynodes.xyz/ilker/llmgw/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sign-in against a directory (auth.ldap), and the accounts kept in step
|
||||||
|
// with it by the leader.
|
||||||
|
|
||||||
|
// ldapDirectory is auth.ldap as the gateway uses it.
|
||||||
|
type ldapDirectory struct {
|
||||||
|
dir *directory.LDAP
|
||||||
|
keeper *directory.Keeper
|
||||||
|
every time.Duration
|
||||||
|
label string
|
||||||
|
log *slog.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
func newLDAPDirectory(cfg config.LDAP, st *storepkg.DB, log *slog.Logger) (*ldapDirectory, error) {
|
||||||
|
dir, err := directory.NewLDAP(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &ldapDirectory{
|
||||||
|
dir: dir,
|
||||||
|
keeper: &directory.Keeper{St: st, Name: "ldap", CreateAccounts: cfg.CreateAccounts,
|
||||||
|
Mapping: directory.Mapping{Roles: cfg.Roles, Memberships: cfg.Memberships}},
|
||||||
|
every: cfg.Interval(),
|
||||||
|
label: cmp.Or(cfg.Label, "Active Directory"),
|
||||||
|
log: log,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SignIn is account.Directory: the person's name and password checked by
|
||||||
|
// the directory, then their account made or kept in step.
|
||||||
|
func (d *ldapDirectory) SignIn(ctx context.Context, login, password string) (storepkg.User, error) {
|
||||||
|
id, err := d.dir.Authenticate(ctx, login, password)
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, directory.ErrBadCredentials):
|
||||||
|
return storepkg.User{}, account.ErrBadCredentials
|
||||||
|
case err != nil:
|
||||||
|
d.log.Warn("the directory could not be asked about a sign-in", "err", err)
|
||||||
|
return storepkg.User{}, fmt.Errorf("the directory could not be asked: %w", err)
|
||||||
|
}
|
||||||
|
u, err := d.keeper.SignIn(ctx, id)
|
||||||
|
switch {
|
||||||
|
case errors.Is(err, directory.ErrSuspended):
|
||||||
|
return storepkg.User{}, account.ErrSuspended
|
||||||
|
case errors.Is(err, directory.ErrNoAccount), errors.Is(err, directory.ErrNoEmail):
|
||||||
|
// Said to the log, not the form: the form says what a wrong
|
||||||
|
// password says.
|
||||||
|
d.log.Info("the directory vouched for a person who has no account here", "dn", id.DN, "err", err)
|
||||||
|
return storepkg.User{}, account.ErrBadCredentials
|
||||||
|
case errors.Is(err, directory.ErrNotActive):
|
||||||
|
return storepkg.User{}, account.ErrNotApproved
|
||||||
|
case err != nil:
|
||||||
|
return storepkg.User{}, err
|
||||||
|
}
|
||||||
|
return u, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// sync reads the directory every interval, on the leader, and brings the
|
||||||
|
// accounts it keeps in step.
|
||||||
|
func (d *ldapDirectory) sync(ctx context.Context, leader func() bool) {
|
||||||
|
if d.every <= 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
t := time.NewTicker(d.every)
|
||||||
|
defer t.Stop()
|
||||||
|
for {
|
||||||
|
if leader() {
|
||||||
|
d.syncOnce(ctx)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-t.C:
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (d *ldapDirectory) syncOnce(ctx context.Context) {
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, 5*time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
people, err := d.dir.People(ctx, d.keeper.Mapping)
|
||||||
|
if err != nil {
|
||||||
|
d.log.Warn("could not read the directory; accounts left as they are", "err", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
rep, err := d.keeper.Sync(ctx, people)
|
||||||
|
if err != nil {
|
||||||
|
d.log.Error("the directory sync stopped", "err", err, "people", len(people))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, where := range rep.Unmapped {
|
||||||
|
d.log.Warn("auth.ldap.memberships maps a group to a place the tree does not have", "place", where)
|
||||||
|
}
|
||||||
|
if rep.Suspended+rep.Reinstated+rep.Updated > 0 {
|
||||||
|
d.log.Info("accounts kept in step with the directory", "people", len(people), "accounts", rep.Seen,
|
||||||
|
"suspended", rep.Suspended, "reinstated", rep.Reinstated, "roles_changed", rep.Updated)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -6,11 +6,15 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/config"
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/notify"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/notify/notifytest"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/router/breaker"
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -87,3 +91,68 @@ func TestAnExternalModelIsNotAModelWithProfiles(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Two keys for one provider, one of them at its limit, or refused: every
|
||||||
|
// request is answered by the other key, the failing one's breaker opens, and
|
||||||
|
// a refused key is said, by its endpoint and never its value.
|
||||||
|
func TestAKeyAtItsLimitOrRefusedIsPassedOverForAnother(t *testing.T) {
|
||||||
|
for _, status := range []int{http.StatusTooManyRequests, http.StatusUnauthorized} {
|
||||||
|
provider := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Header.Get("Authorization") == "Bearer key-a" {
|
||||||
|
http.Error(w, "no", status)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "application/json")
|
||||||
|
fmt.Fprint(w, `{"model":"m","choices":[{"message":{"content":"ok"}}],"usage":{"prompt_tokens":1,"completion_tokens":1}}`)
|
||||||
|
}))
|
||||||
|
t.Cleanup(provider.Close)
|
||||||
|
cfg := baseConfig()
|
||||||
|
cfg.Models = []catalog.Model{{Name: "cloud", Source: "external",
|
||||||
|
External: &catalog.External{URL: provider.URL, Model: "m"}}}
|
||||||
|
rt := newTestRouter(t, cfg)
|
||||||
|
key := func(id, k string) endpoint.Info {
|
||||||
|
return endpoint.Info{ID: id, URL: provider.URL, Models: []string{"cloud"}, APIKey: k,
|
||||||
|
Engine: endpoint.EngineExternal, Pool: "external", Weight: 1, UpstreamModel: "m"}
|
||||||
|
}
|
||||||
|
rt.SetEndpoints(t.Context(), []endpoint.Info{key("external:cloud#1", "key-a"), key("external:cloud#2", "key-b")})
|
||||||
|
n, events := notifytest.Catch(t)
|
||||||
|
rt.SetNotifier(n)
|
||||||
|
|
||||||
|
// Until the failing key's breaker opens: how many requests the
|
||||||
|
// scheduler sends it before then is its own choice.
|
||||||
|
open := func() bool {
|
||||||
|
for _, s := range rt.reg.Snapshots("cloud") {
|
||||||
|
if s.ID == "external:cloud#1" && s.Breaker != breaker.Closed {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for i := 0; i < 100 && !open(); i++ {
|
||||||
|
if rec := post(t, rt.Handler(), map[string]any{"model": "cloud", "messages": []any{}}, nil); rec.Code != http.StatusOK {
|
||||||
|
t.Fatalf("key-a answering %d: request %d got %d: %s", status, i, rec.Code, rec.Body.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, s := range rt.reg.Snapshots("cloud") {
|
||||||
|
if s.ID == "external:cloud#1" && s.Breaker == breaker.Closed {
|
||||||
|
t.Errorf("%d: the failing key's breaker is still closed", status)
|
||||||
|
}
|
||||||
|
if s.ID == "external:cloud#2" && s.Breaker != breaker.Closed {
|
||||||
|
t.Errorf("%d: the good key's breaker opened", status)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var refused []notify.Event
|
||||||
|
for _, e := range events() {
|
||||||
|
if e.Kind == notify.ExternalKeyRefused {
|
||||||
|
refused = append(refused, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case status == http.StatusTooManyRequests && len(refused) != 0:
|
||||||
|
t.Errorf("a key at its limit was said to be refused: %+v", refused)
|
||||||
|
case status == http.StatusUnauthorized && (len(refused) != 1 || refused[0].Subject != "external:cloud#1" ||
|
||||||
|
strings.Contains(refused[0].Message, "key-a")):
|
||||||
|
t.Errorf("a refused key was said as %+v", refused)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -243,6 +243,9 @@ func (r *Router) Handler() http.Handler {
|
|||||||
return web.ModelFacts{ContextLength: m.ContextLength, MaxOutput: m.MaxOutput, Tools: m.Capabilities.Tools}
|
return web.ModelFacts{ContextLength: m.ContextLength, MaxOutput: m.MaxOutput, Tools: m.Capabilities.Tools}
|
||||||
})
|
})
|
||||||
pages.SetPersonalOff(r.personalOff)
|
pages.SetPersonalOff(r.personalOff)
|
||||||
|
if r.directory != nil {
|
||||||
|
pages.SetDirectory(r.directory.label)
|
||||||
|
}
|
||||||
pages.SetLlamaCppVersion(r.llamaVersion)
|
pages.SetLlamaCppVersion(r.llamaVersion)
|
||||||
pages.SetPersonalDefaults(r.personalDefaults)
|
pages.SetPersonalDefaults(r.personalDefaults)
|
||||||
if r.store != nil {
|
if r.store != nil {
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ import (
|
|||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/control/reconciler"
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/guard"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/guard"
|
||||||
|
"gitea.mynodes.xyz/ilker/llmgw/internal/notify"
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/router/admission"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/router/admission"
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/router/breaker"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/router/breaker"
|
||||||
"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
|
"gitea.mynodes.xyz/ilker/llmgw/internal/router/endpoint"
|
||||||
@@ -116,6 +117,10 @@ type Router struct {
|
|||||||
// federation is which cluster this is, to its peers; nil until said.
|
// federation is which cluster this is, to its peers; nil until said.
|
||||||
federation atomic.Pointer[federationID]
|
federation atomic.Pointer[federationID]
|
||||||
acct *accounting.Accountant
|
acct *accounting.Accountant
|
||||||
|
// notifier says what the operator must act on; nil says nothing.
|
||||||
|
notifier *notify.Notifier
|
||||||
|
// directory is auth.ldap's: sign-in and the accounts kept in step.
|
||||||
|
directory *ldapDirectory
|
||||||
// guard judges requests and answers; nil judges nothing. guardLibrary
|
// guard judges requests and answers; nil judges nothing. guardLibrary
|
||||||
// is the policies it was built from, for the pages that show them.
|
// is the policies it was built from, for the pages that show them.
|
||||||
guard guard.Guard
|
guard guard.Guard
|
||||||
@@ -392,6 +397,14 @@ func New(cfg config.Server, log *slog.Logger) (*Router, error) {
|
|||||||
r.accounts = account.New(st, account.Config{
|
r.accounts = account.New(st, account.Config{
|
||||||
SignupOpen: cfg.Auth.SignupOpen,
|
SignupOpen: cfg.Auth.SignupOpen,
|
||||||
}, log)
|
}, log)
|
||||||
|
if cfg.Auth.LDAP.Enabled() {
|
||||||
|
d, err := newLDAPDirectory(cfg.Auth.LDAP, st, log)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
r.directory = d
|
||||||
|
r.accounts.SetDirectory(d)
|
||||||
|
}
|
||||||
|
|
||||||
r.acct = accounting.New(
|
r.acct = accounting.New(
|
||||||
accounting.NewPricing(prices, accounting.Price{
|
accounting.NewPricing(prices, accounting.Price{
|
||||||
@@ -532,6 +545,9 @@ func (r *Router) Start(ctx context.Context) {
|
|||||||
if r.threads != nil {
|
if r.threads != nil {
|
||||||
go r.pruneThreads(ctx)
|
go r.pruneThreads(ctx)
|
||||||
}
|
}
|
||||||
|
if r.directory != nil {
|
||||||
|
go r.directory.sync(ctx, r.leading)
|
||||||
|
}
|
||||||
if r.store != nil && r.usageRetention > 0 {
|
if r.store != nil && r.usageRetention > 0 {
|
||||||
go r.pruneUsage(ctx, r.usageRetention)
|
go r.pruneUsage(ctx, r.usageRetention)
|
||||||
}
|
}
|
||||||
@@ -1193,6 +1209,12 @@ func (r *Router) dispatchOne(ctx context.Context, w http.ResponseWriter, req *pa
|
|||||||
// restart. Not relayed; loaded again and tried again.
|
// restart. Not relayed; loaded again and tried again.
|
||||||
preq.RetryStatus = http.StatusNotFound
|
preq.RetryStatus = http.StatusNotFound
|
||||||
}
|
}
|
||||||
|
// One of an outside provider's several keys refused: the request
|
||||||
|
// is the provider's to answer with another of them, and the key's
|
||||||
|
// breaker counts it. With one key the 401 is the client's to see.
|
||||||
|
if dec.Endpoint.Engine == endpoint.EngineExternal && strings.Contains(dec.Endpoint.ID, "#") {
|
||||||
|
preq.RetryStatus = http.StatusUnauthorized
|
||||||
|
}
|
||||||
if dec.Endpoint.Peer != "" {
|
if dec.Endpoint.Peer != "" {
|
||||||
preq.Extra = r.peerHeaders(ctx, dec.Endpoint, req.Header)
|
preq.Extra = r.peerHeaders(ctx, dec.Endpoint, req.Header)
|
||||||
}
|
}
|
||||||
@@ -1209,6 +1231,9 @@ func (r *Router) dispatchOne(ctx context.Context, w http.ResponseWriter, req *pa
|
|||||||
} else {
|
} else {
|
||||||
out, perr = r.px.Do(uctx, w, preq)
|
out, perr = r.px.Do(uctx, w, preq)
|
||||||
}
|
}
|
||||||
|
if dec.Endpoint.Engine == endpoint.EngineExternal && out.Status == http.StatusUnauthorized {
|
||||||
|
r.keyRefused(model, dec.Endpoint.ID)
|
||||||
|
}
|
||||||
releaseAdapter()
|
releaseAdapter()
|
||||||
uspan.SetAttributes(attribute.Int("http.response.status_code", out.Status))
|
uspan.SetAttributes(attribute.Int("http.response.status_code", out.Status))
|
||||||
endSpan(uspan, perr)
|
endSpan(uspan, perr)
|
||||||
|
|||||||
@@ -27,6 +27,8 @@ var copyOrder = []string{
|
|||||||
"memberships",
|
"memberships",
|
||||||
// After users, which it points at.
|
// After users, which it points at.
|
||||||
"user_excluded_licences",
|
"user_excluded_licences",
|
||||||
|
// After users, which it points at.
|
||||||
|
"user_directory",
|
||||||
"preset_state",
|
"preset_state",
|
||||||
"api_keys",
|
"api_keys",
|
||||||
"key_claims",
|
"key_claims",
|
||||||
|
|||||||
@@ -0,0 +1,92 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// DirectoryLink is an account kept by an outside directory: which one, the
|
||||||
|
// account's id there, and whether the directory's sync is what suspended it.
|
||||||
|
type DirectoryLink struct {
|
||||||
|
UserID string
|
||||||
|
Directory string // "ldap"
|
||||||
|
ExternalID string // an AD objectGUID
|
||||||
|
// SuspendedBySync is an account suspended because the directory
|
||||||
|
// disabled or lost it, and so reinstated when the directory has it
|
||||||
|
// enabled again. One an administrator suspended is not.
|
||||||
|
SuspendedBySync bool
|
||||||
|
SyncedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// DirectoryUser is the account a directory knows by that id.
|
||||||
|
func (s *DB) DirectoryUser(ctx context.Context, directory, externalID string) (DirectoryLink, error) {
|
||||||
|
var l DirectoryLink
|
||||||
|
var suspended int
|
||||||
|
var synced int64
|
||||||
|
err := s.db.QueryRowContext(ctx,
|
||||||
|
`SELECT user_id, directory, external_id, suspended_by_sync, synced_at FROM user_directory
|
||||||
|
WHERE directory = ? AND external_id = ?`, directory, externalID).
|
||||||
|
Scan(&l.UserID, &l.Directory, &l.ExternalID, &suspended, &synced)
|
||||||
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
|
return DirectoryLink{}, ErrNotFound
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return DirectoryLink{}, err
|
||||||
|
}
|
||||||
|
l.SuspendedBySync, l.SyncedAt = suspended != 0, directoryTime(synced)
|
||||||
|
return l, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LinkDirectory records that a directory keeps an account, by its id there.
|
||||||
|
func (s *DB) LinkDirectory(ctx context.Context, userID, directory, externalID string, at time.Time) error {
|
||||||
|
_, err := s.db.ExecContext(ctx,
|
||||||
|
`INSERT INTO user_directory (user_id, directory, external_id, synced_at) VALUES (?, ?, ?, ?)
|
||||||
|
ON CONFLICT (user_id) DO UPDATE SET directory = excluded.directory, external_id = excluded.external_id,
|
||||||
|
synced_at = excluded.synced_at`,
|
||||||
|
userID, directory, externalID, at.UnixMilli())
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// DirectoryLinks are every account a directory keeps.
|
||||||
|
func (s *DB) DirectoryLinks(ctx context.Context, directory string) ([]DirectoryLink, error) {
|
||||||
|
rows, err := s.db.QueryContext(ctx,
|
||||||
|
`SELECT user_id, directory, external_id, suspended_by_sync, synced_at FROM user_directory WHERE directory = ?`,
|
||||||
|
directory)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer func() { _ = rows.Close() }()
|
||||||
|
var out []DirectoryLink
|
||||||
|
for rows.Next() {
|
||||||
|
var l DirectoryLink
|
||||||
|
var suspended int
|
||||||
|
var synced int64
|
||||||
|
if err := rows.Scan(&l.UserID, &l.Directory, &l.ExternalID, &suspended, &synced); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
l.SuspendedBySync, l.SyncedAt = suspended != 0, directoryTime(synced)
|
||||||
|
out = append(out, l)
|
||||||
|
}
|
||||||
|
return out, rows.Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// SetSyncSuspended says whether the directory's sync is what has an account
|
||||||
|
// suspended, and when it last looked.
|
||||||
|
func (s *DB) SetSyncSuspended(ctx context.Context, userID string, suspended bool, at time.Time) error {
|
||||||
|
v := 0
|
||||||
|
if suspended {
|
||||||
|
v = 1
|
||||||
|
}
|
||||||
|
_, err := s.db.ExecContext(ctx,
|
||||||
|
`UPDATE user_directory SET suspended_by_sync = ?, synced_at = ? WHERE user_id = ?`, v, at.UnixMilli(), userID)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func directoryTime(ms int64) time.Time {
|
||||||
|
if ms == 0 {
|
||||||
|
return time.Time{}
|
||||||
|
}
|
||||||
|
return time.UnixMilli(ms)
|
||||||
|
}
|
||||||
@@ -0,0 +1,14 @@
|
|||||||
|
-- Ported from the SQLite migration of the same number.
|
||||||
|
-- Which directory keeps an account, and its id there: an Active Directory
|
||||||
|
-- objectGUID, so a person whose mail changes there is still the same account
|
||||||
|
-- here. suspended_by_sync is an account the directory sync suspended,
|
||||||
|
-- because it was disabled or gone there; one an administrator suspended
|
||||||
|
-- here is not, and is never reinstated by the sync.
|
||||||
|
CREATE TABLE user_directory (
|
||||||
|
user_id TEXT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
directory TEXT NOT NULL,
|
||||||
|
external_id TEXT NOT NULL,
|
||||||
|
suspended_by_sync INTEGER NOT NULL DEFAULT 0,
|
||||||
|
synced_at BIGINT NOT NULL DEFAULT 0,
|
||||||
|
UNIQUE (directory, external_id)
|
||||||
|
);
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
-- Which directory keeps an account, and its id there: an Active Directory
|
||||||
|
-- objectGUID, so a person whose mail changes there is still the same account
|
||||||
|
-- here. suspended_by_sync is an account the directory sync suspended,
|
||||||
|
-- because it was disabled or gone there; one an administrator suspended
|
||||||
|
-- here is not, and is never reinstated by the sync.
|
||||||
|
CREATE TABLE user_directory (
|
||||||
|
user_id TEXT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE,
|
||||||
|
directory TEXT NOT NULL,
|
||||||
|
external_id TEXT NOT NULL,
|
||||||
|
suspended_by_sync INTEGER NOT NULL DEFAULT 0,
|
||||||
|
synced_at INTEGER NOT NULL DEFAULT 0,
|
||||||
|
UNIQUE (directory, external_id)
|
||||||
|
);
|
||||||
@@ -179,10 +179,17 @@ func (s *DB) ApproveUser(ctx context.Context, id, projectID, by string, at time.
|
|||||||
// signup suspended while pending and then reinstated would become active
|
// signup suspended while pending and then reinstated would become active
|
||||||
// without anybody approving it.
|
// without anybody approving it.
|
||||||
func (s *DB) SetUserState(ctx context.Context, id string, state UserState) error {
|
func (s *DB) SetUserState(ctx context.Context, id string, state UserState) error {
|
||||||
|
// An account nobody approved is approved, not set active: that is what
|
||||||
|
// records who let it in. One an identity provider, SCIM or a directory
|
||||||
|
// made — active from the start, its password kept there ("!oidc",
|
||||||
|
// "!scim", "!ldap"), with no approval to record — is set active again
|
||||||
|
// when its provider reinstates it; refusing that left such accounts
|
||||||
|
// suspended for good.
|
||||||
if state == UserActive {
|
if state == UserActive {
|
||||||
var approved sql.NullInt64
|
var approved sql.NullInt64
|
||||||
if err := s.db.QueryRowContext(ctx, `SELECT approved_at FROM users WHERE id = ?`, id).
|
var hash string
|
||||||
Scan(&approved); err == nil && !approved.Valid {
|
if err := s.db.QueryRowContext(ctx, `SELECT approved_at, password_hash FROM users WHERE id = ?`, id).
|
||||||
|
Scan(&approved, &hash); err == nil && !approved.Valid && !strings.HasPrefix(hash, "!") {
|
||||||
return errors.New("this account was never approved; approve it instead")
|
return errors.New("this account was never approved; approve it instead")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+24
-7
@@ -86,6 +86,9 @@ type AccountPages struct {
|
|||||||
hasQuotas bool
|
hasQuotas bool
|
||||||
// oidc is sign-in through an identity provider; nil when there is none.
|
// oidc is sign-in through an identity provider; nil when there is none.
|
||||||
oidc *oidcLogin
|
oidc *oidcLogin
|
||||||
|
// directory is the label of the directory that checks passwords
|
||||||
|
// (auth.ldap); empty when there is none.
|
||||||
|
directory string
|
||||||
// audit records every change made through these pages; nil records none.
|
// audit records every change made through these pages; nil records none.
|
||||||
audit audit.Sink
|
audit audit.Sink
|
||||||
// hasGuard is set when /admin/guard is registered, and guardLibrary
|
// hasGuard is set when /admin/guard is registered, and guardLibrary
|
||||||
@@ -250,6 +253,10 @@ func (p *AccountPages) signup(w http.ResponseWriter, r *http.Request) {
|
|||||||
// --- sign in and out ---
|
// --- sign in and out ---
|
||||||
|
|
||||||
// ssoLabel is what the sign-in button names, or "" without one.
|
// ssoLabel is what the sign-in button names, or "" without one.
|
||||||
|
// SetDirectory says passwords are also checked by a directory, under this
|
||||||
|
// label, so the sign-in form takes a username as well as a mail.
|
||||||
|
func (p *AccountPages) SetDirectory(label string) { p.directory = label }
|
||||||
|
|
||||||
func (p *AccountPages) ssoLabel(l *Lang) string {
|
func (p *AccountPages) ssoLabel(l *Lang) string {
|
||||||
if p.oidc == nil {
|
if p.oidc == nil {
|
||||||
return ""
|
return ""
|
||||||
@@ -265,13 +272,22 @@ type loginData struct {
|
|||||||
Version string
|
Version string
|
||||||
// SSO is the identity provider's label, when sign-in through one is on.
|
// SSO is the identity provider's label, when sign-in through one is on.
|
||||||
SSO string
|
SSO string
|
||||||
|
// Directory is the directory's label when passwords are checked by one
|
||||||
|
// (auth.ldap): the field then takes a username too, not only a mail.
|
||||||
|
Directory string
|
||||||
// CSRF is the form's token, matched by a cookie.
|
// CSRF is the form's token, matched by a cookie.
|
||||||
CSRF string
|
CSRF string
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// loginPage is what every rendering of the sign-in form carries.
|
||||||
|
func (p *AccountPages) loginPage(w http.ResponseWriter, r *http.Request, l *Lang, email string) loginData {
|
||||||
|
return loginData{SSO: p.ssoLabel(l), Directory: p.directory, Email: email, SignupOpen: p.accounts.SignupOpen(),
|
||||||
|
Forgot: p.resets != nil, Version: p.version, CSRF: setLoginToken(w, r)}
|
||||||
|
}
|
||||||
|
|
||||||
func (p *AccountPages) loginForm(w http.ResponseWriter, r *http.Request) {
|
func (p *AccountPages) loginForm(w http.ResponseWriter, r *http.Request) {
|
||||||
noStore(w)
|
noStore(w)
|
||||||
render(w, r, "login", loginData{SSO: p.ssoLabel(langOf(r)), SignupOpen: p.accounts.SignupOpen(), Forgot: p.resets != nil, Version: p.version, CSRF: setLoginToken(w, r)})
|
render(w, r, "login", p.loginPage(w, r, langOf(r), ""))
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *AccountPages) login(w http.ResponseWriter, r *http.Request) {
|
func (p *AccountPages) login(w http.ResponseWriter, r *http.Request) {
|
||||||
@@ -281,15 +297,16 @@ func (p *AccountPages) login(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !loginTokenOK(r) {
|
if !loginTokenOK(r) {
|
||||||
// Not this site's form: another one posting a visitor into an
|
// Not this site's form: another one posting a visitor into an
|
||||||
// account of its choosing, or a form left open past its hour.
|
// account of its choosing, or a form left open past its hour.
|
||||||
renderStatus(w, r, http.StatusBadRequest, "login", loginData{SSO: p.ssoLabel(l), Email: email, SignupOpen: p.accounts.SignupOpen(), Version: p.version,
|
d := p.loginPage(w, r, l, email)
|
||||||
Error: l.T("login.err.form_expired"), CSRF: setLoginToken(w, r)})
|
d.Error = l.T("login.err.form_expired")
|
||||||
|
renderStatus(w, r, http.StatusBadRequest, "login", d)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
ctx := account.WithClient(r.Context(), clientAddr(r))
|
ctx := account.WithClient(r.Context(), clientAddr(r))
|
||||||
token, csrf, err := p.accounts.SignIn(ctx, email, r.FormValue("password"))
|
token, csrf, err := p.accounts.SignIn(ctx, email, r.FormValue("password"))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
d := loginData{SSO: p.ssoLabel(l), Email: email, SignupOpen: p.accounts.SignupOpen(), Forgot: p.resets != nil, Version: p.version, CSRF: setLoginToken(w, r)}
|
d := p.loginPage(w, r, l, email)
|
||||||
switch {
|
switch {
|
||||||
case errors.Is(err, account.ErrBadCredentials),
|
case errors.Is(err, account.ErrBadCredentials),
|
||||||
errors.Is(err, account.ErrNotApproved),
|
errors.Is(err, account.ErrNotApproved),
|
||||||
@@ -309,9 +326,9 @@ func (p *AccountPages) login(w http.ResponseWriter, r *http.Request) {
|
|||||||
if p.oidc != nil && p.oidc.cfg.NoPasswords {
|
if p.oidc != nil && p.oidc.cfg.NoPasswords {
|
||||||
if _, u, err := p.accounts.Resolve(ctx, token); err != nil || !u.IsAdmin {
|
if _, u, err := p.accounts.Resolve(ctx, token); err != nil || !u.IsAdmin {
|
||||||
_ = p.accounts.SignOut(ctx, token)
|
_ = p.accounts.SignOut(ctx, token)
|
||||||
renderStatus(w, r, http.StatusForbidden, "login", loginData{SSO: p.ssoLabel(l), Email: email,
|
d := p.loginPage(w, r, l, email)
|
||||||
SignupOpen: p.accounts.SignupOpen(), Version: p.version, CSRF: setLoginToken(w, r),
|
d.Error = l.T("login.err.passwords_off", p.ssoLabel(l))
|
||||||
Error: l.T("login.err.passwords_off", p.ssoLabel(l))})
|
renderStatus(w, r, http.StatusForbidden, "login", d)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -19,6 +19,8 @@
|
|||||||
"login.title": "Sign in · llmgw",
|
"login.title": "Sign in · llmgw",
|
||||||
"login.heading": "Sign in",
|
"login.heading": "Sign in",
|
||||||
"login.email": "Email",
|
"login.email": "Email",
|
||||||
|
"login.username": "Username or email",
|
||||||
|
"login.with_directory": "Sign in with your %s account.",
|
||||||
"login.password": "Password",
|
"login.password": "Password",
|
||||||
"login.submit": "Sign in",
|
"login.submit": "Sign in",
|
||||||
"login.with_sso": "Sign in with %s",
|
"login.with_sso": "Sign in with %s",
|
||||||
|
|||||||
@@ -19,6 +19,8 @@
|
|||||||
"login.title": "Giriş yap · llmgw",
|
"login.title": "Giriş yap · llmgw",
|
||||||
"login.heading": "Giriş yapın",
|
"login.heading": "Giriş yapın",
|
||||||
"login.email": "E-posta",
|
"login.email": "E-posta",
|
||||||
|
"login.username": "Kullanıcı adı veya e-posta",
|
||||||
|
"login.with_directory": "%s hesabınızla giriş yapın.",
|
||||||
"login.password": "Parola",
|
"login.password": "Parola",
|
||||||
"login.submit": "Giriş yap",
|
"login.submit": "Giriş yap",
|
||||||
"login.with_sso": "%s ile giriş yap",
|
"login.with_sso": "%s ile giriş yap",
|
||||||
|
|||||||
+33
-5
@@ -148,8 +148,9 @@ func (p *AccountPages) oidcStart(w http.ResponseWriter, r *http.Request) {
|
|||||||
pr, err := o.providerFor(r.Context())
|
pr, err := o.providerFor(r.Context())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
p.log.Warn("the identity provider did not answer", "issuer", o.cfg.Issuer, "err", err)
|
p.log.Warn("the identity provider did not answer", "issuer", o.cfg.Issuer, "err", err)
|
||||||
renderStatus(w, r, http.StatusBadGateway, "login", loginData{SignupOpen: p.accounts.SignupOpen(), Version: p.version,
|
d := p.loginPage(w, r, langOf(r), "")
|
||||||
CSRF: setLoginToken(w, r), SSO: o.cfg.Label, Error: langOf(r).T("sso.err.no_answer")})
|
d.Error = langOf(r).T("sso.err.no_answer")
|
||||||
|
renderStatus(w, r, http.StatusBadGateway, "login", d)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
state, nonce, verifier := randomToken(), randomToken(), oauth2.GenerateVerifier()
|
state, nonce, verifier := randomToken(), randomToken(), oauth2.GenerateVerifier()
|
||||||
@@ -172,8 +173,9 @@ func (p *AccountPages) oidcCallback(w http.ResponseWriter, r *http.Request) {
|
|||||||
if errors.As(err, &ae) {
|
if errors.As(err, &ae) {
|
||||||
msg = ae.in(l)
|
msg = ae.in(l)
|
||||||
}
|
}
|
||||||
renderStatus(w, r, http.StatusForbidden, "login", loginData{SignupOpen: p.accounts.SignupOpen(),
|
d := p.loginPage(w, r, l, "")
|
||||||
Version: p.version, CSRF: setLoginToken(w, r), SSO: o.cfg.Label, Error: msg})
|
d.Error = msg
|
||||||
|
renderStatus(w, r, http.StatusForbidden, "login", d)
|
||||||
}
|
}
|
||||||
c, err := r.Cookie(oidcCookie)
|
c, err := r.Cookie(oidcCookie)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -297,7 +299,17 @@ func (p *AccountPages) oidcUser(ctx context.Context, claims map[string]any) (sto
|
|||||||
return store.User{}, accountError{"sso.err.state", []any{email, string(u.State)}}
|
return store.User{}, accountError{"sso.err.state", []any{email, string(u.State)}}
|
||||||
}
|
}
|
||||||
|
|
||||||
groups := claimStrings(claims[or(o.cfg.GroupsClaim, "groups")])
|
claim := or(o.cfg.GroupsClaim, "groups")
|
||||||
|
if groupsOverflowed(claims, claim) {
|
||||||
|
// Read as no groups, this would take an administrator's role away
|
||||||
|
// at their next sign-in: kept as they are instead, and said.
|
||||||
|
p.log.Error("the identity provider left this person's groups out of the token, as Entra ID does past 200 groups; "+
|
||||||
|
"their role and memberships are left as they were. In the app registration's token configuration, "+
|
||||||
|
"emit only the groups assigned to the application, or provision through SCIM",
|
||||||
|
"email", email, "claim", claim)
|
||||||
|
return u, nil
|
||||||
|
}
|
||||||
|
groups := claimStrings(claims[claim])
|
||||||
if len(o.cfg.Roles) > 0 {
|
if len(o.cfg.Roles) > 0 {
|
||||||
role := ""
|
role := ""
|
||||||
for _, r := range []string{store.RoleAdmin, store.RoleOperator, store.RoleBilling, store.RoleViewer} {
|
for _, r := range []string{store.RoleAdmin, store.RoleOperator, store.RoleBilling, store.RoleViewer} {
|
||||||
@@ -333,6 +345,22 @@ func (p *AccountPages) oidcUser(ctx context.Context, claims map[string]any) (sto
|
|||||||
return u, nil
|
return u, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// groupsOverflowed is a token whose groups claim was too long to send:
|
||||||
|
// Entra ID then names a source to fetch it from (_claim_names), or, in a
|
||||||
|
// token from the implicit flow, says only hasgroups.
|
||||||
|
func groupsOverflowed(claims map[string]any, claim string) bool {
|
||||||
|
if _, ok := claims[claim]; ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if names, ok := claims["_claim_names"].(map[string]any); ok {
|
||||||
|
if _, ok := names[claim]; ok {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
has, _ := claims["hasgroups"].(bool)
|
||||||
|
return has
|
||||||
|
}
|
||||||
|
|
||||||
// oidcPlace finds org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT.
|
// oidcPlace finds org:NAME, team:ORG/TEAM, project:ORG/TEAM/PROJECT.
|
||||||
func (p *AccountPages) oidcPlace(ctx context.Context, where string) (store.Scope, string, error) {
|
func (p *AccountPages) oidcPlace(ctx context.Context, where string) (store.Scope, string, error) {
|
||||||
kind, path, _ := strings.Cut(where, ":")
|
kind, path, _ := strings.Cut(where, ":")
|
||||||
|
|||||||
@@ -165,6 +165,15 @@ func TestSigningInThroughAProvider(t *testing.T) {
|
|||||||
t.Error("no session cookie")
|
t.Error("no session cookie")
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Past 200 groups Entra ID sends none, only where to fetch them: the
|
||||||
|
// role stays rather than being taken away as if she had no groups.
|
||||||
|
idp.claims = map[string]any{"email": "ann@example.com", "email_verified": true,
|
||||||
|
"_claim_names": map[string]any{"groups": "src1"},
|
||||||
|
"_claim_sources": map[string]any{"src1": map[string]any{"endpoint": "https://graph.windows.net/t/users/u/getMemberObjects"}}}
|
||||||
|
if rec := signIn(t, mux, idp, false); rec.Code != http.StatusSeeOther || st.roles[u.ID] != "operator" {
|
||||||
|
t.Errorf("an overflowed groups claim: %d, role %q", rec.Code, st.roles[u.ID])
|
||||||
|
}
|
||||||
|
|
||||||
// Refused: a state that is not this browser's, an unverified address,
|
// Refused: a state that is not this browser's, an unverified address,
|
||||||
// and nobody the gateway knows when it does not make accounts.
|
// and nobody the gateway knows when it does not make accounts.
|
||||||
if rec := signIn(t, mux, idp, true); rec.Code != http.StatusForbidden {
|
if rec := signIn(t, mux, idp, true); rec.Code != http.StatusForbidden {
|
||||||
|
|||||||
@@ -38,7 +38,9 @@
|
|||||||
{{if .Error}}<p class="error">{{.Error}}</p>{{end}}
|
{{if .Error}}<p class="error">{{.Error}}</p>{{end}}
|
||||||
<form method="post" autocomplete="on">
|
<form method="post" autocomplete="on">
|
||||||
<input type="hidden" name="csrf" value="{{.CSRF}}">
|
<input type="hidden" name="csrf" value="{{.CSRF}}">
|
||||||
<label>{{t "login.email"}}<input type="email" name="email" value="{{.Email}}" required autocomplete="email"></label>
|
{{if .Directory}}<p class="sub">{{t "login.with_directory" .Directory}}</p>
|
||||||
|
<label>{{t "login.username"}}<input type="text" name="email" value="{{.Email}}" required autocomplete="username" autocapitalize="none" spellcheck="false"></label>
|
||||||
|
{{else}}<label>{{t "login.email"}}<input type="email" name="email" value="{{.Email}}" required autocomplete="email"></label>{{end}}
|
||||||
<label>{{t "login.password"}}<input type="password" name="password" required autocomplete="current-password"></label>
|
<label>{{t "login.password"}}<input type="password" name="password" required autocomplete="current-password"></label>
|
||||||
<button type="submit">{{t "login.submit"}}</button>
|
<button type="submit">{{t "login.submit"}}</button>
|
||||||
</form>
|
</form>
|
||||||
|
|||||||
Reference in new issue
Block a user