ilker/llmgw
2
0 Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 95 Wiki Activity

95 Releases 95 Tags

RSS Feed
  • v0.14.8 2005cea326
    Compare

    v0.14.8
    CI / test (push) Successful in 9m12s
    Details
    Release / build (push) Successful in 4m20s
    Details
    Release / notes (push) Successful in 3s
    Details
    Stable

    gitea-actions released this 2026-10-18 11:18:14 +02:00 | 149 commits to main since this release

    Sign-in with an on-premises Active Directory, and Entra ID made safe for
    people in many groups. Also several keys for one outside provider, and a
    measure of what the gateway adds to a request.

    Upgrading.

    • Migration 0048 adds the user_directory table, applied when the control
      plane starts.
    • Nothing changes until auth.ldap is set; docs/install.md, "Active
      Directory", has an example. It has been tested against a fake directory,
      not yet against a real domain controller.
    • A judge installed by an earlier server init is refitted by the leader
      by itself (below); the steps given for v0.14.7 are no longer needed.

    Added

    • Sign-in with Active Directory (auth.ldap). People sign in on the gateway's form with their AD username or mail
      and password, checked against the domain controllers over LDAPS or StartTLS. Groups, nested ones included, give
      staff roles and places in the tree, as with OIDC. Accounts are tied to the objectGUID and made at first sign-in
      when create_accounts is on. Every 15 minutes the leader reads the directory. People disabled or deleted there are
      suspended and reinstated when they come back; roles and memberships follow their groups. A sync that would suspend
      more than half the accounts is refused. See docs/install.md, "Active Directory".

    • Several keys for one outside provider. An external model takes api_key_files, each with an optional weight, in
      place of api_key_file. Each key is an endpoint of its own (external:MODEL#1, #2), so a key answering 429 is
      passed over while its breaker is open and the request goes to another. A key answering 401 is passed over too, and
      external_key_refused is sent, naming the endpoint, never the key. Usage records which key answered by that name.

    • llmgw ctl bench overhead: what the gateway adds to a request, measured against an engine that answers at
      once. Gateway and engine run in one process on loopback, so no cluster is needed. The load is sent straight to the
      engine and through the gateway: bare, with authentication and accounting, and with the guardrails' pattern checks.
      It reports p50 and p99 latency, what the gateway added, and a rough CPU per thousand requests. On an Apple M5 the
      gateway added 0.05–0.42 ms at p50 and at most 0.49 ms at p99 (README).

    Fixed

    • Entra ID's groups past 200 no longer take a role away. For someone in more than 200 groups Entra sends none in
      the token, only a pointer to Graph. That was read as no groups, so an administrator lost their role at the next
      sign-in. Their role and memberships are now left as they were, and an error is logged saying what to change in the app
      registration. docs/install.md has an Entra ID section.
    • An account made by SCIM or OIDC could not be reinstated. Setting it active was refused as "never approved".
    • An upgrade refits the judge an earlier server init installed. Upgrading to v0.14.7 left such a judge as it was:
      no cores or one, 2 or 4 GB, the prompt cache on. On fewer than eight cores every request was refused until it was
      fixed by hand. Now the leader finds a judge whose entry is exactly what an earlier init wrote and gives it 2 GB, no
      prompt cache and cores by today's rule (up to four for one judge, two each for more). It then restarts the judge's
      replicas onto that. The number of replicas is left as it is. A judge whose entry was changed by hand is left alone,
      and the log says so once.
    • A restart is no longer refused for room another restart is giving back. With every core of a node reserved, two
      models restarted together, one to need more and one less (as after ctl model resources on a judge and a starter),
      had the first refused: "fits on no node even with this replica stopped", though the second was about to give back
      its cores. A restart now waits for other restarts under way, up to the drain grace, before it is refused. A drained
      replica also counts a replica being restarted that still answers as serving, so replicas restarted one after
      another no longer wait out the five-minute grace each.
    Downloads