ilker/llmgw
2
0 Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 95 Wiki Activity

95 Releases 95 Tags

RSS Feed
  • v0.16.0 4ee9ba54ef
    Compare

    v0.16.0
    Release / build (push) Successful in 6m6s
    Details
    Release / notes (push) Successful in 5s
    Details
    CI / test (push) Successful in 11m14s
    Details
    Stable

    gitea-actions released this 2026-10-30 11:30:03 +01:00 | 68 commits to main since this release

    Weights pinned to the commit they were added at and checked, by the nodes and the mirror, against the sha256
    Hugging Face published; the documents llmgw-kb sends embedded by the gateway; Strata offered by the plan, ctl model serve, the explorer and the serve forms; and cluster upgrades that tell the nodes together, say how far they
    are, and do not fail a node that came back on the version.

    Upgrading.

    • No migrations. Upgrade the control planes, then the nodes (llmgw ctl upgrade v0.16.0), then each mirror's host
      (sudo llmgw upgrade v0.16.0 --service llmgw-mirror).
    • A node checks what it fetches against the catalog's checksums only once it runs this version; until then it
      fetches as before.
    • A model added from Hugging Face is now pinned when it is added, which needs the registry reachable from the
      control plane; one it cannot be looked up in is refused, saying why. Models already in the catalog are left as
      they are.
    • A Strata profile must say --size as well as --family, and both must be ones Strata runs.
    • llmgw-kb sends text alone to a gateway of this version; an older llmgw-kb keeps working, sending vectors.

    Added

    • Strata offered where models are chosen. For a repository of a family Strata runs, ctl model plan offers "one
      card with Strata, experts in RAM" with the RAM it takes, ctl model serve --engine strata (or the plan's choice)
      builds its profile, the Hugging Face explorer lists it as running here, and the mirror's serve form serves such a
      version on Strata. Strata's families and sizes are known: the catalog refuses one it does not run and reserves the
      RAM a size keeps its experts in.
    • The gateway embeds the documents llmgw-kb sends. A document may be sent as its passages' text alone; the
      gateway embeds them through its own /v1/embeddings as the caller, 32 at a time and four batches at once,
      waiting out a busy engine rather than failing the document. About a kilobyte a passage leaves the machine instead
      of a vector down and up again, some thirty times more. llmgw-kb does so with a gateway that says it can
      (server_embeds in a collection's answer); --embed-here keeps the old way.
    • Weights pinned to a commit, and checked against what Hugging Face published. A model added from Hugging Face
      (hf:, a resolve URL on a branch, or a repository vLLM or SGLang downloads) is pinned to the commit it is at
      then — the file's URL at that commit, or --revision — and each file's published sha256 goes in its origin. A node
      refuses and deletes a copy that is not it, naming both checksums; the mirror does the same for what ctl model serve fetches, and llmgw mirror add for a Hugging Face URL (--sha256 for others). Nodes need this version to
      check; older ones fetch as before.

    Fixed

    • A cluster upgrade that sat silent, or stopped on a node that had upgraded.

      • The nodes of a wave are told together, each saying when it is done, and what is still awaited is said every 30
        seconds; before, each was told in turn, up to five minutes each, with nothing on the screen.
      • The control plane takes the node registering again on the version as done: the agent's answer, sent a moment
        before it exits, could be lost, and a node that had upgraded was failed as "older than v0.10.4".
      • The agent's download no longer ends with its connection (a control plane restarting, leadership moving), and the
        control plane waits longer than the download may take; a second request while one runs is refused.
      • A cordoned or draining node upgraded is done; a node whose idle model stopped meanwhile is done once nothing on it
        is starting; a node not connected is passed over and named at the end rather than stopping every node after it.
      • The console's node upgrade waits as long as the control plane, and offers no development version.
      • systemd never gives up restarting the agent (StartLimitIntervalSec=0), in units written from now on.

    • Strata. One named by hand in agent.yaml was never reported, so never placed on; its engine, which leaves the
      server's process group, is now stopped with it; Pascal and Volta, which its CUDA 12 build runs, were refused.

    • A name with a slash in it, asked through a standby control plane. llmgw ctl kb delete Demo/hr on a standby
      was answered 405: the standby passed the request on to the leader with Demo%2Fhr unescaped, two segments instead
      of one. The path now reaches the leader as it was sent.

    Downloads