Commit Graph
15 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 ffab3c99e3 llmgw-kb built for Windows, macOS and Linux in CI and attached to releases
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-08 13:44:53 +02:00
Ilker ManapandClaude Opus 5.5 11b5d44702 A Terraform provider for llmgw: models, quotas and guard attachments through the admin API
terraform-provider-llmgw/ is a module of its own, built on
terraform-plugin-framework, so the gateway does not carry its dependencies.
llmgw_model is a model added at runtime: scaling, class, owner and adapters
change in place, anything else replaces it; the definition is kept as
written rather than as the catalog filled it in. llmgw_quota and
llmgw_guard_attachment keep the scope as written beside the ID it resolved
to, and import by that ID. Data sources llmgw_models and llmgw_nodes.

The client lists every endpoint it calls and refuses any other; a test
checks each, and every field it sends and reads, against
docs/admin-api.openapi.json, so a change to the admin API that breaks the
provider fails here. Acceptance tests run Terraform against an admin API in
memory when TF_ACC is set. CI vets and tests the module.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 00:04:35 +02:00
Ilker ManapandClaude Opus 5.5 0bd5305e9a Install only releases signed with the key built in; fetch a node's release on the control plane
trusted.pub now holds the release signing key, so self-upgrade, agent
upgrade and node onboarding refuse a release whose checksums are not signed
with it. Onboarding fetched the release on the node and compared it with a
checksum from the same server; the control plane now fetches it for the
node's architecture with release.Fetch, checks signature and checksum, and
hands the bytes over. Release addresses must be https (loopback excepted).
LLMGW_RELEASE_KEY overrides the key only in development builds. The CI
release job fails instead of publishing an unsigned release while a key is
built in.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 09:48:03 +02:00
Ilker ManapandClaude Opus 5.5 623cdb5ba3 Sign releases, and install only signed ones once a key is trusted
CI / test (push) Successful in 4m6s
ed25519 over checksums.txt: llmgw release keygen makes the pair, CI signs
with the RELEASE_SIGNING_KEY secret when it is set, and release.Install
refuses an unsigned or wrongly signed release when the build embeds a
trusted key (internal/release/trusted.pub, empty for now) or
LLMGW_RELEASE_KEY names one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-30 23:03:48 +02:00
Ilker ManapandClaude Opus 5.5 069d334652 Ship Caddy and nginx configurations for the client port
install.md said the client port belongs behind a reverse proxy and gave
nothing to put there. deploy/proxy/ has both, attached to every release.

What they get right is what a first attempt gets wrong: streamed replies
are not buffered, the read timeout outlasts a model's silence before its
first token (nginx's 60s default is a 504), the body limit matches the
router's, and X-Forwarded-Proto and -Host are set by the proxy rather
than passed through — llmgw marks session cookies Secure from the first
and builds the landing page's base URL from the second.

The Caddyfile was validated and run against a streaming upstream: events
arrive as sent, and forged forwarded headers are replaced. nginx uses
`listen ... http2` because `http2 on;` needs 1.25.1 and Ubuntu 22.04 and
Debian 12 ship older.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 12:55:46 +02:00
Ilker ManapandClaude Opus 5.5 fa7f90880d Put the changelog on the release page
CI / test (push) Successful in 1m42s
Every release on Gitea was a list of binaries and nothing else. Gitea makes
an empty release for each pushed tag, and the workflow only ever attached
files to it, so what had changed was written in CHANGELOG.md and nowhere a
release was looked at.

A job after the build now copies the tag's section of the changelog onto its
release, and fails if there is none — a version nobody described should say
so where it is seen. Run by hand, it syncs every version the changelog has,
which is how the releases made before this get their notes. The changelog
stays the one place notes are written; this copies it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 10:32:58 +02:00
Ilker ManapandClaude Opus 5 7de8bacfc0 Stop reading JSON with a greedy regex, and stop set -e hiding the empty case
CI / test (push) Successful in 1m54s
Release / build (push) Successful in 2m39s
My first fix was wrong about why the release could not be created. Gitea
materialises a release row for every pushed tag, so the lookup should have
found one; the create that followed hit a release that already existed.

The lookup failed because of how it read the id. `sed 's/.*"id":\([0-9]*\).*/\1/'`
is greedy, so `.*"id":` matches the last occurrence in the document, not the
first — "id" appears again inside the nested author object — and it does not
match `"id": 13` with a space after the colon at all. Checked against this
instance: for v0.5.0, whose id is 12, the old pattern returns 1.

It now greps the first "id", tolerates the whitespace, and re-reads the
release after creating rather than parsing the create response, so a 409
resolves to the release that was already there instead of to an error.

`set -e` was hiding the empty case: a command substitution in an assignment
fails the script, and grep exits non-zero when it matches nothing, so the
"no release yet" branch could never be reached — the job would simply stop.
Both extractions are guarded, and both paths are exercised.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 00:33:33 +02:00
Ilker ManapandClaude Opus 5 4815a50c94 Ask for the scope creating a release actually needs, and say when it is refused
CI / test (push) Has been cancelled
The first v0.6.0 run built every binary, including the NVML one, and then
stopped at "could not find or create the release". Two things were wrong.

`releases: write` is not enough to create one. Creating a release can create
its tag, which is a code-unit operation; `contents` is the scope that covers
code and releases together.

And the step threw away the answer. It piped the response through sed looking
for an id and, finding none, reported only that it had failed — so a refusal,
a malformed request and an unreachable server all produced the same line. It
now prints the status code and the body, which is how the above was diagnosed
at all: by reading a log, not by knowing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 00:32:44 +02:00
Ilker ManapandClaude Opus 5 a2b3f63a0d Say what v0.6.0 is, and let the release actually publish it
CI / test (push) Successful in 1m56s
Release / build (push) Failing after 2m31s
The changelog had two "Unreleased" headings. The second sat below v0.5.0 and
was the detail behind it — phases 2 and 3, written while they were in progress
and never folded in when the release was cut — so it claimed as unreleased
work that had been shipped for two days. It is now a section of v0.5.0. The
status table said v0.5.0 was planned, which its own release notes disprove.

The first "Unreleased" becomes v0.6.0: eighty commits, of which phase 4 is the
headline and the rest is what a cluster needs before anyone but its operator
is let near it.

Two things would have stopped the release workflow publishing anything:

Gitea gives every job a built-in GITEA_TOKEN, but what it may do depends on
the instance's default mode, and under the restricted default it is read-only
on releases. The job now asks for `releases: write`, which is correct under
either default. An upload that is refused now fails the job — without `-f`,
curl treats a 403 body as a successful transfer, so the release would end up
with some of its files, or none, and a green tick either way.

`workflow_dispatch` is offered but the upload assumed a tag. Fired from a
branch it would have created a release named "main".

README's status paragraph still said authentication, budgets and the control
plane were missing. It now says what is actually left, including the part
worth saying out loud: the InfiniBand, NVML and NCCL paths have never run on
a machine that has any of them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 00:29:15 +02:00
Ilker ManapandClaude Opus 5 b619546516 Stop trusting hardware facts that were never checked
CI / test (push) Successful in 1m32s
Three places where the system took the infrastructure's word for something
and would have failed silently on real hardware.

A declared fabric adapter that is not on the node. Fabric membership has to
be configuration — OpenSM's default subnet prefix is identical on every
fabric, so a matching prefix proves nothing — but taking it entirely on
trust was a different mistake. A typo, or a card that enumerated
differently after a reboot, produced NCCL_IB_HCA naming an adapter that
does not exist.

An adapter with no ACTIVE port. Physically linked with no subnet manager
sits in INIT: it looks healthy to anything counting adapters, carries no
data, survives a reboot, and looks like a software problem. Discovery had
recorded port state all along and placement only ever asked whether a
fabric had any adapters listed.

Both end the same way — the replica starts, passes its health checks, and
hangs on its first all-reduce, which is the failure the fabric invariant
exists to prevent arriving through a door nobody was watching. The agent
now compares declared against discovered, where both are visible for the
only time, and does not advertise a fabric that cannot carry traffic. The
reason is logged on the machine that knows it, because the operator who
typed the adapter name is the only person who can fix it. The fabric's rate
is the slowest active link, since a collective runs at the speed of its
slowest.

And NVLink topology, which no published binary could see. NVML is its only
source, the bindings need cgo, and CGO_ENABLED=0 is deliberate so the
portable binary runs on an older libc. The cost was silent: tensor groups
fall back to the lowest free indices, which is right on an SXM board with
NVSwitch and wrong on a PCIe box where bridges pair specific cards — a
group straddling two of them runs several times slower and nothing says so.
Releases now also publish an nvml build for those nodes, and
llmgw_gpu_nvlink_peers makes the difference visible: a flat zero across a
fleet means the topology is unknown rather than absent.

A test that declared mlx5_0 and mlx5_1 on a node whose sysfs held neither
had been passing, because nothing compared the two.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 17:29:15 +02:00
Ilker ManapandClaude Opus 5 5b425c32e5 Ship binaries with releases, and ask engines whether our flags still exist
CI / test (push) Successful in 1m31s
A release was a git tag. Installing llmgw on a node meant having a Go
toolchain, and the onboarding script copied a binary from wherever it was run —
which works and quietly makes whoever runs it the build server, with the fleet
ending up on whatever was in their working tree. Tags now build static
linux/amd64, linux/arm64 and darwin binaries and attach them. CGO_ENABLED=0 is
the point: it has to run on a node whose libc is older than the builder's, and
llmgw has no C dependency because modernc.org/sqlite is pure Go precisely so
this stays true. Verified: "not a dynamic executable", runs on a node, reports
its version.

`ctl engine-check` closes the other half of the phase 4 gap. llmgw's launch
arguments are read from each engine's documentation, and engines rename flags
between versions; when that happens the failure arrives minutes into a model
load, in the engine's own error message, on a node nobody was watching. It
builds the widest command llmgw would ever produce — multi-node, since a flag
that only appears there is the one nobody notices has gone — and checks each
flag against the engine's own --help. No GPU: a CLI prints its help anywhere.
Verified against llama.cpp b10991, all seven flags accepted.

Flags are matched on a word boundary, so --port is not satisfied by
--port-range. Without that, a renamed flag would sail through the check that
exists to catch renamed flags.

It also warns when the help text looks truncated, which is how the first run
went: llama.cpp's help cut to 400 of its 725 lines reported five flags as
missing that the running cluster uses.

TestSidecarRunsBeforeEngine was flaky under a full run, and worse than flaky —
its wait loop fell through on timeout, so a slow machine turned the ordering
assertion into a no-op and the failure surfaced later as something else. It now
asserts, with a deadline that suits a machine running every other package at
the same time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 16:44:29 +02:00
ilkerandClaude Opus 5 55417546fc Phase 2 (2/2): NVML backend behind a build tag
CI / test (push) Successful in 1m37s
NVML is the only source of NVLink topology, and that topology is what keeps a
tensor-parallel group off the slow path inside a node. It needs cgo, so it sits
behind the nvml tag and the default build keeps using nvidia-smi.

Discover() picks the backend and falls back to nvidia-smi when NVML fails,
logging the fallback rather than performing it silently: on a node that is meant
to have working NVML, a quiet downgrade hides a broken driver installation and
costs the scheduler its topology.

CI now compiles the tagged build. That immediately paid for itself: ErrNoNVML
was declared only in the untagged file, so the tests did not compile under
-tags nvml. It is now declared where both builds can see it.

Not verified: the NVML path has never run against a driver, because there is no
GPU machine available. The NVLink domains it reports should be treated as
unverified until someone runs `llmgw agent inventory` on a real node.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 02:05:25 +02:00
ilkerandClaude Opus 5 664c8e9126 ci: add workflow_dispatch and take the Go version from go.mod
CI / test (push) Successful in 1m16s
The runner this targets is a containerized act_runner registered with the
ubuntu-latest label, separate from maas-builder: that one runs in host mode as
root on the image build machine, which is not somewhere a test suite belongs.

workflow_dispatch lets the queued runs be retried from the UI once a runner
exists, without needing an empty commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 01:19:12 +02:00
ilkerandClaude Opus 5 28029eb43b Switch the project language to English
CI / test (push) Has been cancelled
Comments, doc comments, test names, error and log messages, CLI help, README,
Makefile, CI step names and the design document are now all in English.
No behaviour changes; the full suite passes under the race detector.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 01:15:21 +02:00
ilkerandClaude Opus 5 e07bc46c57 Faz 0: iskelet ve sim-engine
CI / test (push) Has been cancelled
Plan v0.2'nin Faz 0 çıktısı. Yazılan sıraya göre:

- cobra tabanlı CLI iskeleti; server/agent/ctl komutları henüz stub.
- internal/sim: OpenAI uyumlu sahte inference engine. Router'ı GPU olmadan
  gerçekçi koşullarda test edebilmek için gerçek bir engine'in router
  açısından önemli davranışlarını modeller:
    * continuous batching ve KV blok muhasebesi (head-of-line blocking dahil)
    * zincirlenmiş blok hash'leri + LRU ile otomatik prefix cache
    * batch büyüdükçe artan token gecikmesi
    * istemci koptuğunda isteğin düşürülüp blokların bırakılması
    * vLLM metrik adlarıyla /metrics (plan §7.6 engine kontratı)
- Testler: prefix hash zincirlemesi, LRU tahliyesi, iptal propagasyonu,
  kuyruklama, SSE formatı, usage/cached_tokens, metriklerin gerçek
  Prometheus parser'ıyla okunabilirliği.
- Gitea Actions CI: gofmt, vet, race detector.

Modül yolu gitea.mynodes.xyz/ilker/llmgw; `make rename` ile değiştirilebilir.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 01:00:03 +02:00