Merge C18 offline bundles and the Helm chart

Conflicts: server.go imports (both kept); agent.pb.go regenerated with buf
from the merged proto.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 00:15:47 +02:00
commit 6f85c17fc5
51 files changed
+4177 -63

No files matched your search

+78
View File
@@ -0,0 +1,78 @@
# C18 — Kubernetes/Helm and an air-gapped bundle
README's documentation list should gain a line for `docs/kubernetes.md`
(the control plane in Kubernetes from the Helm chart, and why GPU nodes stay
outside it) — README.md was not edited here.
## CHANGELOG
### Offline sites and Kubernetes
- **Offline bundles.** `llmgw bundle create` (on any machine with internet)
collects what a cluster would otherwise download — llmgw's release for
each `--arch` with its signed checksums, every llama.cpp build for those
architectures (or the `--variant` ones) with the checksums GitHub
publishes, the starter model and guardrails' judge, and any `--model
name/version=hf:owner/repo/file` pinned to the registry's commit and its
checksum — each checked as it arrives, into a directory or one `.tar`
(`--tar`), with a manifest whose sha256 it prints to carry apart from the
bundle. Run again into the same directory, it downloads only what is
missing. `llmgw bundle show --verify` reads one back.
- **`llmgw bundle import`** checks every file against the manifest and
llmgw's release against the project's signing key, and only then puts them
into the mirror — releases and engines under `_llmgw/`, models where the
mirror keeps models, with their checksums — so a damaged bundle leaves the
mirror as it was; a different file under a version already there is
refused. On a control plane it also records the engines' checksums in
`offline.checksums` (`/etc/llmgw/offline.sha256`); `llmgw bundle trust`
does that alone, for a mirror on another machine.
- **`offline.mirror` in server.yaml** makes the cluster reach nothing
outside: nodes install llama.cpp from the mirror, each file held to the
checksum the control plane sends them over the agent's authenticated
connection — never one from the mirror — and `releases` defaults to the
mirror's copy, so `node add`, `ctl upgrade` and agents upgrading fetch
from it; no registry is asked about newer weights. A release may now come
over plain HTTP when the build trusts a signing key, since the signature
holds whatever carries it. vLLM and SGLang come from PyPI and are said to
be unsupported on an offline node rather than attempted.
- **`llmgw server init --bundle PATH --bundle-sha256 …`**, and the same two
fields on the setup form: installs a control plane offline — the bundle
into the mirror, the checksums trusted, `offline:` written to server.yaml,
the starter model and judge served from the bundle or skipped when it does
not carry them.
- `docs/data.md` no longer says only weights and `ctl plan` reach outside:
it lists everything that does, and what an offline cluster changes.
- **A Helm chart for the control plane** (`deploy/helm/llmgw`): one replica
in a StatefulSet, SQLite on a claim or an external PostgreSQL, the agent
port's certificate and the admin token from Secrets, the API and metrics
Service, a LoadBalancer for the agents' mutual TLS, an optional Ingress
and ServiceMonitor, `config` laid into server.yaml, and the offline
checksums from a ConfigMap; read-only root, non-root, ready on `/leader`.
`deploy/container/Dockerfile` builds the image it runs. GPU nodes stay
outside Kubernetes and join with `node add` (docs/kubernetes.md says why).
## BACKLOG
**Done (branch `features-c2`):** offline bundles — `llmgw bundle create`
(releases with signed checksums, every llama.cpp build per arch/variant
with GitHub's checksums, starter/judge and named Hugging Face model files
pinned and checked; directory or tar; manifest sha256), `bundle import`
(checked, all-or-nothing, into the mirror under `_llmgw/`), `bundle trust`,
`bundle show --verify`; `offline.mirror`/`offline.checksums` in server.yaml,
with nodes installing llama.cpp from the mirror against checksums the
control plane sends over the agent stream (proto `EngineWant.offline_*`),
releases defaulting to the mirror for `node add`, `ctl upgrade` and agent
upgrades, update checks off; `server init --bundle` and the setup form's
bundle fields; docs/data.md's offline claim corrected; install.md "Offline
sites". A Helm chart for the control plane (`deploy/helm/llmgw`, one replica,
SQLite PVC or external PostgreSQL, Secrets, Services, Ingress,
ServiceMonitor) with `deploy/container/Dockerfile` and docs/kubernetes.md;
GPU nodes stay outside Kubernetes by design. Left: vLLM/SGLang offline (a
wheelhouse of several GB per CUDA build and a uv `--find-links` install
path); the NVML release binary and uv in bundles; the setup wizard's
preflight still fails a machine without internet; `join`'s printed `curl`
from the mirror checks nothing; models in a bundle are single files
(no whole-repository safetensors sets yet); the chart runs one replica (a
standby is never ready, so HA in Kubernetes needs a different readiness
design), has not been installed on a real cluster, and ships no agent
DaemonSet; no published container image.