# C18 — Kubernetes/Helm and an air-gapped bundle README's documentation list should gain a line for `docs/kubernetes.md` (the control plane in Kubernetes from the Helm chart, and why GPU nodes stay outside it) — README.md was not edited here. ## CHANGELOG ### Offline sites and Kubernetes - **Offline bundles.** `llmgw bundle create` (on any machine with internet) collects what a cluster would otherwise download — llmgw's release for each `--arch` with its signed checksums, every llama.cpp build for those architectures (or the `--variant` ones) with the checksums GitHub publishes, the starter model and guardrails' judge, and any `--model name/version=hf:owner/repo/file` pinned to the registry's commit and its checksum — each checked as it arrives, into a directory or one `.tar` (`--tar`), with a manifest whose sha256 it prints to carry apart from the bundle. Run again into the same directory, it downloads only what is missing. `llmgw bundle show --verify` reads one back. - **`llmgw bundle import`** checks every file against the manifest and llmgw's release against the project's signing key, and only then puts them into the mirror — releases and engines under `_llmgw/`, models where the mirror keeps models, with their checksums — so a damaged bundle leaves the mirror as it was; a different file under a version already there is refused. On a control plane it also records the engines' checksums in `offline.checksums` (`/etc/llmgw/offline.sha256`); `llmgw bundle trust` does that alone, for a mirror on another machine. - **`offline.mirror` in server.yaml** makes the cluster reach nothing outside: nodes install llama.cpp from the mirror, each file held to the checksum the control plane sends them over the agent's authenticated connection — never one from the mirror — and `releases` defaults to the mirror's copy, so `node add`, `ctl upgrade` and agents upgrading fetch from it; no registry is asked about newer weights. A release may now come over plain HTTP when the build trusts a signing key, since the signature holds whatever carries it. vLLM and SGLang come from PyPI and are said to be unsupported on an offline node rather than attempted. - **`llmgw server init --bundle PATH --bundle-sha256 …`**, and the same two fields on the setup form: installs a control plane offline — the bundle into the mirror, the checksums trusted, `offline:` written to server.yaml, the starter model and judge served from the bundle or skipped when it does not carry them. - `docs/data.md` no longer says only weights and `ctl plan` reach outside: it lists everything that does, and what an offline cluster changes. - **A Helm chart for the control plane** (`deploy/helm/llmgw`): one replica in a StatefulSet, SQLite on a claim or an external PostgreSQL, the agent port's certificate and the admin token from Secrets, the API and metrics Service, a LoadBalancer for the agents' mutual TLS, an optional Ingress and ServiceMonitor, `config` laid into server.yaml, and the offline checksums from a ConfigMap; read-only root, non-root, ready on `/leader`. `deploy/container/Dockerfile` builds the image it runs. GPU nodes stay outside Kubernetes and join with `node add` (docs/kubernetes.md says why). ## BACKLOG **Done (branch `features-c2`):** offline bundles — `llmgw bundle create` (releases with signed checksums, every llama.cpp build per arch/variant with GitHub's checksums, starter/judge and named Hugging Face model files pinned and checked; directory or tar; manifest sha256), `bundle import` (checked, all-or-nothing, into the mirror under `_llmgw/`), `bundle trust`, `bundle show --verify`; `offline.mirror`/`offline.checksums` in server.yaml, with nodes installing llama.cpp from the mirror against checksums the control plane sends over the agent stream (proto `EngineWant.offline_*`), releases defaulting to the mirror for `node add`, `ctl upgrade` and agent upgrades, update checks off; `server init --bundle` and the setup form's bundle fields; docs/data.md's offline claim corrected; install.md "Offline sites". A Helm chart for the control plane (`deploy/helm/llmgw`, one replica, SQLite PVC or external PostgreSQL, Secrets, Services, Ingress, ServiceMonitor) with `deploy/container/Dockerfile` and docs/kubernetes.md; GPU nodes stay outside Kubernetes by design. Left: vLLM/SGLang offline (a wheelhouse of several GB per CUDA build and a uv `--find-links` install path); the NVML release binary and uv in bundles; the setup wizard's preflight still fails a machine without internet; `join`'s printed `curl` from the mirror checks nothing; models in a bundle are single files (no whole-repository safetensors sets yet); the chart runs one replica (a standby is never ready, so HA in Kubernetes needs a different readiness design), has not been installed on a real cluster, and ships no agent DaemonSet; no published container image.