Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

A bundle carries what a cluster would download — llmgw's release with its
signed checksums, every llama.cpp build per architecture with GitHub's
checksums, model files pinned and checked against Hugging Face's — into a
directory or a tar whose manifest sha256 is carried apart. Import checks
everything, the release against the signing key, before anything is put in
the mirror. With offline.mirror set, nodes install llama.cpp from the
mirror against checksums the control plane holds and sends over the agent
stream; releases default to the mirror for node add, ctl upgrade and agent
upgrades. server init and the setup form install from a bundle.

The Helm chart runs one control plane with SQLite on a claim or an external
PostgreSQL, Secrets for the agent certificate and admin token, Services,
Ingress and ServiceMonitor; deploy/container/Dockerfile builds its image.
GPU nodes stay outside Kubernetes, as docs/kubernetes.md explains.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 00:13:39 +02:00
1 parent e59ff52932
commit bae16426a2
51 files changed
+4177 -63

No files matched your search

+12
View File
@@ -52,6 +52,18 @@ cluster, a backup and adding a node change this machine or another over ssh;
those close the screen and run in the terminal, so their output is there to
read, and Enter brings the console back.
## Setting up
On a machine with nothing installed, the control plane is set up from one
form (`sudo llmgw server init`): the administrator's email, other names nodes
reach the machine by, and ticks for PostgreSQL, the model mirror, this machine
as a node, the starter model and the guardrails' judge. **Offline bundle**
and **Its sha256** are for a site with no internet: the path of a bundle made
by `llmgw bundle create`, and the sha256 it printed. Given, the bundle goes
into the mirror, every file checked, and the cluster is set up to fetch
nothing from outside (install.md, "Offline sites"). A bundle with no sha256,
or with the mirror unticked, is said in the form and nothing starts.
## Watch
**Overview** is `llmgw ctl status`: nodes, GPUs by class, fabrics, replicas,