Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane

A bundle carries what a cluster would download — llmgw's release with its
signed checksums, every llama.cpp build per architecture with GitHub's
checksums, model files pinned and checked against Hugging Face's — into a
directory or a tar whose manifest sha256 is carried apart. Import checks
everything, the release against the signing key, before anything is put in
the mirror. With offline.mirror set, nodes install llama.cpp from the
mirror against checksums the control plane holds and sends over the agent
stream; releases default to the mirror for node add, ctl upgrade and agent
upgrades. server init and the setup form install from a bundle.

The Helm chart runs one control plane with SQLite on a claim or an external
PostgreSQL, Secrets for the agent certificate and admin token, Services,
Ingress and ServiceMonitor; deploy/container/Dockerfile builds its image.
GPU nodes stay outside Kubernetes, as docs/kubernetes.md explains.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-05 00:13:39 +02:00
1 parent e59ff52932
commit bae16426a2
51 files changed
+4177 -63

No files matched your search

+36
View File
@@ -120,6 +120,8 @@ rather than repeated.
| `roles` | `map[string][]string` | [OIDC](#oidc) |
| `memberships` | `map[string]string` | [OIDC](#oidc) |
| `no_passwords` | `bool` | [OIDC](#oidc) |
| `mirror` | `string` | [Offline](#offline) |
| `checksums` | `string` | [Offline](#offline) |
| `nvidia_smi` | `string` | [Options](#options) |
| `ib_root` | `string` | [Options](#options) |
| `prefer_nvml` | `bool` | [Options](#options) |
@@ -226,6 +228,7 @@ rather than repeated.
| `monitoring` | `Monitoring` | [Server](#server) |
| `tracing` | `Tracing` | [Server](#server) |
| `mirror` | `Mirror` | [Server](#server) |
| `offline` | `Offline` | [Server](#server) |
| `engines` | `map[string]string` | [Server](#server) |
| `engine_overrides` | `[]EngineOverride` | [Server](#server) |
| `backup_dir` | `string` | [Server](#server) |
@@ -1084,6 +1087,29 @@ PasswordSignIn off refuses passwords on the sign-in form: people come
in through the provider only. Administrators still can, so a broken
provider does not lock everybody out.
## Offline
### `mirror`
`string`
Mirror is the mirror's file address, as nodes reach it:
http://10.0.0.5:8081 — the port model files are served on, not the
management API. Set, the cluster is offline: nodes install engines
from it and never from their publishers, releases come from it unless
releases names another place, and no registry is asked whether a
model has newer weights. Python engines (vLLM, SGLang) come from PyPI,
which no bundle carries; on an offline node they are installed by hand.
### `checksums`
`string`
Checksums is the file of engine checksums this control plane trusts,
kept by llmgw bundle import and llmgw bundle trust; every engine file
a node fetches from the mirror is checked against it. Read again
whenever it changes. Empty is /etc/llmgw/offline.sha256.
## Options
### `nvidia_smi`
@@ -1834,6 +1860,16 @@ Empty means there is none, and /admin/mirror stays unmounted.
Takes a [Mirror](#mirror) block.
### `offline`
`Offline`
Offline is a cluster that reaches nothing outside itself: its nodes
install engines, and upgrades fetch releases, from the mirror, out of
a bundle carried in (llmgw bundle create, llmgw bundle import).
Takes a [Offline](#offline) block.
### `engines`
`map[string]string`