Commit Graph
3 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 bae16426a2 Offline sites and Kubernetes: llmgw bundle create, import and trust, offline.mirror, server init --bundle; a Helm chart for the control plane
A bundle carries what a cluster would download — llmgw's release with its
signed checksums, every llama.cpp build per architecture with GitHub's
checksums, model files pinned and checked against Hugging Face's — into a
directory or a tar whose manifest sha256 is carried apart. Import checks
everything, the release against the signing key, before anything is put in
the mirror. With offline.mirror set, nodes install llama.cpp from the
mirror against checksums the control plane holds and sends over the agent
stream; releases default to the mirror for node add, ctl upgrade and agent
upgrades. server init and the setup form install from a bundle.

The Helm chart runs one control plane with SQLite on a claim or an external
PostgreSQL, Secrets for the agent certificate and admin token, Services,
Ingress and ServiceMonitor; deploy/container/Dockerfile builds its image.
GPU nodes stay outside Kubernetes, as docs/kubernetes.md explains.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-05 00:13:39 +02:00
Ilker ManapandClaude Opus 5.5 2f07b2ba0b Document guardrails, correct what data.md said leaves the cluster, and regenerate the reference
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-04 12:36:37 +02:00
Ilker ManapandClaude Opus 5 b7a97468e0 Answer the two questions asked before a gateway is allowed near anything
Neither had a page. "We have GPUs, now what" was left to the reader, and what
llmgw stores was scattered across four documents and a changelog — which is
the same as not being written down when somebody is doing a security review.

docs/data.md describes the behaviour and leaves the legal mapping to whoever
holds the obligation. Every claim in it was checked against the source, and it
names where to look for each: usage rows carry no text field, keys are stored
hashed, conversations are off until a named project turns them on and cannot be
turned on without a retention window.

It also states the egress exactly, which took reading every outbound call to
say honestly: nothing on the serving path leaves the cluster, and the two
things that can reach outside — fetching weights, and ctl plan reading a model's
architecture — are both started by an operator. Weight fetching points wherever
you tell it, including an internal mirror.

docs/choosing-models.md is the opinion llmgw itself does not have. Three jobs
rather than one model, with the third — embeddings — being the one that gets
forgotten and the one that decides whether the deployment can answer questions
about your company or only questions in general.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-27 08:16:08 +02:00