Ilker ManapandClaude Opus 5.5
bae16426a2
A bundle carries what a cluster would download — llmgw's release with its signed checksums, every llama.cpp build per architecture with GitHub's checksums, model files pinned and checked against Hugging Face's — into a directory or a tar whose manifest sha256 is carried apart. Import checks everything, the release against the signing key, before anything is put in the mirror. With offline.mirror set, nodes install llama.cpp from the mirror against checksums the control plane holds and sends over the agent stream; releases default to the mirror for node add, ctl upgrade and agent upgrades. server init and the setup form install from a bundle. The Helm chart runs one control plane with SQLite on a claim or an external PostgreSQL, Secrets for the agent certificate and admin token, Services, Ingress and ServiceMonitor; deploy/container/Dockerfile builds its image. GPU nodes stay outside Kubernetes, as docs/kubernetes.md explains. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
41 lines
1.6 KiB
Docker
41 lines
1.6 KiB
Docker
# The llmgw control plane as a container image, for deploy/helm/llmgw.
|
|
#
|
|
# The project publishes binaries, not an image, so this is built where the
|
|
# image is to live:
|
|
#
|
|
# docker build -f deploy/container/Dockerfile --build-arg VERSION=v0.11.1 \
|
|
# -t registry.internal/llmgw:v0.11.1 .
|
|
# docker push registry.internal/llmgw:v0.11.1
|
|
#
|
|
# The binary is static (CGO_ENABLED=0, as every release is), so the image is
|
|
# that one file on a base with nothing else in it: no shell, no package
|
|
# manager. `kubectl exec ... -- llmgw ctl status` still works, because it
|
|
# runs the binary rather than a shell.
|
|
#
|
|
# Only the control plane runs from it. An agent installs engines on its
|
|
# machine, manages its GPUs and supervises processes there as root, which is
|
|
# what a container is built to prevent: docs/kubernetes.md.
|
|
|
|
FROM golang:1.26 AS build
|
|
WORKDIR /src
|
|
COPY go.mod go.sum ./
|
|
RUN go mod download
|
|
COPY . .
|
|
ARG VERSION=dev
|
|
RUN CGO_ENABLED=0 go build -trimpath \
|
|
-ldflags "-s -w -X gitea.mynodes.xyz/ilker/llmgw/internal/version.Version=${VERSION}" \
|
|
-o /out/llmgw ./cmd/llmgw && \
|
|
mkdir -p /out/state
|
|
|
|
FROM gcr.io/distroless/static-debian12:nonroot
|
|
COPY --from=build /out/llmgw /usr/local/bin/llmgw
|
|
# The chart mounts server.yaml, the certificates and the token under
|
|
# /etc/llmgw, and the SQLite store, when it is one, under /var/lib/llmgw —
|
|
# owned by the user the server runs as, so a volume Docker fills from the
|
|
# image is writable without Kubernetes' fsGroup.
|
|
COPY --from=build --chown=65532:65532 /out/state /var/lib/llmgw
|
|
USER 65532:65532
|
|
EXPOSE 8080 7443 9091
|
|
ENTRYPOINT ["/usr/local/bin/llmgw"]
|
|
CMD ["server", "--config", "/etc/llmgw/server.yaml"]
|