Make a new control plane serve a model by itself

server init now also:
- runs a model mirror beside the control plane (/srv/models, bytes on
  :8081 at an address nodes reach, management API on loopback so it
  needs no certificate) and adds it to server.yaml;
- makes the machine a node of its own control plane: issues its agent
  certificate, writes agent.yaml labelled as the control plane, starts
  llmgw-agent, and waits for it to register and install llama.cpp;
- checks for NVIDIA GPUs with nvidia-smi, and reports a card on the PCI
  bus whose driver does not answer rather than ignoring it;
- fetches Qwen2.5 0.5B (Q4_K_M) into the mirror and adds it to the
  catalog pinned to this node: a llama.cpp profile per GPU class found,
  with a CPU profile as the fallback, then waits for it to serve.

--no-mirror, --no-node and --no-model leave these out. Every step looks
before it acts, so a rerun finishes what an interrupted one started.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-09-28 16:49:55 +02:00
1 parent 5501f77805
commit 42fa602247
5 files changed
+663

No files matched your search

+11
View File
@@ -104,6 +104,17 @@ That is the whole step. In order, it:
and promoted to administrator — the password is generated and **printed
once**
6. installs the systemd unit, starts it, and waits until `/health` answers
7. runs a model mirror beside it (`/srv/models`, bytes on `:8081`, its
management API on loopback) and points the control plane at it
8. makes this machine a node of its own control plane — on its GPUs when
`nvidia-smi` finds any, on CPU otherwise — and waits for it to install its
engine
9. fetches Qwen2.5 0.5B into the mirror and serves it on this node, so the
first thing anyone opens `/chat` to find is a model that answers
`--no-mirror`, `--no-node` and `--no-model` leave the last three out. An NVIDIA
card with no working driver is reported rather than silently ignored; install
the driver and run init again to serve on it.
Add `--postgres` to keep accounts and keys in PostgreSQL rather than SQLite:
init installs it and sets it up, and on a machine already running on SQLite it