Make a new control plane serve a model by itself
server init now also: - runs a model mirror beside the control plane (/srv/models, bytes on :8081 at an address nodes reach, management API on loopback so it needs no certificate) and adds it to server.yaml; - makes the machine a node of its own control plane: issues its agent certificate, writes agent.yaml labelled as the control plane, starts llmgw-agent, and waits for it to register and install llama.cpp; - checks for NVIDIA GPUs with nvidia-smi, and reports a card on the PCI bus whose driver does not answer rather than ignoring it; - fetches Qwen2.5 0.5B (Q4_K_M) into the mirror and adds it to the catalog pinned to this node: a llama.cpp profile per GPU class found, with a CPU profile as the fallback, then waits for it to serve. --no-mirror, --no-node and --no-model leave these out. Every step looks before it acts, so a rerun finishes what an interrupted one started. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
5501f77805
commit
42fa602247
5 files changed
+663
No files matched your search
@@ -104,6 +104,17 @@ That is the whole step. In order, it:
|
||||
and promoted to administrator — the password is generated and **printed
|
||||
once**
|
||||
6. installs the systemd unit, starts it, and waits until `/health` answers
|
||||
7. runs a model mirror beside it (`/srv/models`, bytes on `:8081`, its
|
||||
management API on loopback) and points the control plane at it
|
||||
8. makes this machine a node of its own control plane — on its GPUs when
|
||||
`nvidia-smi` finds any, on CPU otherwise — and waits for it to install its
|
||||
engine
|
||||
9. fetches Qwen2.5 0.5B into the mirror and serves it on this node, so the
|
||||
first thing anyone opens `/chat` to find is a model that answers
|
||||
|
||||
`--no-mirror`, `--no-node` and `--no-model` leave the last three out. An NVIDIA
|
||||
card with no working driver is reported rather than silently ignored; install
|
||||
the driver and run init again to serve on it.
|
||||
|
||||
Add `--postgres` to keep accounts and keys in PostgreSQL rather than SQLite:
|
||||
init installs it and sets it up, and on a machine already running on SQLite it
|
||||
|
||||
Reference in new issue
Block a user