Files
llmgw/docs
Ilker ManapandClaude Opus 5.5 42fa602247 Make a new control plane serve a model by itself
server init now also:
- runs a model mirror beside the control plane (/srv/models, bytes on
  :8081 at an address nodes reach, management API on loopback so it
  needs no certificate) and adds it to server.yaml;
- makes the machine a node of its own control plane: issues its agent
  certificate, writes agent.yaml labelled as the control plane, starts
  llmgw-agent, and waits for it to register and install llama.cpp;
- checks for NVIDIA GPUs with nvidia-smi, and reports a card on the PCI
  bus whose driver does not answer rather than ignoring it;
- fetches Qwen2.5 0.5B (Q4_K_M) into the mirror and adds it to the
  catalog pinned to this node: a llama.cpp profile per GPU class found,
  with a CPU profile as the fallback, then waits for it to serve.

--no-mirror, --no-node and --no-model leave these out. Every step looks
before it acts, so a rerun finishes what an interrupted one started.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 16:49:55 +02:00
..
2026-09-24 04:04:03 +03:00