Ilker ManapandClaude Opus 5.5
bd12b0909a
Adding a model took four steps and a YAML file — plan it, fetch it into the mirror, write its catalog entry with a memory figure somebody estimated, scale it — and whether it would load was found out at the end. `llmgw ctl model serve OWNER/REPO` does it as one job in the control plane, where the registry, the mirror, the catalog and the fleet already are: - the registry: the GGUF file set matching --quant (Q4_K_M by default) or --file, leaving out vision projectors and importance matrices, and naming what the repository does have when nothing matches; gated repositories are refused; - the mirror: fetched at the repository's revision, with progress, and each file's sha256 compared with what the registry published — the mirror is the one place a corrupted download is caught once rather than loaded on every node; - the check: whether the nodes' llama.cpp loads the architecture, and the memory it needs from its own header; - the catalog, then a wait for a replica that stops early, with the reason, when the model is parked or fits on no node; - a smoke test: one short request to the replica, with its engine key, and the answer and tokens per second. ctl starts the job and prints each step as it changes; a download of hours does not depend on the terminal. --on-demand adds the model without starting a replica. Jobs are kept in memory until the control plane restarts. The catalog name a repository suggests moved into the registry package, so the mirror page and this agree on it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>