# A model mirror One machine that holds the model files, so nodes fetch over your own network instead of over the internet. This is the highest-value piece of infrastructure around llmgw and the cheapest. Eight nodes each pulling forty gigabytes from Hugging Face is a day's work, and on plenty of corporate networks it is not possible at all. From a mirror on the same switch it is minutes. llmgw needs nothing special for this: a model's `source` is a URL, and the URL can be yours. ## What it is An HTTP server serving a directory. That is the whole design. ``` /srv/models/ qwen2.5-14b-instruct/ 2026-03/ q4_k_m.gguf bge-m3/ 2026-01/ model.safetensors ``` The date in the path is the point of the layout, not decoration. A model repository that changes upstream changes what you are running, and the first sign is a prompt that worked last week answering differently this week with nobody able to say why. Pin it here and that cannot happen. ## nginx ```nginx server { listen 80; server_name models.internal; root /srv/models; # Read-only. This is deliberate: a mirror that can be written to over HTTP # is a way to replace a model with another one, and the nodes would fetch # it without complaint. location / { autoindex on; # Large files, and the client is a machine with a resumable fetcher. # Sendfile and no buffering keep memory flat regardless of size. sendfile on; tcp_nopush on; } } ``` `autoindex` is there so a person can browse what is available; llmgw does not need it. ## Filling it ``` # On the mirror, once per model and version: mkdir -p /srv/models/qwen2.5-14b-instruct/2026-03 curl -fL -o /srv/models/qwen2.5-14b-instruct/2026-03/q4_k_m.gguf \ "https://huggingface.co//resolve//" ``` Use a revision rather than `main` in the source URL. `main` is what moves. ## Pointing llmgw at it ```yaml - name: qwen2.5-14b-instruct source: http://models.internal/qwen2.5-14b-instruct/2026-03/q4_k_m.gguf scaling: { min: 2, max: 2 } profiles: - name: gpu engine: vllm gpu_class: h100-80g tp: 1 ``` Each node fetches once, in the background, before its engine starts — and `llmgw ctl replicas` shows how far along it is while it does. A killed download resumes rather than restarting, and a partial file is never visible under the model's name, so an interrupted fetch cannot be handed to an engine. ## What this is not It is not a cache: nothing populates it automatically, and a model you did not put there is not there. That is on purpose. A mirror that fetches on demand is a mirror that reaches the internet at the least convenient moment, and in a network where that is blocked it fails in a way that looks like llmgw being broken. It is also not a replacement for a shared filesystem. If you already run one that every node mounts, put the models on it and use a path as the `source` — nothing is fetched at all, and llmgw notices the filesystem is shared and will not count the same free space once per node. ## When this stops being enough Somewhere above ten or twenty nodes, one mirror serving a forty gigabyte file to all of them at once becomes the bottleneck instead of the fix. At that point the options are a filer, a second mirror per rack, or distribution between the nodes themselves. llmgw knows which nodes already hold which weights — it is in `ctl status` as "N models on X/Y nodes" — so distributing between them is a feature it is positioned to add. It has not been built, because a mirror is enough for everyone who has needed it so far.