Make the model mirror a file you can copy, not an idea you have to have
CI / test (push) Successful in 1m39s
CI / test (push) Successful in 1m39s
"Set up a local mirror" was advice. This is the nginx config, the directory layout and the reason the layout has a date in it — a repository that changes upstream changes what you are running, and the first sign is a prompt that worked last week answering differently this week. It says what the mirror is not, because both mistakes are easy. It is not a caching proxy: nothing populates it on demand, deliberately, since a mirror that reaches the internet at the least convenient moment fails in a way that looks like llmgw being broken. And it is not needed at all if you already run a shared filesystem — put the models there, use a path, and nothing is fetched. It also says when a mirror stops being enough, and that llmgw is positioned to distribute between nodes because it already knows which of them hold which weights. That is not built, and the page says so rather than implying it is coming. Every behaviour it relies on was checked against the source: an http:// source is fetched and a path is not, a killed download resumes, a partial file is never visible under the model's name, and a shared filesystem is not counted once per node. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
b7a97468e0
commit
e86457cf02
3 files changed
+114
-1
No files matched your search
@@ -64,6 +64,8 @@ on a machine that has them.
|
||||
- [docs/reference.md](docs/reference.md) — every configuration key, generated
|
||||
from the structs and checked by a test, so it cannot describe a setting that
|
||||
does not exist or miss one that does
|
||||
- [deploy/mirror/](deploy/mirror/) — a model mirror, which is the cheapest
|
||||
and most useful thing to build around llmgw
|
||||
- [deploy/ha/](deploy/ha/) — running two control planes, and the one health
|
||||
check that has to be right
|
||||
- [docs/node-onboarding.md](docs/node-onboarding.md),
|
||||
|
||||
Reference in new issue
Block a user