5 Commits

Include renames

Author	SHA1	Message	Date
Ilker ManapandClaude Opus 5.5	7e57863edd	Hide the mirror's bookkeeping, bound GGUF array nesting, and forget old serve jobs and downloads ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-04 10:56:16 +02:00
Ilker ManapandClaude Opus 5.5	6b1fa65f59	Format kind_live_test.go ... Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 07:20:35 +02:00
Ilker ManapandClaude Opus 5.5	663770226e	Find LoRA adapters on Hugging Face and attach them from the mirror page ... An adapters view in the explorer, filtered by the model a held model was made from; GGUF files told apart as adapter or model by their first bytes; held adapter files marked; and a model's card attaches and detaches held adapters of its architecture, refusing models and mismatched architectures. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-10-01 07:19:08 +02:00
Ilker ManapandClaude Opus 5.5	4a652fdcc1	Size a llama.cpp model's memory from its own shape, not its file size ... The serve form asked for weights plus half again plus two gigabytes. That ignores the one thing that varies most: the KV cache. At the same 16k pool, the models on the live mirror need from 0.44 GB of cache (Qwen2.5-1.5B, two KV heads) to 5.25 GB (Gemma-2-9B, 256-wide heads) — so the flat guess overcounted the small models, which on 7 GB nodes means refusing placements that fit, and undercounted the ones with wide or many KV heads, which means an engine killed for memory. The GGUF header already carries the shape. The reader now also takes each architecture's block_count, head counts, embedding length and key/value lengths, reading past general.* until the tokenizer — and a header whose tokenizer comes first and runs past the fetched bytes still returns what it has rather than failing. The estimate is weights, plus the f16 KV cache for the context pool (llama.cpp's --ctx-size, which --parallel divides rather than multiplies), plus a gigabyte and five percent for the engine. It reuses the header the servability check already fetched, and the form says what it is made of. Checked against all twelve GGUF models on the live mirror, whose shapes read back as their published architectures. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-30 02:40:00 +02:00
Ilker ManapandClaude Opus 5.5	f13b18149a	Refuse weights the nodes' llama.cpp cannot load, before fetching them ... kev-0.5b declared architecture "ggmlc", which llama.cpp does not know. It was fetched, catalogued and placed, and failed on every node — while the answer was in the first kilobytes of the file. internal/gguf reads a GGUF header (v1-v3) up to general.architecture, over HTTP with a Range request, and judges it against the architectures llama.cpp b10991 lists in LLM_ARCH_NAMES (152, copied from the source at that tag); a vision projector is not a model on its own. An unknown build is not refused, only not vouched for, and a header that cannot be fetched does not refuse anything. Checked in three places: the registry listing (from the architecture Hugging Face reports with expand[]=gguf, so one request per repository) shows it and withholds the fetch button; a fetch reads the header and refuses unless "fetch anyway"; and the model service refuses to add such a model, which covers the serve form and ctl model add. Tried on real files: the mirror's qwen2.5-1.5b (qwen2) and bge-m3 (bert), and Qwen2.5-0.5B on Hugging Face through its redirect. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>	2026-09-28 20:44:36 +02:00