ilker/llmgw
2
0 Fork 1
Code Issues Pull Requests Actions Packages Projects Releases 95 Wiki Activity
375 Commits 9 Branches 95 Tags
c19abe95e1e08dae213eaa4521bd248e5033f6cd
Commit Graph
6 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 e12996f7dd Serve and update models in the background from the console, and follow them under Jobs 
The console starts a serve or update job over the admin API and stays,
instead of closing to follow the download. Jobs lists them with their
progress and steps; the sidebar counts those running. ctl model serve and
update take --detach; ctl jobs lists jobs and follows one.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 01:27:02 +02:00
Ilker ManapandClaude Opus 5.5 3b66784f22 Name each file by its quantisation first in the console's plan 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 01:09:05 +02:00
Ilker ManapandClaude Opus 5.5 54db7aecb7 Plan a repository's files against the fleet, and serve the one that suits it the way it fits 
The control plane reads the model's shape from one GGUF header or its
config.json and lists, for every file, the best way it runs here: one card,
several of a node's (by layer for llama.cpp, tensor-parallel for vLLM),
several nodes' over one fabric (vLLM, pipeline-parallel between them), or a
CPU node; what each card holds and whether it is free now. ctl model plan
prints it; model serve takes its file and way unless told, gives a GGUF on
GPUs a GPU profile, and with --fallbacks adds the next best ways. The
console shows it per file and asks for the way and the fallback.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 01:03:48 +02:00
Ilker ManapandClaude Opus 5.5 a16097cfe8 Serve vision models with their projector and original weights on vLLM; group the mirror by version 
model serve takes --projector, fetched beside the weights and loaded by
llama.cpp with --mmproj, and --format safetensors with --gpu-class and --tp,
which fetches the weights with their config and tokenizer and runs them on
vLLM. The console offers both, F switching between GGUF and original
weights. The mirror view is one row per version with its files on Enter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-04 00:42:59 +02:00
Ilker ManapandClaude Opus 5.5 a96b1a54b2 Hugging Face in the console: models that chat when browsing, ages read at a glance, long menus scroll
CI / test (push) Successful in 6m1s
Details 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-03 23:47:26 +02:00
Ilker ManapandClaude Opus 5.5 2bf43ed90c Search Hugging Face from the console and serve what is found
CI / test (push) Successful in 5m3s
Details 
Serve a model is a view of the registry's GGUF models, trending or searched;
Enter on one lists its files, and the one chosen is served with
llmgw ctl model serve. A form of one field is applied by Enter on it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-03 23:44:51 +02:00