The mirror page's Strata tab; Strata's ready-made packs served with Strata fetching them, strata-ple named
CI / test (push) Successful in 10m35s

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-30 12:06:46 +01:00
1 parent 4ee9ba54ef
commit 845de8eef4
13 files changed
+513 -21

No files matched your search

+4
View File
@@ -1889,6 +1889,10 @@ served only from a definition written by hand. In order:
- N7. The web serve page, the explorer's fit, the console's serve form: the same choice. Done 2026-10-30.
- N8. `--context`/`--parallel` tied to `context_length`/`max_output`; `ctl model context` takes Strata.
- N7b. The mirror page's Strata tab: every family and size, fetch (GGUF) or serve (packs Strata fetches itself:
NVFP4, DeepSeek V4.1). Done 2026-10-30. Community `strata-ple` packs Strata 0.2.7 does not list stay unservable;
running them would mean starting the engine past Strata's setup (`--pack`, `--native`, `--ple-gguf`).
**C. Operating it.**
- N9. `ctl engine-check strata`: the flags of setup.py and server.py, which drift between versions.
- N10. `ctl engine-build NODE strata cuda12|cuda13`, as vLLM has.
+10
View File
@@ -26,6 +26,16 @@ is only released once its **exit criterion** is met, not when its code lands.
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
APIs change between minors without a deprecation period.
## Unreleased
### Added
- **The mirror page's Strata tab.** Every family and size Strata runs, with its download, the RAM it keeps and where
it would run here; a GGUF size fetched into the mirror in one press (checked against Hugging Face's checksums),
and the packs Strata fetches itself — NVFP4, DeepSeek V4.1 — served at once, their repository the source.
`ctl model plan` and `ctl model serve` take those repositories too, and the explorer names a `strata-ple`
repository for what it is: part of a pack Strata knows, or one Strata 0.2.7 does not.
## v0.16.0 — 2026-10-30
Weights pinned to the commit they were added at and checked, by the nodes and the mirror, against the sha256
+18 -1
View File
@@ -761,7 +761,24 @@ llmgw ctl model serve ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF \
```
The Hugging Face explorer shows the same ("1× … with Strata, N GB of RAM"),
and the mirror's serve form serves such a version on Strata. A Strata profile
and the mirror's serve form serves such a version on Strata.
The mirror page's **Strata** tab lists every family and size Strata runs, with
what each downloads and keeps in RAM and where it would run here:
- **Fetch into the mirror**, for the GGUF families: the size's files, checked
against Hugging Face's checksums; the version's serve form then serves it on
Strata.
- **Serve**, for the packs Strata fetches itself — the NVFP4 packs
(`Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata` and its NVIDIA
sibling, NVIDIA's FP4-Local) and DeepSeek V4.1: the model is added with the
repository as its source, and its first start fetches tens of gigabytes onto
the node, each file pinned and checked by Strata, before preparing it.
`ctl model serve` does the same for these repositories.
Repositories of architecture `strata-ple` that Strata 0.2.7 does not list —
community packs such as `…-Q8_0-Strata` — are named as such in the explorer:
Strata's setup refuses them, and llama.cpp cannot read them. A Strata profile
says its family and size (`--family qwen --size IQ2_XS`), which the catalog
checks against what Strata runs, and reserves the RAM the size keeps its experts
in (48 GB and the engine's 4 for IQ2_XS) unless it says `min_memory_gb`. Its
+35
View File
@@ -35,6 +35,7 @@ import (
"gitea.mynodes.xyz/ilker/llmgw/internal/modelplan"
"gitea.mynodes.xyz/ilker/llmgw/internal/notify"
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
"gitea.mynodes.xyz/ilker/llmgw/internal/strata"
)
// Registry reads a repository.
@@ -412,6 +413,40 @@ func (s *Server) run(j *Job) {
if req.TP > 0 && req.GPUsPerNode == 0 {
req.GPUsPerNode = req.TP
}
// A pack Strata fetches itself: no mirror; its repository is the source.
if _, size, ok := strata.ForFetchedRepo(repo.ID); ok {
plan, err := s.plan(ctx, repo, PlanRequest{Repo: repo.ID})
if err != nil {
s.fail(j, "registry", err)
return
}
set := hf.FileSet{Name: size.Name}
layout, err := chooseLayout(Request{Format: "gguf", Engine: "strata", GPUClass: req.GPUClass}, plan, true, set)
if err != nil {
s.fail(j, "registry", err)
return
}
name := req.Name
if name == "" {
name = hf.ModelName(repo.ID)
}
s.mu.Lock()
j.Model = name
s.mu.Unlock()
if !req.sized {
req.ContextLength = strataContext
}
s.step(j, "registry", Done, fmt.Sprintf("Strata's ready-made %s pack, revision %s; runs on %s",
size.Name, repo.ShortRevision(), layout[0].Label()))
s.step(j, "mirror", Done, fmt.Sprintf("not through the mirror: Strata fetches about %.0f GB onto the node itself", size.DownloadGB))
profiles := profilesFor(layout, req, 0, 0)
profiles[0].StartupTimeout = 4 * time.Hour
s.mu.Lock()
j.Request = req
s.mu.Unlock()
s.addAndStart(ctx, j, repo, set, name, repo.ID, profiles, hardwareExists(layout))
return
}
// The plan: which file suits the hardware, and how it spreads over it.
planCtx := req.CtxPool
if req.Format == "safetensors" {
+13
View File
@@ -98,6 +98,19 @@ func (s *Server) Plan(ctx context.Context, req PlanRequest) (Plan, error) {
}
func (s *Server) plan(ctx context.Context, repo hf.Repo, req PlanRequest) (Plan, error) {
// A pack Strata fetches itself: one way, Strata's, whatever its files.
if fam, size, ok := strata.ForFetchedRepo(repo.ID); ok {
art := modelplan.Artifact{Name: size.Name, Format: "strata", Bytes: int64(size.DownloadGB * 1e9), Quant: size.Name}
p := Plan{Repo: repo.ID, Revision: repo.Revision, Format: "strata", Arch: repo.Architecture, License: repo.License,
Context: strataContext, Servable: true, Note: "Strata fetches this pack onto the node itself on its first start"}
p.Options = modelplan.StrataOptions(fam.Name, []modelplan.Artifact{art}, s.deps.Pool())
pa := PlannedArtifact{Artifact: art}
if len(p.Options) > 0 {
pa.Best = &p.Options[0]
}
p.Artifacts = []PlannedArtifact{pa}
return p, nil
}
p := Plan{Repo: repo.ID, Revision: repo.Revision, Format: req.Format, Arch: repo.Architecture,
License: repo.License, Params: repo.Params, Context: req.Context, Gated: repo.Gated.Is(), Servable: true,
Projectors: Projectors(repo)}
@@ -35,3 +35,18 @@ func TestAStrataWayIsChosenAndMadeAProfile(t *testing.T) {
t.Errorf("%+v", p)
}
}
func TestAReadyMadePackIsPlannedOnStrataOnly(t *testing.T) {
s := New(Deps{Pool: func() []modelplan.PoolNode {
return []modelplan.PoolNode{{ID: "cuda01", Class: "6000-24g", GPUMemGB: 24, GPUs: 1, FreeGPUs: 1,
HostMemGB: 251, HostFreeGB: 200, Engines: []string{"strata"}}}
}})
p, err := s.plan(t.Context(), hf.Repo{ID: "Maximilian228/Qwen3.8-Flash-Next-NVIDIA-NVFP4-Strata", Revision: "abc",
Architecture: "strata-ple"}, PlanRequest{})
if err != nil {
t.Fatal(err)
}
if !p.Servable || len(p.Options) != 1 || p.Options[0].StrataFamily != "nvfp4" || p.Options[0].StrataSize != "NVFP4-NVIDIA" {
t.Errorf("%+v", p)
}
}
+68 -14
View File
@@ -14,9 +14,9 @@ import (
// Family is a model Strata runs, from one publisher's files.
type Family struct {
// Name is setup's --family.
Name string
Title string
// Name is setup's --family; Model the name Strata gives the model.
Name, Model string
Title string
// Repo is the Hugging Face repository its files are in; empty when
// Strata fetches them itself from several.
Repo string
@@ -42,18 +42,21 @@ type Size struct {
RAMGB float64
// NVIDIAOnly is a size whose kernels only NVIDIA cards have.
NVIDIAOnly bool
// Repo is a size's own repository, for a ready-made pack Strata
// fetches itself.
Repo string
}
// Families are the families Strata 0.2.7 runs.
var Families = []Family{
{Name: "qwen", Title: "Qwen3.8-Flash-Next", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF", Subdir: true},
{Name: "swift", Title: "Swift 1.5 (Qwen3.8-Flash-Next, shorter thinking)", Repo: "ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF"},
{Name: "coder", Title: "Qwen3.8-Flash-Next Coder", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF", Subdir: true},
{Name: "unsloth", Title: "Qwen3.8-Flash-Next (Unsloth)", Repo: "unsloth/Qwen3.8-Flash-Next-GGUF", Subdir: true},
{Name: "gyro", Title: "Qwen3.8-Flash-Next (Gyro)", Repo: "agentionai/Qwen3.8-Flash-Next-Gyro-GGUF"},
{Name: "nvfp4", Title: "Qwen3.8-Flash-Next (NVFP4)"},
{Name: "qwen36", Title: "Qwen3.6-35B-A3B", Repo: "unsloth/Qwen3.6-35B-A3B-MTP-GGUF"},
{Name: "deepseek41", Title: "DeepSeek V4.1-Flash", Repo: "ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF", Engine: true},
{Name: "qwen", Model: "qwen3.8-flash-next", Title: "Qwen3.8-Flash-Next", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF", Subdir: true},
{Name: "swift", Model: "swift-1.5", Title: "Swift 1.5 (Qwen3.8-Flash-Next, shorter thinking)", Repo: "ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF"},
{Name: "coder", Model: "qwen3.8-flash-next-coder", Title: "Qwen3.8-Flash-Next Coder", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF", Subdir: true},
{Name: "unsloth", Model: "qwen3.8-flash-next-unsloth", Title: "Qwen3.8-Flash-Next (Unsloth)", Repo: "unsloth/Qwen3.8-Flash-Next-GGUF", Subdir: true},
{Name: "gyro", Model: "qwen3.8-flash-next-gyro", Title: "Qwen3.8-Flash-Next (Gyro)", Repo: "agentionai/Qwen3.8-Flash-Next-Gyro-GGUF"},
{Name: "nvfp4", Model: "qwen3.8-flash-next-nvfp4", Title: "Qwen3.8-Flash-Next (NVFP4)"},
{Name: "qwen36", Model: "qwen3.6-35b-a3b", Title: "Qwen3.6-35B-A3B", Repo: "unsloth/Qwen3.6-35B-A3B-MTP-GGUF"},
{Name: "deepseek41", Model: "deepseek-v4.1-flash", Title: "DeepSeek V4.1-Flash", Repo: "ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF", Engine: true},
}
// Sizes are the sizes Strata 0.2.7 runs, and which families have each.
@@ -70,9 +73,12 @@ var Sizes = []Size{
{Name: "Q8_0", Families: []string{"unsloth", "qwen36"}, DownloadGB: 188.2, RAMGB: 136},
{Name: "Gyro-S", Families: []string{"gyro"}, DownloadGB: 58.5, RAMGB: 27.6},
{Name: "Gyro-M", Families: []string{"gyro"}, DownloadGB: 92.0, RAMGB: 35},
{Name: "NVFP4", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true},
{Name: "NVFP4-NVIDIA", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true},
{Name: "NVFP4-QAD", Families: []string{"nvfp4"}, DownloadGB: 102.2, DiskGB: 212.2, RAMGB: 80, NVIDIAOnly: true},
{Name: "NVFP4", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true,
Repo: "Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata"},
{Name: "NVFP4-NVIDIA", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true,
Repo: "Maximilian228/Qwen3.8-Flash-Next-NVIDIA-NVFP4-Strata"},
{Name: "NVFP4-QAD", Families: []string{"nvfp4"}, DownloadGB: 102.2, DiskGB: 212.2, RAMGB: 80, NVIDIAOnly: true,
Repo: "nvidia/Qwen3.8-Flash-Next-FP4-Local"},
{Name: "IST-DAS-1.75bpw", Families: []string{"deepseek41"}, DownloadGB: 239, RAMGB: 34},
{Name: "IST-DAS-2.00bpw", Families: []string{"deepseek41"}, DownloadGB: 257, RAMGB: 40},
{Name: "IST-DAS-2.25bpw", Families: []string{"deepseek41"}, DownloadGB: 274, RAMGB: 44},
@@ -172,3 +178,51 @@ func SizeForFile(family, file string) (Size, bool) {
}
return best, best.Name != ""
}
// FetchesItself is a family whose files Strata's setup fetches itself, each
// pinned to a commit and checked: ready-made packs, and DeepSeek's sets with
// an engine of their own. A model of it names its repository as the source.
func (f Family) FetchesItself() bool { return f.Name == "nvfp4" || f.Engine }
// RepoOf is the repository a size of a family is in.
func RepoOf(f Family, s Size) string {
if s.Repo != "" {
return s.Repo
}
return f.Repo
}
// ForFetchedRepo is the family and size a repository is, for one Strata
// fetches itself: a ready-made pack's own, or DeepSeek's (whose size the
// repository does not say, so the smallest).
func ForFetchedRepo(repo string) (Family, Size, bool) {
for _, f := range Families {
if !f.FetchesItself() {
continue
}
for _, s := range SizesOf(f.Name) {
if strings.EqualFold(RepoOf(f, s), repo) {
return f, s, true
}
}
}
return Family{}, Size{}, false
}
// Fetchable says whether a size's files can be fetched by llmgw from the
// family's repository: not where Strata fetches them itself (NVFP4's packs,
// DeepSeek's sets with an engine of their own).
func (f Family) Fetchable() bool { return f.Repo != "" && !f.Engine }
// FilesOf are the weights of a size among a repository's files: its GGUF
// shards, not the picture reader.
func FilesOf(size string, paths []string) []string {
var out []string
for _, p := range paths {
u := strings.ToUpper(p)
if strings.HasSuffix(u, ".GGUF") && !strings.Contains(u, "MMPROJ") && strings.Contains(u, strings.ToUpper(size)) {
out = append(out, p)
}
}
return out
}
+31 -4
View File
@@ -24,6 +24,7 @@ import (
"gitea.mynodes.xyz/ilker/llmgw/internal/safedial"
"gitea.mynodes.xyz/ilker/llmgw/internal/sourcecheck"
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
"gitea.mynodes.xyz/ilker/llmgw/internal/strata"
)
// MirrorAdmin is the mirror's management API as this page needs it.
@@ -169,6 +170,10 @@ type adminMirrorData struct {
// the serve action is then not offered rather than offered and refused.
HasCatalog bool
// Strata is the Strata tab: the models it runs, and where they would
// run here.
Strata *strataView
// The explorer.
HasRegistry bool
Query string
@@ -256,6 +261,14 @@ func (p *AccountPages) RegisterAdminMirror(mux *http.ServeMux, d MirrorDeps) {
render(w, r, "mirror_repo", d)
}))
mux.HandleFunc("POST /admin/mirror/strata", p.mirrorAction(m,
func(ctx context.Context, admin store.User, r *http.Request) (string, error) {
if r.FormValue("action") == "serve" {
return p.serveStrata(ctx, langOf(r), admin.Email, r.FormValue("family"), r.FormValue("size"))
}
return p.fetchStrata(ctx, langOf(r), m, r.FormValue("family"), r.FormValue("size"))
}))
mux.HandleFunc("POST /admin/mirror/fetch", p.mirrorAction(m,
func(ctx context.Context, admin store.User, r *http.Request) (string, error) {
// One form posts one source or fifteen. A sharded model is one
@@ -462,6 +475,9 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
if d.Tab == tabExplore {
p.explore(r, &d)
}
if d.Tab == tabStrata {
d.Strata = p.strataView(l)
}
got, err := m.List(r.Context())
if err != nil {
@@ -618,6 +634,7 @@ const (
tabHeld = "held"
tabExplore = "explore"
tabFetch = "fetch"
tabStrata = "strata"
)
// mirrorTab is one of the page's tabs.
@@ -636,7 +653,7 @@ type mirrorTab struct {
// shown, where its effect is, unless it was refused: then the form it came
// from, to try again.
func mirrorTabFor(r *http.Request, errMsg string) string {
valid := func(t string) bool { return t == tabHeld || t == tabExplore || t == tabFetch }
valid := func(t string) bool { return t == tabHeld || t == tabExplore || t == tabFetch || t == tabStrata }
if r.Method == http.MethodPost {
if t := r.PostFormValue("tab"); errMsg != "" && valid(t) {
return t
@@ -673,6 +690,10 @@ func mirrorTabs(l *Lang, d adminMirrorData) []mirrorTab {
tabs = append(tabs, mirrorTab{Label: l.T("mirror.tab.explore"),
Href: "/admin/mirror?tab=explore", Active: d.Tab == tabExplore})
}
if d.HasRegistry {
tabs = append(tabs, mirrorTab{Label: l.T("mirror.tab.strata"),
Href: "/admin/mirror?tab=strata", Active: d.Tab == tabStrata})
}
return append(tabs, mirrorTab{Label: l.T("mirror.tab.fetch"),
Href: "/admin/mirror?tab=fetch", Active: d.Tab == tabFetch})
}
@@ -823,9 +844,15 @@ func (p *AccountPages) openRepo(r *http.Request, d *adminMirrorData, repo string
Version: got.ShortRevision(),
}
rs.Verdict = p.verdictFor(langOf(r), got.Architecture, set.Name)
// A family Strata runs: one card and the experts in RAM, whether
// or not llama.cpp could load it.
if layout, ok := fit.strataLayout(langOf(r), got.ID, set.Name, set.Bytes); ok {
// A pack made for Strata: served whole, from the Strata tab, when
// Strata knows it; named for what it is when it does not.
if fam, size, ok := strata.ForFetchedRepo(got.ID); ok {
rs.Verdict = gguf.Verdict{Checked: true, Reason: langOf(r).T("strata.pack_files", fam.Title, size.Name)}
} else if got.Architecture == "strata-ple" {
rs.Verdict = gguf.Verdict{Checked: true, Reason: langOf(r).T("strata.unknown_pack")}
} else if layout, ok := fit.strataLayout(langOf(r), got.ID, set.Name, set.Bytes); ok {
// A family Strata runs: one card and the experts in RAM,
// whether or not llama.cpp could load it.
rs.Verdict, rs.GPULayout = gguf.Verdict{Checked: true, Servable: true}, layout
} else if fit != nil && rs.Verdict.Servable {
if v, ok := fit.holds(langOf(r), set.Name, set.Bytes, got.Params); !ok {
+153
View File
@@ -0,0 +1,153 @@
package web
import (
"context"
"errors"
"slices"
"strconv"
"strings"
"time"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
"gitea.mynodes.xyz/ilker/llmgw/internal/modelplan"
"gitea.mynodes.xyz/ilker/llmgw/internal/strata"
)
// The mirror page's Strata tab: the models Strata runs, each size with what
// it downloads and keeps in RAM and where it would run here, and a button
// that fetches a size's files into the mirror, where the version's serve
// form then serves it on Strata.
type strataView struct {
// Nodes are the nodes that have Strata installed.
Nodes []string
Families []strataFamilyRow
}
type strataFamilyRow struct {
Name, Title, Repo string
// Fetchable is a family whose files llmgw fetches into the mirror;
// FetchesItself one whose files Strata fetches onto the node, served
// from here at once.
Fetchable, FetchesItself bool
Sizes []strataSizeRow
}
type strataSizeRow struct {
Size string
DownloadGB, RAMGB string
Runs string
Fits, NVIDIAOnly bool
}
func (p *AccountPages) strataView(l *Lang) *strataView {
v := &strataView{}
var pool []modelplan.PoolNode
if fit := p.fitOf(); fit != nil {
pool = fit.pool
}
for _, n := range pool {
if slices.Contains(n.Engines, "strata") {
v.Nodes = append(v.Nodes, n.ID)
}
}
for _, f := range strata.Families {
row := strataFamilyRow{Name: f.Name, Title: f.Title, Repo: f.Repo, Fetchable: f.Fetchable(),
FetchesItself: f.FetchesItself()}
for _, s := range strata.SizesOf(f.Name) {
sr := strataSizeRow{Size: s.Name, DownloadGB: l.Num(s.DownloadGB, 0), RAMGB: l.Int(int64(s.MemoryGB())),
NVIDIAOnly: s.NVIDIAOnly}
opts := modelplan.StrataOptions(f.Name, []modelplan.Artifact{{Name: s.Name + ".gguf", Format: "gguf",
Bytes: int64(s.DownloadGB * 1e9)}}, pool)
switch {
case len(opts) > 0:
sr.Fits, sr.Runs = true, l.T("explore.strata", opts[0].Class, l.Num(opts[0].NeedGB, 0))
case len(v.Nodes) == 0:
sr.Runs = l.T("strata.no_node")
default:
sr.Runs = l.T("strata.no_room", s.MemoryGB())
}
row.Sizes = append(row.Sizes, sr)
}
v.Families = append(v.Families, row)
}
return v
}
// serveStrata adds a model of a family Strata fetches itself: its source
// the repository, its replica on a card of the class the plan found.
func (p *AccountPages) serveStrata(ctx context.Context, l *Lang, by, family, size string) (string, error) {
fam, ok := strata.FamilyNamed(family)
s, err := strata.SizeOf(family, size)
if !ok || err != nil {
return "", errors.New(l.T("strata.err.unknown", family, size))
}
if !fam.FetchesItself() || p.catalog == nil {
return "", errors.New(l.T("strata.err.not_servable", fam.Title))
}
var pool []modelplan.PoolNode
if fit := p.fitOf(); fit != nil {
pool = fit.pool
}
opts := modelplan.StrataOptions(fam.Name, []modelplan.Artifact{{Name: s.Name + ".gguf", Format: "gguf"}}, pool)
if len(opts) == 0 {
return "", errors.New(l.T("strata.no_room", s.MemoryGB()))
}
name := fam.Model + "-" + strings.ToLower(s.Name)
disk := s.DownloadGB
if s.DiskGB > disk {
disk = s.DiskGB
}
m := catalog.Model{Name: name, Source: strata.RepoOf(fam, s), RoutingProfile: "chat", ContextLength: strataContext,
Scaling: catalog.Scaling{Min: 1, Max: 1},
Profiles: []catalog.Profile{{Name: "strata-" + opts[0].Class, Engine: "strata", GPUClass: opts[0].Class,
GPUsPerNode: 1, Weight: 1, Priority: 100, MinMemoryGB: s.MemoryGB(), MinDiskGB: int(disk) + 10,
Args: []string{"--family", fam.Name, "--size", s.Name, "--context", strconv.Itoa(strataContext), "--vision", "no"},
// Its files come onto the node on its first start, tens of
// gigabytes from the internet, before it is prepared.
StartupTimeout: 4 * time.Hour}}}
if err := p.catalog.Add(ctx, m, by); err != nil {
return "", err
}
return l.T("strata.msg.served", name, l.Num(disk, 0)), nil
}
// strataContext is the context a Strata model is served with from here.
const strataContext = 32768
// fetchStrata fetches a Strata size's files from its family's repository
// into the mirror, each checked against the sha256 the registry published.
func (p *AccountPages) fetchStrata(ctx context.Context, l *Lang, m MirrorAdmin, family, size string) (string, error) {
fam, ok := strata.FamilyNamed(family)
if _, err := strata.SizeOf(family, size); !ok || err != nil {
return "", errors.New(l.T("strata.err.unknown", family, size))
}
if !fam.Fetchable() || p.registry == nil {
return "", errors.New(l.T("strata.err.not_fetchable", fam.Title))
}
repo, err := p.registry.Repo(ctx, fam.Repo)
if err != nil {
return "", err
}
var paths []string
sums := map[string]string{}
for _, f := range repo.Files {
paths = append(paths, f.Path)
sums[f.Path] = f.SHA256
}
files := strata.FilesOf(size, paths)
if len(files) == 0 {
return "", errors.New(l.T("strata.err.no_files", fam.Repo, size))
}
model := fam.Model + "-" + strings.ToLower(size)
version := repo.ShortRevision()
for i, path := range files {
req := mirror.FetchRequest{Source: repo.DownloadURL("", path), Model: model, Version: version,
File: mirror.LastPathElement(path), SHA256: sums[path]}
if _, err := m.Fetch(ctx, req); err != nil && !strings.Contains(err.Error(), "already") {
return "", errors.New(l.T("mirror.err.partly_started", i, len(files), req.File, l.Err(err)))
}
}
return l.T("mirror.msg.started", model, version, l.N("mirror.files", int64(len(files)))), nil
}
+88
View File
@@ -0,0 +1,88 @@
package web
import (
"net/url"
"strings"
"testing"
"time"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
"gitea.mynodes.xyz/ilker/llmgw/internal/hf"
)
// strataFleet is one node with an RTX 6000, 251 GB and Strata installed.
type strataFleet struct{ fakeFleet }
func (strataFleet) Nodes() []adminapi.NodeView {
return []adminapi.NodeView{{ID: "cuda01", State: "ready", GPUTotal: 1, GPUFree: 1, MemoryGB: 251,
Classes: []string{"6000-24g"}, Engines: []string{"llamacpp", "strata"}}}
}
func TestTheStrataTabListsWhatRunsHereAndFetchesASize(t *testing.T) {
reg := &fakeRegistry{repo: hf.Repo{ID: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF",
Revision: "0123456789abcdef0123456789abcdef01234567", Files: []hf.File{
{Path: "IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf", SHA256: "aa"},
{Path: "IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf", SHA256: "bb"},
{Path: "IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf"},
{Path: "mmproj-Qwen3.8-Flash-Next-BF16.gguf"},
}}}
m := &fakeMirror{}
mux := mirrorPageDeps(t, MirrorDeps{Mirror: m, Registry: reg, Fleet: strataFleet{}}, true)
body := mirrorGet(t, mux, "/admin/mirror?tab=strata").Body.String()
for _, want := range []string{"Strata is installed on: cuda01", "Qwen3.8-Flash-Next", "IQ2_XS",
"1× 6000-24g with Strata, 52 GB of RAM", `name="size" value="IQ2_XS"`, "Strata fetches these onto the node itself"} {
if !strings.Contains(body, want) {
t.Errorf("the tab lacks %q", want)
}
}
// Unsloth's Q8_0 keeps 140 GB: it runs here; nothing past the node's RAM does.
if strings.Contains(body, `value="UD-Q8_K_XL"`) {
t.Error("offered DeepSeek's 306 GB size, which no node holds, and which Strata fetches itself")
}
mirrorPost(t, mux, "/admin/mirror/strata", url.Values{"csrf": {"tok"}, "family": {"qwen"}, "size": {"IQ2_XS"}})
if len(m.all) != 2 {
t.Fatalf("fetched %+v", m.all)
}
f := m.all[0]
if f.Model != "qwen3.8-flash-next-iq2_xs" || f.Version != "01234567" || f.SHA256 != "aa" ||
!strings.Contains(f.Source, "/resolve/0123456789abcdef0123456789abcdef01234567/IQ2_XS/") {
t.Errorf("%+v", f)
}
}
// A ready-made pack is served at once, its repository the source, Strata
// fetching it onto the node.
func TestAReadyMadeStrataPackIsServedFromTheTab(t *testing.T) {
cat := &fakeCatalog{}
mux := mirrorPageDeps(t, MirrorDeps{Mirror: &fakeMirror{}, Registry: &fakeRegistry{}, Catalog: cat, Fleet: strataFleet{}}, true)
if body := mirrorGet(t, mux, "/admin/mirror?tab=strata").Body.String(); !strings.Contains(body, `name="action" value="serve"`) {
t.Fatal("no serve button for a pack")
}
mirrorPost(t, mux, "/admin/mirror/strata", url.Values{"csrf": {"tok"}, "action": {"serve"}, "family": {"nvfp4"}, "size": {"NVFP4"}})
if len(cat.added) != 1 {
t.Fatalf("added %+v", cat.added)
}
m := cat.added[0]
p := m.Profiles[0]
if m.Name != "qwen3.8-flash-next-nvfp4-nvfp4" || m.Source != "Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata" ||
p.Engine != "strata" || p.GPUClass != "6000-24g" || p.MinMemoryGB != 84 || p.StartupTimeout < time.Hour ||
strings.Join(p.Args, " ") != "--family nvfp4 --size NVFP4 --context 32768 --vision no" {
t.Errorf("%+v", m)
}
}
func TestAStrataPacksFilesAreNamedForWhatTheyAre(t *testing.T) {
for repo, want := range map[string]string{
"Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata": "ready-made Qwen3.8-Flash-Next (NVFP4) NVFP4 pack",
"trymirai/Qwen3.8-Flash-Next-S-strata-experimental": "Strata 0.2.7 does not know",
} {
reg := &fakeRegistry{repo: hf.Repo{ID: repo, Architecture: "strata-ple", Revision: "abc",
Files: []hf.File{{Path: "ple-fp8.gguf", Bytes: 51e9}}}}
mux := mirrorPageDeps(t, MirrorDeps{Mirror: &fakeMirror{}, Registry: reg, Fleet: strataFleet{}}, true)
if body := mirrorGet(t, mux, "/admin/mirror?repo="+repo).Body.String(); !strings.Contains(body, want) {
t.Errorf("%s: no %q", repo, want)
}
}
}
+22 -1
View File
@@ -257,5 +257,26 @@
"mirror.err.wrong_arch": "that adapter is for %s models and %s is %s",
"mirror.msg.adapter_added": "%[1]s: adapter %[2]s added; ask for it by that name once the replicas are restarted (llmgw ctl restart --model %[1]s)",
"mirror.err.adapter_op": "add or remove an adapter",
"explore.strata": "1× %s with Strata, %s GB of RAM"
"explore.strata": "1× %s with Strata, %s GB of RAM",
"mirror.tab.strata": "Strata",
"strata.intro": "Strata runs these mixture-of-experts models with the experts in a node's RAM and the rest on one card of 12 GB or more: a model that fits no card, served from one. Fetch a size into the mirror; its serve form on the mirror's tab then serves it on Strata. The first start prepares it on the node, which takes minutes.",
"strata.nodes": "Strata is installed on: %s",
"strata.none": "No node has Strata installed. Name it among the cluster's engines (<code>llmgw ctl config set engines.strata 0.2.7</code>) and the nodes with an NVIDIA card install it.",
"strata.self": "Strata fetches these onto the node itself, each file pinned and checked, on the model's first start: served from here, they do not go through the mirror.",
"strata.col.size": "Size",
"strata.col.download": "Download",
"strata.col.ram": "RAM it keeps",
"strata.col.runs": "Here",
"strata.fetch": "Fetch into the mirror",
"strata.nvidia": "NVIDIA only",
"strata.no_node": "no node has Strata",
"strata.no_room": "no node with Strata has %d GB of RAM and a card of 12 GB",
"strata.err.unknown": "Strata runs no %s %s",
"strata.err.not_fetchable": "Strata fetches %s itself; serve it with a definition that names its family and size.",
"strata.err.no_files": "%s has no files of size %s",
"strata.serve": "Serve",
"strata.err.not_servable": "%s is fetched into the mirror first, then served from its version",
"strata.msg.served": "%s added: its first start fetches about %s GB onto the node and prepares it, which can take hours on a slow link; Replicas shows how far it is.",
"strata.pack_files": "part of Strata's ready-made %s %s pack: served whole from the mirror's Strata tab, Strata fetching it onto the node",
"strata.unknown_pack": "a pack made for Strata that Strata 0.2.7 does not know: its setup refuses it, and llama.cpp cannot read it"
}
+22 -1
View File
@@ -250,5 +250,26 @@
"mirror.err.wrong_arch": "bu adaptör %s modelleri için; %s ise %s",
"mirror.msg.adapter_added": "%[1]s: %[2]s adaptörü eklendi; replikalar yeniden başlatılınca bu adla isteyin (llmgw ctl restart --model %[1]s)",
"mirror.err.adapter_op": "bir adaptör ekleyin ya da kaldırın",
"explore.strata": "Strata ile 1× %s, %s GB RAM"
"explore.strata": "Strata ile 1× %s, %s GB RAM",
"mirror.tab.strata": "Strata",
"strata.intro": "Strata bu uzman karışımı (MoE) modelleri, uzmanları node'un RAM'inde, gerisini 12 GB ya da daha büyük tek bir kartta tutarak çalıştırır: hiçbir karta sığmayan bir model tek karttan sunulur. Bir boyutu aynaya indirin; aynadakiler sekmesindeki sunma formu onu Strata ile sunar. İlk açılışta model node üzerinde hazırlanır; bu birkaç dakika sürer.",
"strata.nodes": "Strata kurulu node'lar: %s",
"strata.none": "Hiçbir node'da Strata kurulu değil. Kümenin motorları arasında adını verin (<code>llmgw ctl config set engines.strata 0.2.7</code>); NVIDIA kartı olan node'lar onu kurar.",
"strata.self": "Bunları Strata, modelin ilk açılışında her dosyayı sabitleyip denetleyerek node'a kendisi indirir: buradan sunulur, aynadan geçmez.",
"strata.col.size": "Boyut",
"strata.col.download": "İndirme",
"strata.col.ram": "Ayırdığı RAM",
"strata.col.runs": "Burada",
"strata.fetch": "Aynaya indir",
"strata.nvidia": "Yalnız NVIDIA",
"strata.no_node": "Strata hiçbir node'da yok",
"strata.no_room": "Strata kurulu hiçbir node'da %d GB RAM ve 12 GB'lık kart yok",
"strata.err.unknown": "Strata %s %s çalıştırmıyor",
"strata.err.not_fetchable": "%s dosyalarını Strata kendisi indirir; ailesini ve boyutunu veren bir tanımla sunun.",
"strata.err.no_files": "%s içinde %s boyutunda dosya yok",
"strata.serve": "Sun",
"strata.err.not_servable": "%s önce aynaya indirilir, sonra sürümünden sunulur",
"strata.msg.served": "%s eklendi: ilk açılışında node'a yaklaşık %s GB indirip hazırlar; yavaş bağlantıda saatler sürebilir. İlerleyişi Replikalar gösterir.",
"strata.pack_files": "Strata'nın hazır %s %s paketinin parçası: aynanın Strata sekmesinden bütün olarak sunulur, Strata onu node'a kendisi indirir",
"strata.unknown_pack": "Strata için hazırlanmış ama Strata 0.2.7'nin tanımadığı bir paket: kurulumu onu reddeder, llama.cpp de okuyamaz"
}
+34
View File
@@ -424,6 +424,40 @@ document.querySelector("table.explore").addEventListener("submit", async functio
{{if .Repo}}{{template "mirror_repo" .}}{{end}}
{{end}}
{{if and (eq .Tab "strata") .Strata}}
<p class="note">{{t "strata.intro"}}</p>
{{if .Strata.Nodes}}<p class="sub">{{t "strata.nodes" (join .Strata.Nodes)}}</p>
{{else}}<p class="note error-row">{{th "strata.none"}}</p>{{end}}
{{range .Strata.Families}}
<h3>{{.Title}} <span class="opt"><code>--family {{.Name}}</code>{{if .Repo}} · <a href="/admin/mirror?repo={{.Repo}}">{{.Repo}}</a>{{end}}</span></h3>
{{if .FetchesItself}}<p class="sub">{{t "strata.self"}}</p>{{end}}
<table class="rows">
<tr><th>{{t "strata.col.size"}}</th><th>{{t "strata.col.download"}}</th><th>{{t "strata.col.ram"}}</th><th>{{t "strata.col.runs"}}</th><th></th></tr>
{{$fam := .}}
{{range .Sizes}}
<tr>
<td><code>{{.Size}}</code>{{if .NVIDIAOnly}} <span class="opt">{{t "strata.nvidia"}}</span>{{end}}</td>
<td>{{.DownloadGB}} GB</td>
<td>{{.RAMGB}} GB</td>
<td>{{if .Fits}}{{.Runs}}{{else}}<span class="opt">{{.Runs}}</span>{{end}}</td>
<td>{{if and $fam.FetchesItself .Fits}}<form method="post" action="/admin/mirror/strata" class="inline">
<input type="hidden" name="csrf" value="{{$.CSRF}}">
<input type="hidden" name="tab" value="strata">
<input type="hidden" name="action" value="serve">
<input type="hidden" name="family" value="{{$fam.Name}}">
<input type="hidden" name="size" value="{{.Size}}">
<button type="submit">{{t "strata.serve"}}</button></form>{{end}}{{if and $fam.Fetchable .Fits}}<form method="post" action="/admin/mirror/strata" class="inline">
<input type="hidden" name="csrf" value="{{$.CSRF}}">
<input type="hidden" name="tab" value="strata">
<input type="hidden" name="family" value="{{$fam.Name}}">
<input type="hidden" name="size" value="{{.Size}}">
<button type="submit">{{t "strata.fetch"}}</button></form>{{end}}</td>
</tr>
{{end}}
</table>
{{end}}
{{end}}
{{if eq .Tab "fetch"}}
<p class="note">{{t "mirror.fetch.intro"}}</p>
<form method="post" action="/admin/mirror/fetch">