Compare commits

...
5 Commits
Author SHA1 Message Date
Ilker ManapandClaude Opus 5.5 a5fefbdaaa v0.16.1: changelog and README
CI / test (push) Successful in 12m33s
Release / build (push) Successful in 6m32s
Release / notes (push) Successful in 4s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-30 13:14:25 +01:00
Ilker ManapandClaude Opus 5.5 2eb96f4872 The plan says why Strata's DeepSeek does not run here
CI / test (push) Successful in 11m38s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-30 13:09:09 +01:00
Ilker ManapandClaude Opus 5.5 f41c96d30f Strata's DeepSeek not offered on cards its engine was not built for, and refused there by the agent
CI / test (push) Successful in 9m42s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-30 13:00:21 +01:00
Ilker ManapandClaude Opus 5.5 104b60fbf8 A repository's files read from every directory, not only its top
CI / test (push) Successful in 11m40s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-30 12:16:11 +01:00
Ilker ManapandClaude Opus 5.5 845de8eef4 The mirror page's Strata tab; Strata's ready-made packs served with Strata fetching them, strata-ple named
CI / test (push) Successful in 10m35s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-30 12:06:46 +01:00
22 changed files with 645 additions and 24 deletions

No files matched your search

+6
View File
@@ -1889,9 +1889,15 @@ served only from a definition written by hand. In order:
- N7. The web serve page, the explorer's fit, the console's serve form: the same choice. Done 2026-10-30.
- N8. `--context`/`--parallel` tied to `context_length`/`max_output`; `ctl model context` takes Strata.
- N7b. The mirror page's Strata tab: every family and size, fetch (GGUF) or serve (packs Strata fetches itself:
NVFP4, DeepSeek V4.1). Done 2026-10-30. Community `strata-ple` packs Strata 0.2.7 does not list stay unservable;
running them would mean starting the engine past Strata's setup (`--pack`, `--native`, `--ple-gguf`).
**C. Operating it.**
- N9. `ctl engine-check strata`: the flags of setup.py and server.py, which drift between versions.
- N10. `ctl engine-build NODE strata cuda12|cuda13`, as vLLM has.
- N11a. Strata's per-replica state directories (engine-home/strata/<replica>) are left behind when a replica stops;
remove them with it.
- N11. Disk for the prepared pack and draft layer (and DeepSeek V4.1's 240–280 GB), cleaned with the weights; a
first start's preparation within the startup timeout.
- N12. docs/install.md: Strata in the engines table, a section with a profile, its memory and first-run time;
+27 -1
View File
@@ -20,12 +20,38 @@ is only released once its **exit criterion** is met, not when its code lands.
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider; v0.14.9: sign-in set up from the console, a node's console that shows the machine) |
| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released (v0.15.1: Strata, a 125B model of experts on one card; a second mirror kept in step; pickled weights refused; a served model's context changed; v0.15.2: a served model moved to more cards; v0.15.3: a model stopped and kept, or removed with its files) |
| v0.16.0 | — | Weights pinned to a commit and checked against what Hugging Face published; documents embedded by the gateway; Strata offered where models are chosen; cluster upgrades that say how far they are | released |
| v0.16.0 | — | Weights pinned to a commit and checked against what Hugging Face published; documents embedded by the gateway; Strata offered where models are chosen; cluster upgrades that say how far they are | released (v0.16.1: the mirror's Strata tab, Strata's ready-made packs) |
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
Until v1.0.0 the minor version is the breaking one: configuration formats and Go
APIs change between minors without a deprecation period.
## v0.16.1 — 2026-10-30
The mirror page's Strata tab, Strata's ready-made packs served with Strata fetching them, and two fixes found
serving Strata's models: repositories that keep each size in a directory, and DeepSeek on cards its engine does not
run on.
**Upgrading.**
- No migrations. The control planes carry the tab, the plan and the serving; the agents carry the refusal of a
Strata profile its card cannot run (`llmgw ctl upgrade v0.16.1`). Mirrors need nothing new.
### Added
- **The mirror page's Strata tab.** Every family and size Strata runs, with its download, the RAM it keeps and where
it would run here; a GGUF size fetched into the mirror in one press (checked against Hugging Face's checksums),
and the packs Strata fetches itself — NVFP4, DeepSeek V4.1 — served at once, their repository the source.
`ctl model plan` and `ctl model serve` take those repositories too, and the explorer names a `strata-ple`
repository for what it is: part of a pack Strata knows, or one Strata 0.2.7 does not.
### Fixed
- **A repository that keeps each quantisation in a directory of its own** (ISTA-DASLab's `IQ2_XS/`, `IQ3_S/`) listed
no weights: its file tree is now read recursively, by the explorer, the plan and `ctl model serve`.
- **DeepSeek V4.1 offered on a card its engine was not built for.** Strata's DeepSeek engine runs on RTX 30 (compute
8.6) and newer; on a Turing card the tab and the plan offered it, and Strata's setup refused it once placed. Now
neither offers it there, the tab says why, and the agent refuses such a profile before starting it.
## v0.16.0 — 2026-10-30
Weights pinned to the commit they were added at and checked, by the nodes and the mirror, against the sha256
+1 -1
View File
@@ -42,7 +42,7 @@ Releases track the roadmap phases; see [CHANGELOG.md](CHANGELOG.md).
| v0.13.0 | — | The whole system stopped and started from one command; Qwen3Guard judges, on a GPU and CPUs together | released |
| v0.14.0 | — | Knowledge bases: documents from people's own machines, kept apart per tenant, answered from | released (v0.14.1: adapters, the tree over the API, the firewall on every machine; v0.14.2: pictures from ComfyUI; v0.14.3: 3D models as jobs; v0.14.4: any ComfyUI workflow as a job, presets, licences; v0.14.5: picture and 3D models that add themselves; v0.14.6: setup that explains every part and lets it be chosen, the gateway on all three HA machines; v0.14.7: the judge sized for its machine; v0.14.8: sign-in with Active Directory, several keys for one outside provider; v0.14.9: sign-in set up from the console, a node's console that shows the machine) |
| v0.15.0 | — | The company's MCP tools and VS Code through the gateway; guardrails that ask outside services and put redacted values back; answers to similar questions | released (v0.15.1: Strata, a 125B model of experts on one card; a second mirror kept in step; pickled weights refused; a served model's context changed; v0.15.2: a served model moved to more cards; v0.15.3: a model stopped and kept, or removed with its files) |
| v0.16.0 | — | Weights pinned to a commit and checked against what Hugging Face published; documents embedded by the gateway; Strata offered where models are chosen; cluster upgrades that say how far they are | released |
| v0.16.0 | — | Weights pinned to a commit and checked against what Hugging Face published; documents embedded by the gateway; Strata offered where models are chosen; cluster upgrades that say how far they are | released (v0.16.1: the mirror's Strata tab, Strata's ready-made packs) |
| v1.0.0 | 5 | What phase 5 has left: precise KV routing proven on vLLM, the block cost picker, defragmentation across NVLink domains, prefill/decode disaggregation | planned |
As of v0.6.0 a cluster runs unattended: agents report their hardware, the
+18 -1
View File
@@ -761,7 +761,24 @@ llmgw ctl model serve ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF \
```
The Hugging Face explorer shows the same ("1× … with Strata, N GB of RAM"),
and the mirror's serve form serves such a version on Strata. A Strata profile
and the mirror's serve form serves such a version on Strata.
The mirror page's **Strata** tab lists every family and size Strata runs, with
what each downloads and keeps in RAM and where it would run here:
- **Fetch into the mirror**, for the GGUF families: the size's files, checked
against Hugging Face's checksums; the version's serve form then serves it on
Strata.
- **Serve**, for the packs Strata fetches itself — the NVFP4 packs
(`Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata` and its NVIDIA
sibling, NVIDIA's FP4-Local) and DeepSeek V4.1: the model is added with the
repository as its source, and its first start fetches tens of gigabytes onto
the node, each file pinned and checked by Strata, before preparing it.
`ctl model serve` does the same for these repositories.
Repositories of architecture `strata-ple` that Strata 0.2.7 does not list —
community packs such as `…-Q8_0-Strata` — are named as such in the explorer:
Strata's setup refuses them, and llama.cpp cannot read them. A Strata profile
says its family and size (`--family qwen --size IQ2_XS`), which the catalog
checks against what Strata runs, and reserves the RAM the size keeps its experts
in (48 GB and the engine's 4 for IQ2_XS) unless it says `min_memory_gb`. Its
+9
View File
@@ -9,6 +9,8 @@ import (
"regexp"
"strconv"
"sync/atomic"
"gitea.mynodes.xyz/ilker/llmgw/internal/strata"
)
// StrataConfig locates a Strata installation.
@@ -133,6 +135,13 @@ func (*Strata) Validate(p Profile, gpus []GPUInfo) error {
if err := requireComputeCapability(p, gpus, 6.0, "strata"); err != nil {
return err
}
// A family whose engine needs more: said now, rather than by Strata's
// setup after the replica was placed here.
if fam, ok := strata.FamilyNamed(strata.ArgValue(p.Args, "--family")); ok && fam.MinCC > 0 {
if err := requireComputeCapability(p, gpus, fam.MinCC, "strata's "+fam.Title+" engine"); err != nil {
return err
}
}
return validateCommon(p, gpus)
}
+13
View File
@@ -98,3 +98,16 @@ func TestAStrataNamedByHandIsReportedWithItsVersion(t *testing.T) {
t.Error("reported without its engine")
}
}
func TestStrataRefusesDeepSeekOnACardItsEngineWasNotBuiltFor(t *testing.T) {
s := &Strata{}
p := Profile{Name: "p", Engine: "strata", ModelPath: "/m", TP: 1, GPUsPerNode: 1, Args: []string{"--family", "deepseek41", "--size", "IST-DAS-1.75bpw"}}
gpus := []GPUInfo{{Index: 0, Name: "Quadro RTX 6000", ComputeCapability: "7.5"}}
if err := s.Validate(p, gpus); err == nil || !strings.Contains(err.Error(), "8.6") {
t.Errorf("DeepSeek on 7.5: %v", err)
}
p.Args = []string{"--family", "qwen", "--size", "IQ2_XS"}
if err := s.Validate(p, gpus); err != nil {
t.Errorf("Qwen on 7.5: %v", err)
}
}
+5
View File
@@ -1954,6 +1954,11 @@ func PoolNodesOf(nodes []NodeView) []modelplan.PoolNode {
if len(n.Fabrics) > 0 {
p.Fabric = n.Fabrics[0]
}
for _, c := range n.ComputeCapabilities {
if v, err := strconv.ParseFloat(c, 64); err == nil && v > p.CC {
p.CC = v
}
}
if l := n.Load; l != nil && l.MemTotal > 0 {
p.HostFreeGB = int((l.MemTotal - l.MemUsed) >> 30)
}
+35
View File
@@ -35,6 +35,7 @@ import (
"gitea.mynodes.xyz/ilker/llmgw/internal/modelplan"
"gitea.mynodes.xyz/ilker/llmgw/internal/notify"
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
"gitea.mynodes.xyz/ilker/llmgw/internal/strata"
)
// Registry reads a repository.
@@ -412,6 +413,40 @@ func (s *Server) run(j *Job) {
if req.TP > 0 && req.GPUsPerNode == 0 {
req.GPUsPerNode = req.TP
}
// A pack Strata fetches itself: no mirror; its repository is the source.
if _, size, ok := strata.ForFetchedRepo(repo.ID); ok {
plan, err := s.plan(ctx, repo, PlanRequest{Repo: repo.ID})
if err != nil {
s.fail(j, "registry", err)
return
}
set := hf.FileSet{Name: size.Name}
layout, err := chooseLayout(Request{Format: "gguf", Engine: "strata", GPUClass: req.GPUClass}, plan, true, set)
if err != nil {
s.fail(j, "registry", err)
return
}
name := req.Name
if name == "" {
name = hf.ModelName(repo.ID)
}
s.mu.Lock()
j.Model = name
s.mu.Unlock()
if !req.sized {
req.ContextLength = strataContext
}
s.step(j, "registry", Done, fmt.Sprintf("Strata's ready-made %s pack, revision %s; runs on %s",
size.Name, repo.ShortRevision(), layout[0].Label()))
s.step(j, "mirror", Done, fmt.Sprintf("not through the mirror: Strata fetches about %.0f GB onto the node itself", size.DownloadGB))
profiles := profilesFor(layout, req, 0, 0)
profiles[0].StartupTimeout = 4 * time.Hour
s.mu.Lock()
j.Request = req
s.mu.Unlock()
s.addAndStart(ctx, j, repo, set, name, repo.ID, profiles, hardwareExists(layout))
return
}
// The plan: which file suits the hardware, and how it spreads over it.
planCtx := req.CtxPool
if req.Format == "safetensors" {
+17
View File
@@ -98,6 +98,23 @@ func (s *Server) Plan(ctx context.Context, req PlanRequest) (Plan, error) {
}
func (s *Server) plan(ctx context.Context, repo hf.Repo, req PlanRequest) (Plan, error) {
// A pack Strata fetches itself: one way, Strata's, whatever its files.
if fam, size, ok := strata.ForFetchedRepo(repo.ID); ok {
art := modelplan.Artifact{Name: size.Name, Format: "strata", Bytes: int64(size.DownloadGB * 1e9), Quant: size.Name}
p := Plan{Repo: repo.ID, Revision: repo.Revision, Format: "strata", Arch: repo.Architecture, License: repo.License,
Context: strataContext, Servable: true, Note: "Strata fetches this pack onto the node itself on its first start"}
p.Options = modelplan.StrataOptions(fam.Name, []modelplan.Artifact{art}, s.deps.Pool())
if len(p.Options) == 0 && fam.MinCC > 0 {
p.Note = fmt.Sprintf("Strata's %s engine needs an NVIDIA card of compute capability %.1f or newer (RTX 30 on); "+
"a node with Strata and such a card, and %d GB of RAM, runs it", fam.Title, fam.MinCC, size.MemoryGB())
}
pa := PlannedArtifact{Artifact: art}
if len(p.Options) > 0 {
pa.Best = &p.Options[0]
}
p.Artifacts = []PlannedArtifact{pa}
return p, nil
}
p := Plan{Repo: repo.ID, Revision: repo.Revision, Format: req.Format, Arch: repo.Architecture,
License: repo.License, Params: repo.Params, Context: req.Context, Gated: repo.Gated.Is(), Servable: true,
Projectors: Projectors(repo)}
@@ -35,3 +35,18 @@ func TestAStrataWayIsChosenAndMadeAProfile(t *testing.T) {
t.Errorf("%+v", p)
}
}
func TestAReadyMadePackIsPlannedOnStrataOnly(t *testing.T) {
s := New(Deps{Pool: func() []modelplan.PoolNode {
return []modelplan.PoolNode{{ID: "cuda01", Class: "6000-24g", GPUMemGB: 24, GPUs: 1, FreeGPUs: 1,
HostMemGB: 251, HostFreeGB: 200, Engines: []string{"strata"}}}
}})
p, err := s.plan(t.Context(), hf.Repo{ID: "Maximilian228/Qwen3.8-Flash-Next-NVIDIA-NVFP4-Strata", Revision: "abc",
Architecture: "strata-ple"}, PlanRequest{})
if err != nil {
t.Fatal(err)
}
if !p.Servable || len(p.Options) != 1 || p.Options[0].StrataFamily != "nvfp4" || p.Options[0].StrataSize != "NVFP4-NVIDIA" {
t.Errorf("%+v", p)
}
}
+3 -1
View File
@@ -469,7 +469,9 @@ func (c *Client) RepoAt(ctx context.Context, id, rev string) (Repo, error) {
if rev == "" {
rev = "main"
}
if err := c.get(ctx, "/api/models/"+id+"/tree/"+rev, &tree); err != nil {
// Recursive: a repository that keeps each quantisation in a directory of
// its own — ISTA-DASLab's IQ2_XS/, IQ3_S/ — listed no weights at all.
if err := c.get(ctx, "/api/models/"+id+"/tree/"+rev+"?recursive=true", &tree); err != nil {
return Repo{}, err
}
+26
View File
@@ -266,3 +266,29 @@ func TestAListingNamesTheBaseModelAndLicenseFromItsTags(t *testing.T) {
t.Error("a base model with no relation should read as itself")
}
}
// A repository that keeps each quantisation in a directory of its own lists
// its weights: the tree is asked for recursively.
func TestFilesInDirectoriesAreListed(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.Contains(r.URL.Path, "/tree/"):
if r.URL.Query().Get("recursive") != "true" {
_, _ = w.Write([]byte(`[{"type":"directory","path":"IQ2_XS"}]`))
return
}
_, _ = w.Write([]byte(`[{"type":"directory","path":"IQ2_XS"},
{"type":"file","path":"IQ2_XS/m-IQ2_XS-00001-of-00002.gguf","size":100,"lfs":{"oid":"aa","size":5000}}]`))
default:
_, _ = w.Write([]byte(`{"id":"o/r","sha":"abc"}`))
}
}))
defer srv.Close()
r, err := New(srv.URL, 0).Repo(t.Context(), "o/r")
if err != nil {
t.Fatal(err)
}
if len(r.Files) != 1 || r.Files[0].Path != "IQ2_XS/m-IQ2_XS-00001-of-00002.gguf" || r.Files[0].SHA256 != "aa" {
t.Errorf("%+v", r.Files)
}
}
+2
View File
@@ -55,6 +55,8 @@ type PoolNode struct {
Kinds []PoolKind
// Engines are the engines the node has installed.
Engines []string
// CC is its newest card's compute capability, 0 when not known.
CC float64
}
// PoolKind is one class of a node's devices.
+6
View File
@@ -24,12 +24,18 @@ func StrataOptions(family string, arts []Artifact, nodes []PoolNode) []Option {
continue
}
need := float64(size.MemoryGB())
fam, _ := strata.FamilyNamed(family)
byClass := map[string]*Option{}
var order []string
for _, n := range nodes {
if !slices.Contains(n.Engines, "strata") || n.HostMemGB <= 0 {
continue
}
// A family whose engine needs a newer card: refused by Strata on
// this one, after the node was chosen.
if fam.MinCC > 0 && n.CC > 0 && n.CC < fam.MinCC {
continue
}
have := float64(n.HostMemGB) * hostShare
if need > have {
continue
+19
View File
@@ -30,3 +30,22 @@ func TestStrataIsOfferedOnANodeWithItTheCardAndTheRAM(t *testing.T) {
t.Errorf("%q %q", o.Label(), o.MemoryText())
}
}
// DeepSeek's engine is built for RTX 30 and newer: not offered on a Turing
// card, which Strata's setup refuses after the replica is placed.
func TestDeepSeekOnStrataNeedsANewerCard(t *testing.T) {
arts := []Artifact{{Name: "IST-DAS-1.75bpw.gguf", Format: "gguf"}}
node := func(cc float64) []PoolNode {
return []PoolNode{{ID: "n", Class: "6000-24g", GPUMemGB: 24, GPUs: 1, FreeGPUs: 1, HostMemGB: 251,
Engines: []string{"strata"}, CC: cc}}
}
if got := StrataOptions("deepseek41", arts, node(7.5)); len(got) != 0 {
t.Errorf("offered on 7.5: %+v", got)
}
if got := StrataOptions("deepseek41", arts, node(8.9)); len(got) != 1 {
t.Errorf("not offered on 8.9: %+v", got)
}
if got := StrataOptions("qwen", []Artifact{{Name: "IQ2_XS.gguf"}}, node(7.5)); len(got) != 1 {
t.Errorf("Qwen not offered on 7.5: %+v", got)
}
}
+72 -14
View File
@@ -14,9 +14,9 @@ import (
// Family is a model Strata runs, from one publisher's files.
type Family struct {
// Name is setup's --family.
Name string
Title string
// Name is setup's --family; Model the name Strata gives the model.
Name, Model string
Title string
// Repo is the Hugging Face repository its files are in; empty when
// Strata fetches them itself from several.
Repo string
@@ -26,6 +26,10 @@ type Family struct {
// Engine is a family that runs on an engine path of its own, with a
// download of its own (DeepSeek V4.1).
Engine bool
// MinCC is the compute capability its engine needs of an NVIDIA card,
// above Strata's own 6.0; 0 for none. DeepSeek's engine is built for
// RTX 30 (8.6) and newer only.
MinCC float64
}
// Size is one of a family's sizes: setup's --model, which llmgw's profile
@@ -42,18 +46,21 @@ type Size struct {
RAMGB float64
// NVIDIAOnly is a size whose kernels only NVIDIA cards have.
NVIDIAOnly bool
// Repo is a size's own repository, for a ready-made pack Strata
// fetches itself.
Repo string
}
// Families are the families Strata 0.2.7 runs.
var Families = []Family{
{Name: "qwen", Title: "Qwen3.8-Flash-Next", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF", Subdir: true},
{Name: "swift", Title: "Swift 1.5 (Qwen3.8-Flash-Next, shorter thinking)", Repo: "ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF"},
{Name: "coder", Title: "Qwen3.8-Flash-Next Coder", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF", Subdir: true},
{Name: "unsloth", Title: "Qwen3.8-Flash-Next (Unsloth)", Repo: "unsloth/Qwen3.8-Flash-Next-GGUF", Subdir: true},
{Name: "gyro", Title: "Qwen3.8-Flash-Next (Gyro)", Repo: "agentionai/Qwen3.8-Flash-Next-Gyro-GGUF"},
{Name: "nvfp4", Title: "Qwen3.8-Flash-Next (NVFP4)"},
{Name: "qwen36", Title: "Qwen3.6-35B-A3B", Repo: "unsloth/Qwen3.6-35B-A3B-MTP-GGUF"},
{Name: "deepseek41", Title: "DeepSeek V4.1-Flash", Repo: "ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF", Engine: true},
{Name: "qwen", Model: "qwen3.8-flash-next", Title: "Qwen3.8-Flash-Next", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF", Subdir: true},
{Name: "swift", Model: "swift-1.5", Title: "Swift 1.5 (Qwen3.8-Flash-Next, shorter thinking)", Repo: "ukisai/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-GGUF"},
{Name: "coder", Model: "qwen3.8-flash-next-coder", Title: "Qwen3.8-Flash-Next Coder", Repo: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-Coder-GGUF", Subdir: true},
{Name: "unsloth", Model: "qwen3.8-flash-next-unsloth", Title: "Qwen3.8-Flash-Next (Unsloth)", Repo: "unsloth/Qwen3.8-Flash-Next-GGUF", Subdir: true},
{Name: "gyro", Model: "qwen3.8-flash-next-gyro", Title: "Qwen3.8-Flash-Next (Gyro)", Repo: "agentionai/Qwen3.8-Flash-Next-Gyro-GGUF"},
{Name: "nvfp4", Model: "qwen3.8-flash-next-nvfp4", Title: "Qwen3.8-Flash-Next (NVFP4)"},
{Name: "qwen36", Model: "qwen3.6-35b-a3b", Title: "Qwen3.6-35B-A3B", Repo: "unsloth/Qwen3.6-35B-A3B-MTP-GGUF"},
{Name: "deepseek41", Model: "deepseek-v4.1-flash", Title: "DeepSeek V4.1-Flash", Repo: "ISTA-DASLab/DeepSeek-V4.1-Flash-1017-GSQ-RCO-GGUF", Engine: true, MinCC: 8.6},
}
// Sizes are the sizes Strata 0.2.7 runs, and which families have each.
@@ -70,9 +77,12 @@ var Sizes = []Size{
{Name: "Q8_0", Families: []string{"unsloth", "qwen36"}, DownloadGB: 188.2, RAMGB: 136},
{Name: "Gyro-S", Families: []string{"gyro"}, DownloadGB: 58.5, RAMGB: 27.6},
{Name: "Gyro-M", Families: []string{"gyro"}, DownloadGB: 92.0, RAMGB: 35},
{Name: "NVFP4", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true},
{Name: "NVFP4-NVIDIA", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true},
{Name: "NVFP4-QAD", Families: []string{"nvfp4"}, DownloadGB: 102.2, DiskGB: 212.2, RAMGB: 80, NVIDIAOnly: true},
{Name: "NVFP4", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true,
Repo: "Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata"},
{Name: "NVFP4-NVIDIA", Families: []string{"nvfp4"}, DownloadGB: 128, RAMGB: 80, NVIDIAOnly: true,
Repo: "Maximilian228/Qwen3.8-Flash-Next-NVIDIA-NVFP4-Strata"},
{Name: "NVFP4-QAD", Families: []string{"nvfp4"}, DownloadGB: 102.2, DiskGB: 212.2, RAMGB: 80, NVIDIAOnly: true,
Repo: "nvidia/Qwen3.8-Flash-Next-FP4-Local"},
{Name: "IST-DAS-1.75bpw", Families: []string{"deepseek41"}, DownloadGB: 239, RAMGB: 34},
{Name: "IST-DAS-2.00bpw", Families: []string{"deepseek41"}, DownloadGB: 257, RAMGB: 40},
{Name: "IST-DAS-2.25bpw", Families: []string{"deepseek41"}, DownloadGB: 274, RAMGB: 44},
@@ -172,3 +182,51 @@ func SizeForFile(family, file string) (Size, bool) {
}
return best, best.Name != ""
}
// FetchesItself is a family whose files Strata's setup fetches itself, each
// pinned to a commit and checked: ready-made packs, and DeepSeek's sets with
// an engine of their own. A model of it names its repository as the source.
func (f Family) FetchesItself() bool { return f.Name == "nvfp4" || f.Engine }
// RepoOf is the repository a size of a family is in.
func RepoOf(f Family, s Size) string {
if s.Repo != "" {
return s.Repo
}
return f.Repo
}
// ForFetchedRepo is the family and size a repository is, for one Strata
// fetches itself: a ready-made pack's own, or DeepSeek's (whose size the
// repository does not say, so the smallest).
func ForFetchedRepo(repo string) (Family, Size, bool) {
for _, f := range Families {
if !f.FetchesItself() {
continue
}
for _, s := range SizesOf(f.Name) {
if strings.EqualFold(RepoOf(f, s), repo) {
return f, s, true
}
}
}
return Family{}, Size{}, false
}
// Fetchable says whether a size's files can be fetched by llmgw from the
// family's repository: not where Strata fetches them itself (NVFP4's packs,
// DeepSeek's sets with an engine of their own).
func (f Family) Fetchable() bool { return f.Repo != "" && !f.Engine }
// FilesOf are the weights of a size among a repository's files: its GGUF
// shards, not the picture reader.
func FilesOf(size string, paths []string) []string {
var out []string
for _, p := range paths {
u := strings.ToUpper(p)
if strings.HasSuffix(u, ".GGUF") && !strings.Contains(u, "MMPROJ") && strings.Contains(u, strings.ToUpper(size)) {
out = append(out, p)
}
}
return out
}
+31 -4
View File
@@ -24,6 +24,7 @@ import (
"gitea.mynodes.xyz/ilker/llmgw/internal/safedial"
"gitea.mynodes.xyz/ilker/llmgw/internal/sourcecheck"
"gitea.mynodes.xyz/ilker/llmgw/internal/store"
"gitea.mynodes.xyz/ilker/llmgw/internal/strata"
)
// MirrorAdmin is the mirror's management API as this page needs it.
@@ -169,6 +170,10 @@ type adminMirrorData struct {
// the serve action is then not offered rather than offered and refused.
HasCatalog bool
// Strata is the Strata tab: the models it runs, and where they would
// run here.
Strata *strataView
// The explorer.
HasRegistry bool
Query string
@@ -256,6 +261,14 @@ func (p *AccountPages) RegisterAdminMirror(mux *http.ServeMux, d MirrorDeps) {
render(w, r, "mirror_repo", d)
}))
mux.HandleFunc("POST /admin/mirror/strata", p.mirrorAction(m,
func(ctx context.Context, admin store.User, r *http.Request) (string, error) {
if r.FormValue("action") == "serve" {
return p.serveStrata(ctx, langOf(r), admin.Email, r.FormValue("family"), r.FormValue("size"))
}
return p.fetchStrata(ctx, langOf(r), m, r.FormValue("family"), r.FormValue("size"))
}))
mux.HandleFunc("POST /admin/mirror/fetch", p.mirrorAction(m,
func(ctx context.Context, admin store.User, r *http.Request) (string, error) {
// One form posts one source or fifteen. A sharded model is one
@@ -462,6 +475,9 @@ func (p *AccountPages) renderMirror(w http.ResponseWriter, r *http.Request,
if d.Tab == tabExplore {
p.explore(r, &d)
}
if d.Tab == tabStrata {
d.Strata = p.strataView(l)
}
got, err := m.List(r.Context())
if err != nil {
@@ -618,6 +634,7 @@ const (
tabHeld = "held"
tabExplore = "explore"
tabFetch = "fetch"
tabStrata = "strata"
)
// mirrorTab is one of the page's tabs.
@@ -636,7 +653,7 @@ type mirrorTab struct {
// shown, where its effect is, unless it was refused: then the form it came
// from, to try again.
func mirrorTabFor(r *http.Request, errMsg string) string {
valid := func(t string) bool { return t == tabHeld || t == tabExplore || t == tabFetch }
valid := func(t string) bool { return t == tabHeld || t == tabExplore || t == tabFetch || t == tabStrata }
if r.Method == http.MethodPost {
if t := r.PostFormValue("tab"); errMsg != "" && valid(t) {
return t
@@ -673,6 +690,10 @@ func mirrorTabs(l *Lang, d adminMirrorData) []mirrorTab {
tabs = append(tabs, mirrorTab{Label: l.T("mirror.tab.explore"),
Href: "/admin/mirror?tab=explore", Active: d.Tab == tabExplore})
}
if d.HasRegistry {
tabs = append(tabs, mirrorTab{Label: l.T("mirror.tab.strata"),
Href: "/admin/mirror?tab=strata", Active: d.Tab == tabStrata})
}
return append(tabs, mirrorTab{Label: l.T("mirror.tab.fetch"),
Href: "/admin/mirror?tab=fetch", Active: d.Tab == tabFetch})
}
@@ -823,9 +844,15 @@ func (p *AccountPages) openRepo(r *http.Request, d *adminMirrorData, repo string
Version: got.ShortRevision(),
}
rs.Verdict = p.verdictFor(langOf(r), got.Architecture, set.Name)
// A family Strata runs: one card and the experts in RAM, whether
// or not llama.cpp could load it.
if layout, ok := fit.strataLayout(langOf(r), got.ID, set.Name, set.Bytes); ok {
// A pack made for Strata: served whole, from the Strata tab, when
// Strata knows it; named for what it is when it does not.
if fam, size, ok := strata.ForFetchedRepo(got.ID); ok {
rs.Verdict = gguf.Verdict{Checked: true, Reason: langOf(r).T("strata.pack_files", fam.Title, size.Name)}
} else if got.Architecture == "strata-ple" {
rs.Verdict = gguf.Verdict{Checked: true, Reason: langOf(r).T("strata.unknown_pack")}
} else if layout, ok := fit.strataLayout(langOf(r), got.ID, set.Name, set.Bytes); ok {
// A family Strata runs: one card and the experts in RAM,
// whether or not llama.cpp could load it.
rs.Verdict, rs.GPULayout = gguf.Verdict{Checked: true, Servable: true}, layout
} else if fit != nil && rs.Verdict.Servable {
if v, ok := fit.holds(langOf(r), set.Name, set.Bytes, got.Params); !ok {
+171
View File
@@ -0,0 +1,171 @@
package web
import (
"context"
"errors"
"slices"
"strconv"
"strings"
"time"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/catalog"
"gitea.mynodes.xyz/ilker/llmgw/internal/mirror"
"gitea.mynodes.xyz/ilker/llmgw/internal/modelplan"
"gitea.mynodes.xyz/ilker/llmgw/internal/strata"
)
// The mirror page's Strata tab: the models Strata runs, each size with what
// it downloads and keeps in RAM and where it would run here, and a button
// that fetches a size's files into the mirror, where the version's serve
// form then serves it on Strata.
type strataView struct {
// Nodes are the nodes that have Strata installed.
Nodes []string
Families []strataFamilyRow
}
type strataFamilyRow struct {
Name, Title, Repo string
// Fetchable is a family whose files llmgw fetches into the mirror;
// FetchesItself one whose files Strata fetches onto the node, served
// from here at once.
Fetchable, FetchesItself bool
Sizes []strataSizeRow
}
type strataSizeRow struct {
Size string
DownloadGB, RAMGB string
Runs string
Fits, NVIDIAOnly bool
}
func (p *AccountPages) strataView(l *Lang) *strataView {
v := &strataView{}
var pool []modelplan.PoolNode
if fit := p.fitOf(); fit != nil {
pool = fit.pool
}
for _, n := range pool {
if slices.Contains(n.Engines, "strata") {
v.Nodes = append(v.Nodes, n.ID)
}
}
for _, f := range strata.Families {
row := strataFamilyRow{Name: f.Name, Title: f.Title, Repo: f.Repo, Fetchable: f.Fetchable(),
FetchesItself: f.FetchesItself()}
for _, s := range strata.SizesOf(f.Name) {
sr := strataSizeRow{Size: s.Name, DownloadGB: l.Num(s.DownloadGB, 0), RAMGB: l.Int(int64(s.MemoryGB())),
NVIDIAOnly: s.NVIDIAOnly}
opts := modelplan.StrataOptions(f.Name, []modelplan.Artifact{{Name: s.Name + ".gguf", Format: "gguf",
Bytes: int64(s.DownloadGB * 1e9)}}, pool)
switch {
case len(opts) > 0:
sr.Fits, sr.Runs = true, l.T("explore.strata", opts[0].Class, l.Num(opts[0].NeedGB, 0))
case len(v.Nodes) == 0:
sr.Runs = l.T("strata.no_node")
case f.MinCC > 0 && cardsOlderThan(pool, f.MinCC):
sr.Runs = l.T("strata.old_card", l.Num(f.MinCC, 1))
default:
sr.Runs = l.T("strata.no_room", s.MemoryGB())
}
row.Sizes = append(row.Sizes, sr)
}
v.Families = append(v.Families, row)
}
return v
}
// cardsOlderThan says whether every node with Strata has a card of a
// compute capability below cc, as far as they say.
func cardsOlderThan(pool []modelplan.PoolNode, cc float64) bool {
older := false
for _, n := range pool {
if !slices.Contains(n.Engines, "strata") {
continue
}
if n.CC == 0 || n.CC >= cc {
return false
}
older = true
}
return older
}
// serveStrata adds a model of a family Strata fetches itself: its source
// the repository, its replica on a card of the class the plan found.
func (p *AccountPages) serveStrata(ctx context.Context, l *Lang, by, family, size string) (string, error) {
fam, ok := strata.FamilyNamed(family)
s, err := strata.SizeOf(family, size)
if !ok || err != nil {
return "", errors.New(l.T("strata.err.unknown", family, size))
}
if !fam.FetchesItself() || p.catalog == nil {
return "", errors.New(l.T("strata.err.not_servable", fam.Title))
}
var pool []modelplan.PoolNode
if fit := p.fitOf(); fit != nil {
pool = fit.pool
}
opts := modelplan.StrataOptions(fam.Name, []modelplan.Artifact{{Name: s.Name + ".gguf", Format: "gguf"}}, pool)
if len(opts) == 0 {
return "", errors.New(l.T("strata.no_room", s.MemoryGB()))
}
name := fam.Model + "-" + strings.ToLower(s.Name)
disk := s.DownloadGB
if s.DiskGB > disk {
disk = s.DiskGB
}
m := catalog.Model{Name: name, Source: strata.RepoOf(fam, s), RoutingProfile: "chat", ContextLength: strataContext,
Scaling: catalog.Scaling{Min: 1, Max: 1},
Profiles: []catalog.Profile{{Name: "strata-" + opts[0].Class, Engine: "strata", GPUClass: opts[0].Class,
GPUsPerNode: 1, Weight: 1, Priority: 100, MinMemoryGB: s.MemoryGB(), MinDiskGB: int(disk) + 10,
Args: []string{"--family", fam.Name, "--size", s.Name, "--context", strconv.Itoa(strataContext), "--vision", "no"},
// Its files come onto the node on its first start, tens of
// gigabytes from the internet, before it is prepared.
StartupTimeout: 4 * time.Hour}}}
if err := p.catalog.Add(ctx, m, by); err != nil {
return "", err
}
return l.T("strata.msg.served", name, l.Num(disk, 0)), nil
}
// strataContext is the context a Strata model is served with from here.
const strataContext = 32768
// fetchStrata fetches a Strata size's files from its family's repository
// into the mirror, each checked against the sha256 the registry published.
func (p *AccountPages) fetchStrata(ctx context.Context, l *Lang, m MirrorAdmin, family, size string) (string, error) {
fam, ok := strata.FamilyNamed(family)
if _, err := strata.SizeOf(family, size); !ok || err != nil {
return "", errors.New(l.T("strata.err.unknown", family, size))
}
if !fam.Fetchable() || p.registry == nil {
return "", errors.New(l.T("strata.err.not_fetchable", fam.Title))
}
repo, err := p.registry.Repo(ctx, fam.Repo)
if err != nil {
return "", err
}
var paths []string
sums := map[string]string{}
for _, f := range repo.Files {
paths = append(paths, f.Path)
sums[f.Path] = f.SHA256
}
files := strata.FilesOf(size, paths)
if len(files) == 0 {
return "", errors.New(l.T("strata.err.no_files", fam.Repo, size))
}
model := fam.Model + "-" + strings.ToLower(size)
version := repo.ShortRevision()
for i, path := range files {
req := mirror.FetchRequest{Source: repo.DownloadURL("", path), Model: model, Version: version,
File: mirror.LastPathElement(path), SHA256: sums[path]}
if _, err := m.Fetch(ctx, req); err != nil && !strings.Contains(err.Error(), "already") {
return "", errors.New(l.T("mirror.err.partly_started", i, len(files), req.File, l.Err(err)))
}
}
return l.T("mirror.msg.started", model, version, l.N("mirror.files", int64(len(files)))), nil
}
+89
View File
@@ -0,0 +1,89 @@
package web
import (
"net/url"
"strings"
"testing"
"time"
"gitea.mynodes.xyz/ilker/llmgw/internal/control/adminapi"
"gitea.mynodes.xyz/ilker/llmgw/internal/hf"
)
// strataFleet is one node with an RTX 6000, 251 GB and Strata installed.
type strataFleet struct{ fakeFleet }
func (strataFleet) Nodes() []adminapi.NodeView {
return []adminapi.NodeView{{ID: "cuda01", State: "ready", GPUTotal: 1, GPUFree: 1, MemoryGB: 251,
Classes: []string{"6000-24g"}, Engines: []string{"llamacpp", "strata"}, ComputeCapabilities: []string{"7.5"}}}
}
func TestTheStrataTabListsWhatRunsHereAndFetchesASize(t *testing.T) {
reg := &fakeRegistry{repo: hf.Repo{ID: "ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF",
Revision: "0123456789abcdef0123456789abcdef01234567", Files: []hf.File{
{Path: "IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00001-of-00002.gguf", SHA256: "aa"},
{Path: "IQ2_XS/Qwen3.8-Flash-Next-GSQ-RCO-IQ2_XS-00002-of-00002.gguf", SHA256: "bb"},
{Path: "IQ3_S/Qwen3.8-Flash-Next-GSQ-RCO-IQ3_S-00001-of-00002.gguf"},
{Path: "mmproj-Qwen3.8-Flash-Next-BF16.gguf"},
}}}
m := &fakeMirror{}
mux := mirrorPageDeps(t, MirrorDeps{Mirror: m, Registry: reg, Fleet: strataFleet{}}, true)
body := mirrorGet(t, mux, "/admin/mirror?tab=strata").Body.String()
for _, want := range []string{"Strata is installed on: cuda01", "Qwen3.8-Flash-Next", "IQ2_XS",
"1× 6000-24g with Strata, 52 GB of RAM", `name="size" value="IQ2_XS"`, "Strata fetches these onto the node itself",
"compute capability 8.6 or newer"} {
if !strings.Contains(body, want) {
t.Errorf("the tab lacks %q", want)
}
}
// Unsloth's Q8_0 keeps 140 GB: it runs here; nothing past the node's RAM does.
if strings.Contains(body, `value="UD-Q8_K_XL"`) || strings.Contains(body, `value="IST-DAS-1.75bpw"`) {
t.Error("offered DeepSeek's 306 GB size, which no node holds, and which Strata fetches itself")
}
mirrorPost(t, mux, "/admin/mirror/strata", url.Values{"csrf": {"tok"}, "family": {"qwen"}, "size": {"IQ2_XS"}})
if len(m.all) != 2 {
t.Fatalf("fetched %+v", m.all)
}
f := m.all[0]
if f.Model != "qwen3.8-flash-next-iq2_xs" || f.Version != "01234567" || f.SHA256 != "aa" ||
!strings.Contains(f.Source, "/resolve/0123456789abcdef0123456789abcdef01234567/IQ2_XS/") {
t.Errorf("%+v", f)
}
}
// A ready-made pack is served at once, its repository the source, Strata
// fetching it onto the node.
func TestAReadyMadeStrataPackIsServedFromTheTab(t *testing.T) {
cat := &fakeCatalog{}
mux := mirrorPageDeps(t, MirrorDeps{Mirror: &fakeMirror{}, Registry: &fakeRegistry{}, Catalog: cat, Fleet: strataFleet{}}, true)
if body := mirrorGet(t, mux, "/admin/mirror?tab=strata").Body.String(); !strings.Contains(body, `name="action" value="serve"`) {
t.Fatal("no serve button for a pack")
}
mirrorPost(t, mux, "/admin/mirror/strata", url.Values{"csrf": {"tok"}, "action": {"serve"}, "family": {"nvfp4"}, "size": {"NVFP4"}})
if len(cat.added) != 1 {
t.Fatalf("added %+v", cat.added)
}
m := cat.added[0]
p := m.Profiles[0]
if m.Name != "qwen3.8-flash-next-nvfp4-nvfp4" || m.Source != "Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata" ||
p.Engine != "strata" || p.GPUClass != "6000-24g" || p.MinMemoryGB != 84 || p.StartupTimeout < time.Hour ||
strings.Join(p.Args, " ") != "--family nvfp4 --size NVFP4 --context 32768 --vision no" {
t.Errorf("%+v", m)
}
}
func TestAStrataPacksFilesAreNamedForWhatTheyAre(t *testing.T) {
for repo, want := range map[string]string{
"Maximilian228/Qwen3.8-Flash-Next-NVFP4-GPTQ-Strata": "ready-made Qwen3.8-Flash-Next (NVFP4) NVFP4 pack",
"trymirai/Qwen3.8-Flash-Next-S-strata-experimental": "Strata 0.2.7 does not know",
} {
reg := &fakeRegistry{repo: hf.Repo{ID: repo, Architecture: "strata-ple", Revision: "abc",
Files: []hf.File{{Path: "ple-fp8.gguf", Bytes: 51e9}}}}
mux := mirrorPageDeps(t, MirrorDeps{Mirror: &fakeMirror{}, Registry: reg, Fleet: strataFleet{}}, true)
if body := mirrorGet(t, mux, "/admin/mirror?repo="+repo).Body.String(); !strings.Contains(body, want) {
t.Errorf("%s: no %q", repo, want)
}
}
}
+23 -1
View File
@@ -257,5 +257,27 @@
"mirror.err.wrong_arch": "that adapter is for %s models and %s is %s",
"mirror.msg.adapter_added": "%[1]s: adapter %[2]s added; ask for it by that name once the replicas are restarted (llmgw ctl restart --model %[1]s)",
"mirror.err.adapter_op": "add or remove an adapter",
"explore.strata": "1× %s with Strata, %s GB of RAM"
"explore.strata": "1× %s with Strata, %s GB of RAM",
"mirror.tab.strata": "Strata",
"strata.intro": "Strata runs these mixture-of-experts models with the experts in a node's RAM and the rest on one card of 12 GB or more: a model that fits no card, served from one. Fetch a size into the mirror; its serve form on the mirror's tab then serves it on Strata. The first start prepares it on the node, which takes minutes.",
"strata.nodes": "Strata is installed on: %s",
"strata.none": "No node has Strata installed. Name it among the cluster's engines (<code>llmgw ctl config set engines.strata 0.2.7</code>) and the nodes with an NVIDIA card install it.",
"strata.self": "Strata fetches these onto the node itself, each file pinned and checked, on the model's first start: served from here, they do not go through the mirror.",
"strata.col.size": "Size",
"strata.col.download": "Download",
"strata.col.ram": "RAM it keeps",
"strata.col.runs": "Here",
"strata.fetch": "Fetch into the mirror",
"strata.nvidia": "NVIDIA only",
"strata.no_node": "no node has Strata",
"strata.no_room": "no node with Strata has %d GB of RAM and a card of 12 GB",
"strata.err.unknown": "Strata runs no %s %s",
"strata.err.not_fetchable": "Strata fetches %s itself; serve it with a definition that names its family and size.",
"strata.err.no_files": "%s has no files of size %s",
"strata.serve": "Serve",
"strata.err.not_servable": "%s is fetched into the mirror first, then served from its version",
"strata.msg.served": "%s added: its first start fetches about %s GB onto the node and prepares it, which can take hours on a slow link; Replicas shows how far it is.",
"strata.pack_files": "part of Strata's ready-made %s %s pack: served whole from the mirror's Strata tab, Strata fetching it onto the node",
"strata.unknown_pack": "a pack made for Strata that Strata 0.2.7 does not know: its setup refuses it, and llama.cpp cannot read it",
"strata.old_card": "its engine needs an NVIDIA card of compute capability %s or newer (RTX 30 on); the nodes with Strata have older ones"
}
+23 -1
View File
@@ -250,5 +250,27 @@
"mirror.err.wrong_arch": "bu adaptör %s modelleri için; %s ise %s",
"mirror.msg.adapter_added": "%[1]s: %[2]s adaptörü eklendi; replikalar yeniden başlatılınca bu adla isteyin (llmgw ctl restart --model %[1]s)",
"mirror.err.adapter_op": "bir adaptör ekleyin ya da kaldırın",
"explore.strata": "Strata ile 1× %s, %s GB RAM"
"explore.strata": "Strata ile 1× %s, %s GB RAM",
"mirror.tab.strata": "Strata",
"strata.intro": "Strata bu uzman karışımı (MoE) modelleri, uzmanları node'un RAM'inde, gerisini 12 GB ya da daha büyük tek bir kartta tutarak çalıştırır: hiçbir karta sığmayan bir model tek karttan sunulur. Bir boyutu aynaya indirin; aynadakiler sekmesindeki sunma formu onu Strata ile sunar. İlk açılışta model node üzerinde hazırlanır; bu birkaç dakika sürer.",
"strata.nodes": "Strata kurulu node'lar: %s",
"strata.none": "Hiçbir node'da Strata kurulu değil. Kümenin motorları arasında adını verin (<code>llmgw ctl config set engines.strata 0.2.7</code>); NVIDIA kartı olan node'lar onu kurar.",
"strata.self": "Bunları Strata, modelin ilk açılışında her dosyayı sabitleyip denetleyerek node'a kendisi indirir: buradan sunulur, aynadan geçmez.",
"strata.col.size": "Boyut",
"strata.col.download": "İndirme",
"strata.col.ram": "Ayırdığı RAM",
"strata.col.runs": "Burada",
"strata.fetch": "Aynaya indir",
"strata.nvidia": "Yalnız NVIDIA",
"strata.no_node": "Strata hiçbir node'da yok",
"strata.no_room": "Strata kurulu hiçbir node'da %d GB RAM ve 12 GB'lık kart yok",
"strata.err.unknown": "Strata %s %s çalıştırmıyor",
"strata.err.not_fetchable": "%s dosyalarını Strata kendisi indirir; ailesini ve boyutunu veren bir tanımla sunun.",
"strata.err.no_files": "%s içinde %s boyutunda dosya yok",
"strata.serve": "Sun",
"strata.err.not_servable": "%s önce aynaya indirilir, sonra sürümünden sunulur",
"strata.msg.served": "%s eklendi: ilk açılışında node'a yaklaşık %s GB indirip hazırlar; yavaş bağlantıda saatler sürebilir. İlerleyişi Replikalar gösterir.",
"strata.pack_files": "Strata'nın hazır %s %s paketinin parçası: aynanın Strata sekmesinden bütün olarak sunulur, Strata onu node'a kendisi indirir",
"strata.unknown_pack": "Strata için hazırlanmış ama Strata 0.2.7'nin tanımadığı bir paket: kurulumu onu reddeder, llama.cpp de okuyamaz",
"strata.old_card": "motoru compute capability %s ya da daha yeni bir NVIDIA kartı ister (RTX 30 ve sonrası); Strata kurulu node'ların kartları daha eski"
}
+34
View File
@@ -424,6 +424,40 @@ document.querySelector("table.explore").addEventListener("submit", async functio
{{if .Repo}}{{template "mirror_repo" .}}{{end}}
{{end}}
{{if and (eq .Tab "strata") .Strata}}
<p class="note">{{t "strata.intro"}}</p>
{{if .Strata.Nodes}}<p class="sub">{{t "strata.nodes" (join .Strata.Nodes)}}</p>
{{else}}<p class="note error-row">{{th "strata.none"}}</p>{{end}}
{{range .Strata.Families}}
<h3>{{.Title}} <span class="opt"><code>--family {{.Name}}</code>{{if .Repo}} · <a href="/admin/mirror?repo={{.Repo}}">{{.Repo}}</a>{{end}}</span></h3>
{{if .FetchesItself}}<p class="sub">{{t "strata.self"}}</p>{{end}}
<table class="rows">
<tr><th>{{t "strata.col.size"}}</th><th>{{t "strata.col.download"}}</th><th>{{t "strata.col.ram"}}</th><th>{{t "strata.col.runs"}}</th><th></th></tr>
{{$fam := .}}
{{range .Sizes}}
<tr>
<td><code>{{.Size}}</code>{{if .NVIDIAOnly}} <span class="opt">{{t "strata.nvidia"}}</span>{{end}}</td>
<td>{{.DownloadGB}} GB</td>
<td>{{.RAMGB}} GB</td>
<td>{{if .Fits}}{{.Runs}}{{else}}<span class="opt">{{.Runs}}</span>{{end}}</td>
<td>{{if and $fam.FetchesItself .Fits}}<form method="post" action="/admin/mirror/strata" class="inline">
<input type="hidden" name="csrf" value="{{$.CSRF}}">
<input type="hidden" name="tab" value="strata">
<input type="hidden" name="action" value="serve">
<input type="hidden" name="family" value="{{$fam.Name}}">
<input type="hidden" name="size" value="{{.Size}}">
<button type="submit">{{t "strata.serve"}}</button></form>{{end}}{{if and $fam.Fetchable .Fits}}<form method="post" action="/admin/mirror/strata" class="inline">
<input type="hidden" name="csrf" value="{{$.CSRF}}">
<input type="hidden" name="tab" value="strata">
<input type="hidden" name="family" value="{{$fam.Name}}">
<input type="hidden" name="size" value="{{.Size}}">
<button type="submit">{{t "strata.fetch"}}</button></form>{{end}}</td>
</tr>
{{end}}
</table>
{{end}}
{{end}}
{{if eq .Tab "fetch"}}
<p class="note">{{t "mirror.fetch.intro"}}</p>
<form method="post" action="/admin/mirror/fetch">