diff --git a/BACKLOG.md b/BACKLOG.md index 5e0c032..5b7c9f8 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -1065,6 +1065,27 @@ training yet; quality gating of a published adapter is by hand (ctl eval). --- +### F5. Large imports (M) — measured 2026-10-09, open +The metallurgy corpus (2,314 files, 108,779 chunks, bge-m3 on the one +GPU) went in at about 25 chunks/s, the same over a phone hotspot and the +LAN. The engine's slots were the limit: llama.cpp `--parallel 4` embeds +four texts per pass, while llmgw-kb sends 32 a request from 6 workers. The +usage rows show 4.6 s a request on average, 1.8 s of it in the gateway's +queue and 0.04 s in the guard. The GPU sat at 18–50% and the store machine +at a third of its CPUs. +- An embedding model's profile with many slots by default (`--parallel` + 32–64, `--ctx-size` to match, `--batch-size`/`--ubatch-size` at least the + longest chunk), or `ctl model serve` choosing that for a model with + `capabilities.embeddings`. With the default 512, one 516-token chunk failed. +- Embedding on the gateway's side of a document PUT, so the client sends + text only: today an embeddings answer is ~21 KB of JSON a chunk, and + the text and vector go up again (~11 KB). +- The store next: HNSW inserts took half of one of pg01's two cores; at a + million chunks the vectors alone outgrow its 4 GB. halfvec (pgvector ≥ + 0.7) and a larger store machine. +- Done: a batch of embeddings checked text by text against the context + (branch `embeddings-batch`); it was refused as one over-long prompt. + ## D. Facts worth knowing before starting - Admin API: `127.0.0.1:9090`, no auth, `ctlClient` in `internal/cli/ctl.go`.