From 457a1eefac502b79379fb60f649d726ed45fa44a Mon Sep 17 00:00:00 2001 From: Ilker Manap Date: Thu, 8 Oct 2026 11:35:38 +0200 Subject: [PATCH] Backlog F: company documents through retrieval, isolated per tenant, then adapters Co-Authored-By: Claude Opus 5.5 (1M context) --- BACKLOG.md | 72 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/BACKLOG.md b/BACKLOG.md index 073ff4d..2427092 100644 --- a/BACKLOG.md +++ b/BACKLOG.md @@ -818,6 +818,78 @@ The same document, edited on /admin/settings, for whoever has no terminal. --- +## F. Company documents: knowledge bases (RAG), then adapters (2026-10-08) + +A company wants a model to answer from its own documents. Retrieval is the +tool for *knowledge* (sources shown, re-indexed in minutes, access per +tenant); LoRA training is for *behaviour* (vocabulary, format) and has no +access control — whatever goes into weights reaches everyone who may use the +adapter. RAG first. Decisions taken: access per collection (no per-document +ACL yet); a personal key or session names a project or team and gets that +chain only, never the union of its person's memberships; PostgreSQL store +only, off on SQLite with a sentence saying why. + +### F1. Store: collections, grants, chunks (M) +- `pgvector` ≥ 0.8 installed on every Patroni member by `server init` + (`internal/cli/serverinit_postgres.go`), `CREATE EXTENSION` in the + migration; the SQLite migration has no tables, and the store reports the + feature unavailable. +- Tables: `kb_collections` (id, org_id, owner `team:…`/`project:…`/`user:…`, + name, embed_model, dims, created_at), `kb_grants` (scope, scope_id, + collection, perm read|write — like `guard_attachments`), `kb_documents` + (name, digest, bytes, state, chunk count), `kb_chunks` (org_id, + collection, doc_id, ord, body, tsv, embedding) `PARTITION BY LIST + (org_id)`, a partition per organization, HNSW and GIN indexes per + partition. +- Row-level security on `kb_chunks` against `current_setting('llmgw.org')` + and `llmgw.collections`, retrieval through its own role. +- Rules, refused before anything is written: a grant never crosses + organizations; owner and grants resolve to existing scopes; deleting a + project/team/org and `users forget` delete their collections (`forget.go`). +- Done: store tests on PostgreSQL, including an RLS test that a query + without the settings returns no rows. + +### F2. Ingest (M) +- `llmgw ctl kb create|list|delete|grant|revoke|docs`, `ctl kb add + `, admin API under `/api/v1/kb`, audit events. +- Text from PDF (text layer), DOCX, HTML, Markdown, plain text — in Go, no + Python. Split on headings/paragraphs, ~500 tokens with overlap. Embedded + through the router at the bronze band, accounted to the collection's + owner. One transaction per document; same digest skipped, changed digest + replaces. OCR out of scope. +- Console: a Knowledge screen (collections, documents, grants; add, grant, + revoke, delete); `docs/console.md`, `docs/knowledge.md`. + +### F3. Retrieval on the request path (M) +- `llmgw.knowledge: [collections]` in the body (stripped before the + engine), or default collections per project/key. +- Allowed set from the caller's chain (§7.9.2), cached like limits; grants + add up down the tree. Personal keys and sessions need `x-llmgw-project` + (or the playground's picker) naming a scope the person is a member of. +- A collection not allowed answers 404, exactly as a missing one. +- Hybrid search with the filter inside the query (`hnsw.iterative_scan`, + `WHERE org_id = $1 AND collection = ANY($2)`), full-text arm merged by + reciprocal rank, `/v1/rerank` when a reranker is in the catalog. Top + chunks into a system message with document and position; + `x-llmgw-sources` lists them. +- Leaks closed: answer-cache key includes the allowed set; per-organization + `cache_salt` on RAG requests; injected passages under `no_store` and the + guard's prompt-injection check; returned doc ids in the audit. +- Done: router tests that tenant B never receives tenant A's chunks + (including via cache), that a personal key without a project is refused, + metrics (retrieval latency, chunks returned), docs. + +### F4. Adapter training (L, after F1–F3) +A run-to-completion job placed by the control plane like a replica and +supervised by the agent like an engine, in a Python environment the agent +installs (TRL + PEFT, QLoRA). Data: Q/A pairs generated from a collection by +a cluster model through `/v1/batches`, reviewed by people. Output to the +mirror, into the model's `adapters`, through canary and a quality suite. +Runs when the scaling schedule frees a GPU; 8–14B dense bases on one card. +Never train on a collection some users of the adapter may not read. + +--- + ## D. Facts worth knowing before starting - Admin API: `127.0.0.1:9090`, no auth, `ctlClient` in `internal/cli/ctl.go`.