Backlog F: company documents through retrieval, isolated per tenant, then adapters

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 11:35:38 +02:00
1 parent 0bf7e46fbf
commit 457a1eefac
1 file changed
+72
+72
View File
@@ -818,6 +818,78 @@ The same document, edited on /admin/settings, for whoever has no terminal.
---
## F. Company documents: knowledge bases (RAG), then adapters (2026-10-08)
A company wants a model to answer from its own documents. Retrieval is the
tool for *knowledge* (sources shown, re-indexed in minutes, access per
tenant); LoRA training is for *behaviour* (vocabulary, format) and has no
access control — whatever goes into weights reaches everyone who may use the
adapter. RAG first. Decisions taken: access per collection (no per-document
ACL yet); a personal key or session names a project or team and gets that
chain only, never the union of its person's memberships; PostgreSQL store
only, off on SQLite with a sentence saying why.
### F1. Store: collections, grants, chunks (M)
- `pgvector` ≥ 0.8 installed on every Patroni member by `server init`
(`internal/cli/serverinit_postgres.go`), `CREATE EXTENSION` in the
migration; the SQLite migration has no tables, and the store reports the
feature unavailable.
- Tables: `kb_collections` (id, org_id, owner `team:…`/`project:…`/`user:…`,
name, embed_model, dims, created_at), `kb_grants` (scope, scope_id,
collection, perm read|write — like `guard_attachments`), `kb_documents`
(name, digest, bytes, state, chunk count), `kb_chunks` (org_id,
collection, doc_id, ord, body, tsv, embedding) `PARTITION BY LIST
(org_id)`, a partition per organization, HNSW and GIN indexes per
partition.
- Row-level security on `kb_chunks` against `current_setting('llmgw.org')`
and `llmgw.collections`, retrieval through its own role.
- Rules, refused before anything is written: a grant never crosses
organizations; owner and grants resolve to existing scopes; deleting a
project/team/org and `users forget` delete their collections (`forget.go`).
- Done: store tests on PostgreSQL, including an RLS test that a query
without the settings returns no rows.
### F2. Ingest (M)
- `llmgw ctl kb create|list|delete|grant|revoke|docs`, `ctl kb add
<collection> <files…>`, admin API under `/api/v1/kb`, audit events.
- Text from PDF (text layer), DOCX, HTML, Markdown, plain text — in Go, no
Python. Split on headings/paragraphs, ~500 tokens with overlap. Embedded
through the router at the bronze band, accounted to the collection's
owner. One transaction per document; same digest skipped, changed digest
replaces. OCR out of scope.
- Console: a Knowledge screen (collections, documents, grants; add, grant,
revoke, delete); `docs/console.md`, `docs/knowledge.md`.
### F3. Retrieval on the request path (M)
- `llmgw.knowledge: [collections]` in the body (stripped before the
engine), or default collections per project/key.
- Allowed set from the caller's chain (§7.9.2), cached like limits; grants
add up down the tree. Personal keys and sessions need `x-llmgw-project`
(or the playground's picker) naming a scope the person is a member of.
- A collection not allowed answers 404, exactly as a missing one.
- Hybrid search with the filter inside the query (`hnsw.iterative_scan`,
`WHERE org_id = $1 AND collection = ANY($2)`), full-text arm merged by
reciprocal rank, `/v1/rerank` when a reranker is in the catalog. Top
chunks into a system message with document and position;
`x-llmgw-sources` lists them.
- Leaks closed: answer-cache key includes the allowed set; per-organization
`cache_salt` on RAG requests; injected passages under `no_store` and the
guard's prompt-injection check; returned doc ids in the audit.
- Done: router tests that tenant B never receives tenant A's chunks
(including via cache), that a personal key without a project is refused,
metrics (retrieval latency, chunks returned), docs.
### F4. Adapter training (L, after F1–F3)
A run-to-completion job placed by the control plane like a replica and
supervised by the agent like an engine, in a Python environment the agent
installs (TRL + PEFT, QLoRA). Data: Q/A pairs generated from a collection by
a cluster model through `/v1/batches`, reviewed by people. Output to the
mirror, into the model's `adapters`, through canary and a quality suite.
Runs when the scaling schedule frees a GPU; 8–14B dense bases on one card.
Never train on a collection some users of the adapter may not read.
---
## D. Facts worth knowing before starting
- Admin API: `127.0.0.1:9090`, no auth, `ctlClient` in `internal/cli/ctl.go`.