Knowledge bases in the store: collections, grants and chunks kept apart per tenant
Collections owned by an organization, team, project or person; grants that add up down the tree and never cross an organization; chunks in PostgreSQL with pgvector, partitioned by organization, an HNSW index per collection, and row-level security as a second line behind each query's filter. Search fuses nearest-neighbour and full-text ranks. Off on SQLite and without pgvector, saying why. server init and the HA database setup install and enable it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
0a7fb821d8
commit
d35630defe
11 files changed
+1408
-25
No files matched your search
@@ -0,0 +1,55 @@
|
||||
-- Knowledge bases: collections of a company's documents that requests are
|
||||
-- answered from (BACKLOG F). PostgreSQL only: there is no SQLite file of this
|
||||
-- number, and on SQLite the feature is off.
|
||||
--
|
||||
-- The chunks themselves are not here. They need the pgvector extension, which
|
||||
-- only a superuser can create and which a cluster may not have installed; a
|
||||
-- migration that failed for want of it would stop the control plane from
|
||||
-- starting at all. The store creates kb_chunks when it opens, if it can
|
||||
-- (knowledge.go), and says why not when it cannot.
|
||||
|
||||
-- A collection has one owner in the tenancy tree and lives inside one
|
||||
-- organization, which no grant can cross.
|
||||
CREATE TABLE kb_collections (
|
||||
id TEXT PRIMARY KEY,
|
||||
org_id TEXT NOT NULL REFERENCES organizations(id),
|
||||
owner_scope TEXT NOT NULL,
|
||||
owner_id TEXT NOT NULL,
|
||||
name TEXT NOT NULL,
|
||||
-- Vectors from two models are not comparable, so a collection is embedded
|
||||
-- by one model and its dimension is fixed with it.
|
||||
embed_model TEXT NOT NULL,
|
||||
dims INTEGER NOT NULL,
|
||||
created_at BIGINT NOT NULL,
|
||||
created_by TEXT NOT NULL DEFAULT '',
|
||||
UNIQUE (org_id, name)
|
||||
);
|
||||
CREATE INDEX kb_collections_by_owner ON kb_collections(owner_scope, owner_id);
|
||||
|
||||
-- Who else may read or write a collection. Like guard_attachments, keyed by
|
||||
-- scope with everything below the scope included; unlike guard policies,
|
||||
-- grants add up rather than narrow.
|
||||
CREATE TABLE kb_grants (
|
||||
scope TEXT NOT NULL,
|
||||
scope_id TEXT NOT NULL,
|
||||
collection TEXT NOT NULL REFERENCES kb_collections(id) ON DELETE CASCADE,
|
||||
perm TEXT NOT NULL,
|
||||
added_at BIGINT NOT NULL,
|
||||
added_by TEXT NOT NULL DEFAULT '',
|
||||
PRIMARY KEY (scope, scope_id, collection)
|
||||
);
|
||||
CREATE INDEX kb_grants_by_collection ON kb_grants(collection);
|
||||
|
||||
-- A document as added, by name within its collection. The digest is what
|
||||
-- decides whether adding it again changes anything.
|
||||
CREATE TABLE kb_documents (
|
||||
id TEXT PRIMARY KEY,
|
||||
collection TEXT NOT NULL REFERENCES kb_collections(id) ON DELETE CASCADE,
|
||||
name TEXT NOT NULL,
|
||||
digest TEXT NOT NULL,
|
||||
bytes BIGINT NOT NULL,
|
||||
chunks INTEGER NOT NULL,
|
||||
added_at BIGINT NOT NULL,
|
||||
added_by TEXT NOT NULL DEFAULT '',
|
||||
UNIQUE (collection, name)
|
||||
);
|
||||
Reference in new issue
Block a user