Knowledge bases at scale: halfvec, a text index per collection, ef_search raised to the candidates wanted

A search asking for more than 10 passages got at most 40 candidates from
HNSW, pgvector's default ef_search. The full-text arm scored every chunk
holding a word of the question before limiting, over the whole
organization's GIN index; it now takes candidates by ts_rank from the
collection's own partial index and scores those. New collections index
halfvec on pgvector 0.7 and later (up to 4000 dimensions).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 22:08:29 +02:00
1 parent 42d3b7ea22
commit 9c675e7293
7 files changed
+223 -26

No files matched your search

+3
View File
@@ -52,6 +52,9 @@ type DB struct {
// kbErr is why this store cannot hold knowledge bases, nil when it can;
// set once at open (knowledge.go).
kbErr error
// kbHalfvec is pgvector 0.7 or later here: new collections index their
// vectors as halfvec.
kbHalfvec bool
}
// OpenSQLite opens (creating if needed) a database and brings it up to date.