Training data from a collection: llmgw ctl kb dataset

A chat model of the cluster reads each chunk and writes questions a person
might ask of it with the answers it holds, streamed as NDJSON a chunk at a
time, for people to correct before an adapter is trained on it. A chunk that
gave no pairs says so on its own line.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 20:04:37 +02:00
1 parent 0a6a586dfe
commit ae95ac8261
8 files changed
+455

No files matched your search

+2
View File
@@ -30,6 +30,7 @@ type KBStore interface {
DeleteDocument(ctx context.Context, collection, name string) (bool, error)
SearchChunks(ctx context.Context, q store.KBSearch) ([]store.KBHit, error)
SetKBDefault(ctx context.Context, d store.KBDefault) error
CollectionChunks(ctx context.Context, c store.KBCollection, limit int) ([]store.KBHit, error)
KBDefaults(ctx context.Context) ([]store.KBDefault, error)
}
@@ -161,6 +162,7 @@ func (a *API) registerKB(mux Mux) {
mux.HandleFunc("DELETE /api/v1/kb/grants", a.kbGrant(false))
mux.HandleFunc("POST /api/v1/kb/search", a.kbSearch)
mux.HandleFunc("POST /api/v1/kb/defaults", a.kbSetDefault)
mux.HandleFunc("POST /api/v1/kb/collections/{id}/dataset", a.kbDataset)
}
// kbReady answers for a gateway without knowledge bases, and says whether