Collections a part of the tree answers from when a request names none

On an organization, team, project or key, the narrowest winning: llmgw ctl
kb default, POST /api/v1/kb/defaults, and the console's Knowledge screen. A
request's own X-Llmgw-Knowledge overrides it and none turns it off. A default
never refuses a request: a collection the caller cannot read is left out, and
a failed search answers without passages. Removing a collection takes it out
of every default.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 15:52:43 +02:00
1 parent 54ba04c52a
commit 0093cc3d45
16 files changed
+552 -12

No files matched your search

+3 -3
View File
@@ -946,7 +946,7 @@ PostgreSQL tests of store, router and adminapi share LLMGW_TEST_POSTGRES and
the store's TRUNCATE each test; run them with `go test -p 1` or one package
at a time, or a router test loses its rows mid-run.
### F3. Retrieval on the request path (M) — done 2026-10-08 (branch `rag`), but defaults and cache_salt
### F3. Retrieval on the request path (M) — done 2026-10-08 (branch `rag`)
- `llmgw.knowledge: [collections]` in the body (stripped before the
engine), or default collections per project/key.
- Allowed set from the caller's chain (§7.9.2), cached like limits; grants
@@ -972,8 +972,8 @@ collections from the key's chain (kbEffective, shared with the kb
endpoints), 404 for one not readable; the question embedded through the
ordinary path on the caller's principal; SearchChunks; passages trimmed to
the model's context; added to the one leading system message (or a new
one); `x-llmgw-sources`; before guardInput; never cached; metrics. Left:
default collections per project or key (none yet: a client must name them);
one); `x-llmgw-sources`; before guardInput; never cached; metrics. Default collections built (kb_defaults on org/team/project/key, narrowest
wins, 30 s cache, `ctl kb default`, console action; never refuse). Left:
`cache_salt` per organization (not sent: llama.cpp may refuse an unknown
field, and the router does not know the engine before choosing the
replica); the chat playground has no collection picker; returned passages
+93
View File
@@ -1275,6 +1275,49 @@
}
}
},
"/api/v1/kb/defaults": {
"post": {
"operationId": "postKbDefaults",
"summary": "Set the collections a part of the tree answers from when a request names none",
"description": "On an organization, team, project or key; the narrowest one with a default wins. No collections clears it. A request's own X-Llmgw-Knowledge overrides it, and X-Llmgw-Knowledge: none turns it off.",
"tags": [
"kb"
],
"parameters": [
{
"$ref": "#/components/parameters/Actor"
}
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/KBDefaultRequest"
}
}
}
},
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/KBDefaultView"
}
}
}
},
"401": {
"$ref": "#/components/responses/Unauthorized"
},
"default": {
"$ref": "#/components/responses/Error"
}
}
}
},
"/api/v1/kb/grants": {
"post": {
"operationId": "postKbGrants",
@@ -5016,6 +5059,49 @@
}
}
},
"KBDefaultRequest": {
"type": "object",
"description": "KBDefaultRequest sets the collections a part of the tree answers from when a request names none; none clears it.",
"properties": {
"target": {
"type": "string",
"description": "Target is org:, team:, project: (by path or ID) or key:KEY-ID."
},
"collections": {
"type": "array",
"items": {
"type": "string"
},
"description": "Collections are IDs, names when unique, or ORG/NAME."
}
}
},
"KBDefaultView": {
"type": "object",
"description": "KBDefaultView is one part of the tree's default collections.",
"properties": {
"target": {
"type": "string"
},
"name": {
"type": "string"
},
"collections": {
"type": "array",
"items": {
"type": "string"
},
"description": "Collections are ORG/NAME, in order."
},
"at": {
"type": "string",
"format": "date-time"
},
"by": {
"type": "string"
}
}
},
"KBDocumentView": {
"type": "object",
"description": "KBDocumentView is one document in a collection.",
@@ -5131,6 +5217,13 @@
"items": {
"$ref": "#/components/schemas/KBCollectionView"
}
},
"defaults": {
"type": "array",
"items": {
"$ref": "#/components/schemas/KBDefaultView"
},
"description": "Defaults are the collections parts of the tree answer from when a request names none."
}
}
},
+2 -1
View File
@@ -168,7 +168,8 @@ team or project is guaranteed and may at most hold, and what it holds now.
model that embeds it, how many documents and chunks it holds and who else it
is granted to. **A** makes one. On a row: its **Documents**, **Grant…** a
project, team, organization, person or key read or write access, **Revoke**
a grant, **Try a search…** as an administrator, or **Delete…** it. People
a grant, **Answer from it by default…** for a part of the tree, **Try a
search…** as an administrator, or **Delete…** it. People
fill collections from their own machines with `llmgw-kb`
([knowledge.md](knowledge.md)). Off on a SQLite store, and says why.
+26 -1
View File
@@ -88,6 +88,31 @@ How access works:
is answered exactly as one that does not exist, so even its name does not
leak.
### Answer from collections by default
Clients that cannot add a header still get answers from the documents when
their part of the tree has a default:
```sh
llmgw ctl kb default team:Acme/hr Acme/policies Acme/forms
llmgw ctl kb default key:KEY-ID Acme/policies
llmgw ctl kb default team:Acme/hr --clear
```
- **Which default applies.** The narrowest one: a key's over its project's,
a project's over its team's, a team's over its organization's.
- **A request can override it.** A request that names collections uses
those instead, and `X-Llmgw-Knowledge: none` turns the default off for
that request.
- **A default never refuses a request.** A collection the caller may not
read is left out for them. If the search fails (the embedding model is
down, say), the request is answered without passages and the failure is
logged.
- **Changes take effect within 30 seconds.**
In the console: **Knowledge**, Enter on a collection, then **Answer from it
by default…**.
### Look after it
```sh
@@ -274,7 +299,7 @@ How it is shaped:
| | |
|---|---|
| `X-Llmgw-Knowledge` | Collections, comma separated. |
| `X-Llmgw-Knowledge` | Collections, comma separated; `none` for no passages, whatever the default. |
| `X-Llmgw-Knowledge-K` | How many passages, 1 to 20; 5 by default. |
| `X-Llmgw-Project` | For a personal key: the project it acts in. |
| `x-llmgw-sources` (answer) | The passages given, as `document#chunk`, in their numbered order; empty when nothing matched. |
+44 -1
View File
@@ -57,7 +57,18 @@ func newKBCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
fmt.Fprintf(w, "%s/%s\t%s\t%s (%d)\t%d\t%d\t%s\n", or(c.OrgName, c.Org), c.Name, or(c.OwnerName, c.Owner),
c.EmbedModel, c.Dims, c.Documents, c.Chunks, strings.Join(grants, ", "))
}
return w.Flush()
if err := w.Flush(); err != nil {
return err
}
if len(v.Defaults) > 0 {
fmt.Fprintln(out, "\nAnswered from by default, when a request names no collection:")
w = tab()
for _, d := range v.Defaults {
fmt.Fprintf(w, " %s\t%s\n", or(d.Name, d.Target), strings.Join(d.Collections, ", "))
}
return w.Flush()
}
return nil
},
})
@@ -169,6 +180,38 @@ func newKBCmd(client func() *ctlClient, asJSON *bool) *cobra.Command {
},
})
var clear bool
def := &cobra.Command{
Use: "default TARGET [COLLECTION...]",
Short: "Answer a part of the tree's chat requests from collections, when they name none",
Long: "Requests from TARGET and everything below it are then answered from these\n" +
"collections without naming them (X-Llmgw-Knowledge). The narrowest default\n" +
"wins: a key's over its project's, a project's over its team's. A request's own\n" +
"header overrides it, and X-Llmgw-Knowledge: none turns it off. A collection a\n" +
"caller may not read is left out for them, and a search that fails answers\n" +
"without passages rather than refusing. In force within 30 seconds.",
Example: " llmgw ctl kb default team:Acme/hr Acme/policies Acme/forms\n" +
" llmgw ctl kb default team:Acme/hr --clear",
Args: cobra.MinimumNArgs(1),
RunE: func(cmd *cobra.Command, args []string) error {
if clear != (len(args) == 1) {
return fmt.Errorf("name collections, or --clear")
}
body, _ := json.Marshal(adminapi.KBDefaultRequest{Target: args[0], Collections: args[1:]})
if err := client().do(cmd.Context(), "POST", "/api/v1/kb/defaults", json.RawMessage(body), nil); err != nil {
return err
}
if clear {
fmt.Fprintf(cmd.OutOrStdout(), "%s answers from no collection by default\n", args[0])
} else {
fmt.Fprintf(cmd.OutOrStdout(), "%s answers from %s by default\n", args[0], strings.Join(args[1:], ", "))
}
return nil
},
}
def.Flags().BoolVar(&clear, "clear", false, "no default any more")
c.AddCommand(def)
var k int
search := &cobra.Command{
Use: "search COLLECTION QUERY...",
+25
View File
@@ -75,6 +75,14 @@ func (m consoleModel) kbPanel(pw int) (string, string, string) {
fmt.Sprintf("%s (%d)", c.EmbedModel, c.Dims), fmt.Sprint(c.Documents), fmt.Sprint(c.Chunks), dash(strings.Join(grants, ", "))})
}
b.WriteString(table([]string{"COLLECTION", "OWNER", "MODEL", "DOCUMENTS", "CHUNKS", "GRANTED TO"}, rows, m.row, pw, m.inPanel))
if len(k.view.Defaults) > 0 {
b.WriteString("\n\n " + sTitle.Render("BY DEFAULT") + " " + sDim.Render("when a request names no collection") + "\n")
var drows [][]string
for _, d := range k.view.Defaults {
drows = append(drows, []string{or(d.Name, d.Target), strings.Join(d.Collections, ", ")})
}
b.WriteString(table([]string{"FOR", "ANSWERED FROM"}, drows, -1, pw, false))
}
b.WriteString("\n\n " + sDim.Render("On a person's machine: llmgw-kb sync COLLECTION FOLDER"))
return "Knowledge", b.String(), keys
}
@@ -153,6 +161,23 @@ func (m consoleModel) kbMenu(c adminapi.KBCollectionView) *menuDialog {
}})
}
items = append(items,
menuItem{"Answer from it by default…", func(m *consoleModel) tea.Cmd {
m.openForm(&formDialog{title: "Answer from " + name + " by default",
note: "Chat requests from there that name no collection are answered from this one.\n" +
"Replaces that part's default; llmgw ctl kb default sets several.",
fields: []formField{
choiceF("for", []string{"project", "team", "org", "key"}, "team"),
textF("which", "", "ORG/TEAM/PROJECT, ORG/TEAM, ORG, or the ID"),
},
submit: func(m *consoleModel, v []string) (tea.Cmd, error) {
if strings.TrimSpace(v[1]) == "" {
return nil, fmt.Errorf("which %s?", v[0])
}
req := adminapi.KBDefaultRequest{Target: v[0] + ":" + strings.TrimSpace(v[1]), Collections: []string{c.ID}}
return m.call("Default "+c.Name, http.MethodPost, "/api/v1/kb/defaults", req, nil), nil
}})
return nil
}},
menuItem{"Try a search…", func(m *consoleModel) tea.Cmd {
m.openForm(&formDialog{title: "Search " + name,
note: "As an administrator: no tenancy applies. Shows what a question would find.",
+75 -3
View File
@@ -29,6 +29,8 @@ type KBStore interface {
Documents(ctx context.Context, collection string) ([]store.KBDocument, error)
DeleteDocument(ctx context.Context, collection, name string) (bool, error)
SearchChunks(ctx context.Context, q store.KBSearch) ([]store.KBHit, error)
SetKBDefault(ctx context.Context, d store.KBDefault) error
KBDefaults(ctx context.Context) ([]store.KBDefault, error)
}
// KBAdmin is what the knowledge-base endpoints need: the store, and a way to
@@ -47,6 +49,28 @@ type KBView struct {
// Reason is why not, when not.
Reason string `json:"reason,omitempty"`
Collections []KBCollectionView `json:"collections"`
// Defaults are the collections parts of the tree answer from when a
// request names none.
Defaults []KBDefaultView `json:"defaults"`
}
// KBDefaultView is one part of the tree's default collections.
type KBDefaultView struct {
Target string `json:"target"`
Name string `json:"name,omitempty"`
// Collections are ORG/NAME, in order.
Collections []string `json:"collections"`
At time.Time `json:"at"`
By string `json:"by,omitempty"`
}
// KBDefaultRequest sets the collections a part of the tree answers from
// when a request names none; none clears it.
type KBDefaultRequest struct {
// Target is org:, team:, project: (by path or ID) or key:KEY-ID.
Target string `json:"target"`
// Collections are IDs, names when unique, or ORG/NAME.
Collections []string `json:"collections"`
}
// KBCollectionView is a collection as an administrator sees it.
@@ -136,6 +160,7 @@ func (a *API) registerKB(mux Mux) {
mux.HandleFunc("POST /api/v1/kb/grants", a.kbGrant(true))
mux.HandleFunc("DELETE /api/v1/kb/grants", a.kbGrant(false))
mux.HandleFunc("POST /api/v1/kb/search", a.kbSearch)
mux.HandleFunc("POST /api/v1/kb/defaults", a.kbSetDefault)
}
// kbReady answers for a gateway without knowledge bases, and says whether
@@ -165,11 +190,11 @@ func (a *API) names(ctx context.Context) map[string]string {
func (a *API) kbList(w http.ResponseWriter, r *http.Request) {
if a.kb == nil {
writeJSON(w, http.StatusOK, KBView{Reason: "knowledge bases need the store", Collections: []KBCollectionView{}})
writeJSON(w, http.StatusOK, KBView{Reason: "knowledge bases need the store", Collections: []KBCollectionView{}, Defaults: []KBDefaultView{}})
return
}
if err := a.kb.Store.KnowledgeAvailable(); err != nil {
writeJSON(w, http.StatusOK, KBView{Reason: err.Error(), Collections: []KBCollectionView{}})
writeJSON(w, http.StatusOK, KBView{Reason: err.Error(), Collections: []KBCollectionView{}, Defaults: []KBDefaultView{}})
return
}
cs, err := a.kb.Store.Collections(r.Context())
@@ -183,7 +208,7 @@ func (a *API) kbList(w http.ResponseWriter, r *http.Request) {
return
}
names := a.names(r.Context())
v := KBView{Available: true, Collections: []KBCollectionView{}}
v := KBView{Available: true, Collections: []KBCollectionView{}, Defaults: []KBDefaultView{}}
for _, c := range cs {
cv := KBCollectionView{ID: c.ID, Name: c.Name, Org: c.OrgID, OrgName: names["org:"+c.OrgID],
Owner: string(c.OwnerScope) + ":" + c.OwnerID, OwnerName: names[string(c.OwnerScope)+":"+c.OwnerID],
@@ -202,6 +227,23 @@ func (a *API) kbList(w http.ResponseWriter, r *http.Request) {
}
v.Collections = append(v.Collections, cv)
}
label := map[string]string{}
for _, c := range cs {
label[c.ID] = or(names["org:"+c.OrgID], c.OrgID) + "/" + c.Name
}
defaults, err := a.kb.Store.KBDefaults(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error())
return
}
for _, d := range defaults {
key := string(d.Scope) + ":" + d.ScopeID
dv := KBDefaultView{Target: key, Name: names[key], Collections: []string{}, At: d.AddedAt, By: d.AddedBy}
for _, id := range d.Collections {
dv.Collections = append(dv.Collections, or(label[id], id))
}
v.Defaults = append(v.Defaults, dv)
}
writeJSON(w, http.StatusOK, v)
}
@@ -471,3 +513,33 @@ func (a *API) kbSearch(w http.ResponseWriter, r *http.Request) {
}
writeJSON(w, http.StatusOK, out)
}
func (a *API) kbSetDefault(w http.ResponseWriter, r *http.Request) {
if !a.kbReady(w) {
return
}
var in KBDefaultRequest
if err := json.NewDecoder(r.Body).Decode(&in); err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
scope, id, err := a.kbTarget(r.Context(), in.Target)
if err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
d := store.KBDefault{Scope: scope, ScopeID: id, AddedBy: actorOf(r)}
for _, ref := range in.Collections {
c, err := a.kbFind(r.Context(), ref)
if err != nil {
writeError(w, http.StatusNotFound, err.Error())
return
}
d.Collections = append(d.Collections, c.ID)
}
if err := a.kb.Store.SetKBDefault(r.Context(), d); err != nil {
writeError(w, http.StatusBadRequest, err.Error())
return
}
writeJSON(w, http.StatusOK, KBDefaultView{Target: string(scope) + ":" + id, Collections: in.Collections, At: time.Now(), By: actorOf(r)})
}
+4
View File
@@ -284,6 +284,10 @@ var Routes = []Route{
Request: KBGrantRequest{}, Response: KBGrantView{}},
{Method: "DELETE", Path: "/api/v1/kb/grants", Summary: "Take a grant away",
Request: KBGrantRequest{}, Response: KBGrantView{}},
{Method: "POST", Path: "/api/v1/kb/defaults", Summary: "Set the collections a part of the tree answers from when a request names none",
Description: "On an organization, team, project or key; the narrowest one with a default wins. No collections clears it. " +
"A request's own X-Llmgw-Knowledge overrides it, and X-Llmgw-Knowledge: none turns it off.",
Request: KBDefaultRequest{}, Response: KBDefaultView{}},
{Method: "POST", Path: "/api/v1/kb/search", Summary: "Try a search of a collection, as an administrator",
Description: "No tenancy applies: this is for checking what a collection holds and finds.",
Request: KBSearchRequest{}, Response: []KBHitView{}},
+1 -1
View File
@@ -253,7 +253,7 @@ func reusable(req *parsedRequest, anyAnswer bool) bool {
}
// An answer from knowledge bases is the caller's own: two keys of one
// project can read different collections.
if req.Header.Get(knowledgeHeader) != "" {
if req.knowledge || req.Header.Get(knowledgeHeader) != "" {
return false
}
if v := strings.ToLower(strings.TrimSpace(req.Header.Get(cacheHeader))); v == "no" || v == "off" || v == "bypass" {
+2
View File
@@ -407,6 +407,8 @@ type parsedRequest struct {
reservation accounting.Reservation
// guardIn is what the guard decided about the request.
guardIn guard.Verdict
// knowledge is set when passages from knowledge bases were added.
knowledge bool
// noStore is a policy that keeps nothing of this request.
noStore bool
// inCluster is a project whose requests may not leave the cluster.
+69 -1
View File
@@ -82,10 +82,34 @@ func (e *kbError) Error() string { return e.msg }
// when it has answered the client itself.
func (r *Router) retrieve(ctx context.Context, w http.ResponseWriter, req *parsedRequest, start time.Time) bool {
names := splitList(req.Header.Get(knowledgeHeader))
if len(names) == 0 {
if len(names) == 1 && strings.EqualFold(names[0], "none") {
// Asked for no passages, whatever the default.
return true
}
// Without the header, the default of the caller's part of the tree, if
// it has one. A default is the administrator's choice, not the
// caller's, so it never refuses a request: a collection the caller
// cannot read is left out, and a search that fails is answered without
// passages.
explicit := len(names) > 0
if !explicit {
if req.Path != "/v1/chat/completions" || req.internal || r.store == nil || r.store.KnowledgeAvailable() != nil {
return true
}
p, ok := auth.FromContext(ctx)
if !ok {
return true
}
if names = r.kbDefaults(ctx, p, req.Header.Get(projectHeader)); len(names) == 0 {
return true
}
}
fail := func(err error) bool {
if !explicit {
r.log.Warn("a knowledge-base default could not be searched; answering without it", "model", req.Model, "err", err)
r.m.kbRetrievals.WithLabelValues("skipped").Inc()
return true
}
var ke *kbError
if !errors.As(err, &ke) {
ke = &kbError{status: http.StatusInternalServerError, kind: "server_error", msg: err.Error()}
@@ -131,6 +155,9 @@ func (r *Router) retrieve(ctx context.Context, w http.ResponseWriter, req *parse
var colls []storepkg.KBCollection
for _, n := range names {
c, ok := pickCollection(allowed, n)
if !ok && !explicit {
continue
}
if !ok {
msg := "no collection " + n
if e.OrgID == "" {
@@ -140,8 +167,14 @@ func (r *Router) retrieve(ctx context.Context, w http.ResponseWriter, req *parse
}
colls = append(colls, c)
}
if len(colls) == 0 {
return true
}
question := lastUserText(req.UpstreamBody)
if strings.TrimSpace(question) == "" && !explicit {
return true
}
if strings.TrimSpace(question) == "" {
return fail(&kbError{status: http.StatusBadRequest, kind: "invalid_request_error",
msg: knowledgeHeader + " needs a last user message with text to search for"})
@@ -189,6 +222,7 @@ func (r *Router) retrieve(ctx context.Context, w http.ResponseWriter, req *parse
}
req.UpstreamBody = body
req.PromptTokens = len(body) / 4
req.knowledge = true
// Set now: the proxy writes the headers when the answer starts.
w.Header().Set(sourcesHeader, strings.Join(sources, ","))
result := "found"
@@ -357,3 +391,37 @@ func withSystemText(body []byte, text string) ([]byte, error) {
}
return json.Marshal(top)
}
// kbDefaultTTL is how long a caller's default is remembered: a change to one
// is in force within it, and a chat request does not cost a query.
const kbDefaultTTL = 30 * time.Second
type kbDefaultEntry struct {
at time.Time
names []string
}
// kbDefaults is the default collections in force for a caller.
func (r *Router) kbDefaults(ctx context.Context, p auth.Principal, project string) []string {
key := p.KeyID + "|" + p.ProjectID + "|" + project
r.kbDefMu.Lock()
e, ok := r.kbDefCache[key]
r.kbDefMu.Unlock()
if ok && time.Since(e.at) < kbDefaultTTL {
return e.names
}
var names []string
if eff, err := r.kbEffective(ctx, p, project); err == nil {
names, err = r.store.KBDefaultFor(ctx, eff)
if err != nil {
r.log.Warn("could not read the knowledge-base defaults", "err", err)
}
}
r.kbDefMu.Lock()
if r.kbDefCache == nil || len(r.kbDefCache) > 10000 {
r.kbDefCache = map[string]kbDefaultEntry{}
}
r.kbDefCache[key] = kbDefaultEntry{at: time.Now(), names: names}
r.kbDefMu.Unlock()
return names
}
+35 -1
View File
@@ -113,7 +113,9 @@ func TestAChatIsAnsweredFromItsCollections(t *testing.T) {
req, _ := http.NewRequest(http.MethodPost, srv.URL+"/v1/chat/completions", strings.NewReader(
`{"model":"qwen2.5-14b-instruct","messages":[{"role":"system","content":"Kısa yaz."},{"role":"user","content":"kaç gün yıllık izin var"}]}`))
req.Header.Set("Authorization", "Bearer "+key)
req.Header.Set(knowledgeHeader, collections)
if collections != "" {
req.Header.Set(knowledgeHeader, collections)
}
req.Header.Set(knowledgeKHeader, "1")
res, err := http.DefaultClient.Do(req)
if err != nil {
@@ -140,6 +142,23 @@ func TestAChatIsAnsweredFromItsCollections(t *testing.T) {
t.Errorf("the model was sent:\n%s", sent)
}
// A project default answers a request that names nothing, and none
// turns it off.
if err := s.SetKBDefault(ctx, store.KBDefault{Scope: store.ScopeProject, ScopeID: proj,
Collections: []string{mustID(t, s, "politika-"+sfx)}}); err != nil {
t.Fatal(err)
}
if res := chat(key, ""); res.Header.Get(sourcesHeader) != "izin.md#0" {
t.Errorf("by default: sources %q", res.Header.Get(sourcesHeader))
}
if res := chat(key, "none"); res.Header.Get(sourcesHeader) != "" {
t.Errorf("with none: sources %q", res.Header.Get(sourcesHeader))
}
sent, _ = e.last.Load().([]byte)
if strings.Contains(string(sent), "Below are passages") {
t.Errorf("none still added passages:\n%s", sent)
}
// Another organization's key names it: as if it did not exist, and the
// model is not asked.
calls := e.calls.Load()
@@ -147,3 +166,18 @@ func TestAChatIsAnsweredFromItsCollections(t *testing.T) {
t.Errorf("another organization: %d, engine asked %d more times", res.StatusCode, e.calls.Load()-calls)
}
}
func mustID(t *testing.T, s *store.DB, name string) string {
t.Helper()
cs, err := s.Collections(t.Context())
if err != nil {
t.Fatal(err)
}
for _, c := range cs {
if c.Name == name {
return c.ID
}
}
t.Fatalf("no collection %s", name)
return ""
}
+6
View File
@@ -17,6 +17,7 @@ import (
"net/netip"
"slices"
"strings"
"sync"
"sync/atomic"
"time"
@@ -171,6 +172,11 @@ type Router struct {
// none is configured, which leaves the charts off the page.
history web.History
histWindow time.Duration
// kbDefCache remembers each caller's default knowledge-base
// collections for kbDefaultTTL.
kbDefMu sync.Mutex
kbDefCache map[string]kbDefaultEntry
}
// SetClusterView gives the router the data the /admin page renders.
+109
View File
@@ -369,6 +369,14 @@ func (s *DB) DeleteCollection(ctx context.Context, id string) error {
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
// Out of every default that named it; a default left empty goes.
if _, err := c.ExecContext(ctx, `UPDATE kb_defaults SET collections =
trim(both ',' from replace(',' || collections || ',', ',' || ? || ',', ','))`, id); err != nil {
return err
}
if _, err := c.ExecContext(ctx, `DELETE FROM kb_defaults WHERE collections = ''`); err != nil {
return err
}
_, err = c.ExecContext(ctx, `DROP INDEX IF EXISTS `+kbVectorIndex(id))
return err
})
@@ -860,6 +868,9 @@ func (s *DB) kbForgetScope(ctx context.Context, c conn, scope Scope, id string)
if _, err := c.ExecContext(ctx, `DELETE FROM kb_grants WHERE scope = ? AND scope_id = ?`, string(scope), id); err != nil {
return err
}
if _, err := c.ExecContext(ctx, `DELETE FROM kb_defaults WHERE scope = ? AND scope_id = ?`, string(scope), id); err != nil {
return err
}
if scope != ScopeUser {
return nil
}
@@ -917,3 +928,101 @@ func (s *DB) OwnerName(ctx context.Context, scope Scope, id string) string {
}
return name
}
// KBDefault is the collections a scope's requests answer from when they name
// none.
type KBDefault struct {
Scope Scope
ScopeID string
Collections []string
AddedAt time.Time
AddedBy string
}
// SetKBDefault makes collections a scope's default; none clears it. Each must
// be in the scope's organization.
func (s *DB) SetKBDefault(ctx context.Context, d KBDefault) error {
if err := s.KnowledgeAvailable(); err != nil {
return err
}
switch d.Scope {
case ScopeOrg, ScopeTeam, ScopeProject, ScopeKey:
default:
return fmt.Errorf("a default is set on an org, team, project or key, not a %s", d.Scope)
}
if len(d.Collections) == 0 {
_, err := s.db.ExecContext(ctx, `DELETE FROM kb_defaults WHERE scope = ? AND scope_id = ?`, string(d.Scope), d.ScopeID)
return err
}
if d.AddedAt.IsZero() {
d.AddedAt = time.Now()
}
return s.inTx(ctx, func(c conn) error {
for _, id := range d.Collections {
var org string
err := c.QueryRowContext(ctx, `SELECT org_id FROM kb_collections WHERE id = ?`, id).Scan(&org)
if errors.Is(err, sql.ErrNoRows) {
return fmt.Errorf("collection %s: %w", id, ErrNotFound)
}
if err != nil {
return err
}
got, err := s.orgOfScope(ctx, c, d.Scope, d.ScopeID, org)
if err != nil {
return err
}
if got != org {
return fmt.Errorf("collection %s is in another organization than %s %s", id, d.Scope, d.ScopeID)
}
}
_, err := c.ExecContext(ctx, `INSERT INTO kb_defaults (scope, scope_id, collections, added_at, added_by)
VALUES (?, ?, ?, ?, ?) ON CONFLICT (scope, scope_id) DO UPDATE SET collections = excluded.collections,
added_at = excluded.added_at, added_by = excluded.added_by`,
string(d.Scope), d.ScopeID, strings.Join(d.Collections, ","), millis(d.AddedAt), d.AddedBy)
return err
})
}
// KBDefaults lists every default.
func (s *DB) KBDefaults(ctx context.Context) ([]KBDefault, error) {
if err := s.KnowledgeAvailable(); err != nil {
return nil, err
}
return queryAll(ctx, s.db, `SELECT scope, scope_id, collections, added_at, added_by FROM kb_defaults ORDER BY scope, scope_id`, nil,
func(r *sql.Rows) (KBDefault, error) {
var d KBDefault
var scope, cs string
var at int64
err := r.Scan(&scope, &d.ScopeID, &cs, &at, &d.AddedBy)
d.Scope, d.AddedAt = Scope(scope), fromMillis(at)
d.Collections = strings.Split(cs, ",")
return d, err
})
}
// KBDefaultFor is the default in force for a caller: its key's, else its
// project's, team's or organization's. Nil when none.
func (s *DB) KBDefaultFor(ctx context.Context, e Effective) ([]string, error) {
if s.KnowledgeAvailable() != nil {
return nil, nil
}
for _, sc := range []struct {
scope Scope
id string
}{{ScopeKey, e.KeyID}, {ScopeProject, e.ProjectID}, {ScopeTeam, e.TeamID}, {ScopeOrg, e.OrgID}} {
if sc.id == "" {
continue
}
var cs string
err := s.db.QueryRowContext(ctx, `SELECT collections FROM kb_defaults WHERE scope = ? AND scope_id = ?`,
string(sc.scope), sc.id).Scan(&cs)
if errors.Is(err, sql.ErrNoRows) {
continue
}
if err != nil {
return nil, err
}
return strings.Split(cs, ","), nil
}
return nil, nil
}
+46
View File
@@ -398,3 +398,49 @@ func TestOrQueryMatchesAnyWord(t *testing.T) {
t.Errorf("no words gave %q", got)
}
}
func TestPostgresDefaultsAreTheNarrowestAndStayInTheirOrganization(t *testing.T) {
ctx := context.Background()
s := openKB(t)
kbTwoOrgs(t, s)
hand := mustCollection(t, s, ScopeOrg, "org1", "", "handbook")
team := mustCollection(t, s, ScopeTeam, "team1", "", "research")
foreign := mustCollection(t, s, ScopeOrg, "org2", "", "theirs")
if err := s.SetKBDefault(ctx, KBDefault{Scope: ScopeProject, ScopeID: "proj1", Collections: []string{foreign.ID}}); err == nil {
t.Error("another organization's collection became a default")
}
for _, d := range []KBDefault{
{Scope: ScopeOrg, ScopeID: "org1", Collections: []string{hand.ID}},
{Scope: ScopeTeam, ScopeID: "team1", Collections: []string{team.ID, hand.ID}},
} {
if err := s.SetKBDefault(ctx, d); err != nil {
t.Fatal(err)
}
}
in := func(e Effective) string {
got, err := s.KBDefaultFor(ctx, e)
if err != nil {
t.Fatal(err)
}
return strings.Join(got, ",")
}
if got := in(Effective{KeyID: "k", ProjectID: "proj1", TeamID: "team1", OrgID: "org1"}); got != team.ID+","+hand.ID {
t.Errorf("in team1: %q", got)
}
if got := in(Effective{KeyID: "k", ProjectID: "proj2", TeamID: "team2", OrgID: "org1"}); got != hand.ID {
t.Errorf("in team2: %q", got)
}
if got := in(Effective{KeyID: "k", ProjectID: "proj3", TeamID: "team3", OrgID: "org2"}); got != "" {
t.Errorf("in org2: %q", got)
}
// A collection removed leaves every default that named it.
if err := s.DeleteCollection(ctx, hand.ID); err != nil {
t.Fatal(err)
}
if got := in(Effective{KeyID: "k", ProjectID: "proj1", TeamID: "team1", OrgID: "org1"}); got != team.ID {
t.Errorf("after removing one: %q", got)
}
if got := in(Effective{KeyID: "k", ProjectID: "proj2", TeamID: "team2", OrgID: "org1"}); got != "" {
t.Errorf("a default left empty stayed: %q", got)
}
}
@@ -0,0 +1,12 @@
-- The collections a part of the tree answers from when a request names
-- none (BACKLOG F3): on an organization, team, project or key, the narrowest
-- one that has any winning. PostgreSQL only, as the collections are.
CREATE TABLE kb_defaults (
scope TEXT NOT NULL,
scope_id TEXT NOT NULL,
-- Collection IDs, comma separated, in the order given.
collections TEXT NOT NULL,
added_at BIGINT NOT NULL,
added_by TEXT NOT NULL DEFAULT '',
PRIMARY KEY (scope, scope_id)
);