A knowledge-base import sends far more than the cluster can take at once,
and every band waited the same 30 s. A band now has its own queue_ttl, so
one for bulk work waits behind people instead of being turned away.
And a request turned away because the cluster stayed busy — the queue
full (503) or waited as long as its band may (504) — now carries
Retry-After: 10. Without it llmgw-kb, which retries only what says when
to, counted each such passage as failed; that is what turned parts of the
metallurgy import away.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Qwen3-Embedding and models like it are trained with an instruction on the
query side only. The prefix goes before the question a knowledge-base
search embeds, at answer time and in the admin API's search; documents
reach /v1/embeddings from llmgw-kb as they are.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A search asking for more than 10 passages got at most 40 candidates from
HNSW, pgvector's default ef_search. The full-text arm scored every chunk
holding a word of the question before limiting, over the whole
organization's GIN index; it now takes candidates by ts_rank from the
collection's own partial index and scores those. New collections index
halfvec on pgvector 0.7 and later (up to 4000 dimensions).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An adapter now has an owner (org, team, project, user or key): only callers
there see it, and to anyone else it is a model that does not exist, absent
from /v1/models too. A training job's owner defaults to the one collection
its pairs come from, must be able to read every collection they come from,
and pairs from a knowledge base are never made public. ctl train publish
keeps the GGUF or PEFT files on the mirror, which accepts uploads now, and
sets the owned adapter after the usual architecture check.
The knowledge bases' cache_salt follows the collections searched rather
than the organization, so two teams of one organization no longer share
cached passages.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A request given passages carries a cache_salt derived from its organization
when the replica chosen is vLLM, so how fast an answer starts cannot say what
another tenant's documents held. Other engines are not sent the field.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A Documents picker on /chat offers the collections the person may read in
the chosen project, the default when there is one, or none; the sources an
answer was given are listed under it. The page's choice becomes the
X-Llmgw-Knowledge header; one the page sent itself is dropped. In English
and Turkish.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On an organization, team, project or key, the narrowest winning: llmgw ctl
kb default, POST /api/v1/kb/defaults, and the console's Knowledge screen. A
request's own X-Llmgw-Knowledge overrides it and none turns it off. A default
never refuses a request: a collection the caller cannot read is left out, and
a failed search answers without passages. Removing a collection takes it out
of every default.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A request naming collections in X-Llmgw-Knowledge has its last question
embedded on the caller's account, the best passages found and added to the
system message, numbered, with an instruction to cite them and to say when
they do not hold the answer; x-llmgw-sources says which were given. Done
before the guardrails judge the request, never served from the answer cache,
and a collection the key may not read is answered as one that does not
exist. Metrics llmgw_kb_retrievals_total and llmgw_kb_retrieval_seconds.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Collections listed with owner, model, documents and grants; A makes one; on a
row its documents, grant, revoke, a search to try, delete. docs/knowledge.md
says what administrators do and how people use llmgw-kb on Windows, macOS
and Linux: what it reads, what leaves their machine, where its ledger is.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
