An embedding model's query_prefix, put before questions and never before documents

Qwen3-Embedding and models like it are trained with an instruction on the
query side only. The prefix goes before the question a knowledge-base
search embeds, at answer time and in the admin API's search; documents
reach /v1/embeddings from llmgw-kb as they are.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-09 18:03:00 +02:00
1 parent a133270725
commit 6b7750a0dd
8 files changed
+65 -1

No files matched your search

+4
View File
@@ -8710,6 +8710,10 @@
"$ref": "#/components/schemas/catalog.Capabilities",
"description": "Capabilities is what this model can do beyond plain completion."
},
"query_prefix": {
"type": "string",
"description": "QueryPrefix is put before a question an embedding model embeds for a knowledge-base search; documents are embedded without it. Models trained with an instruction on the query side want it, Qwen3-Embedding \"Instruct: Given a web search query, retrieve relevant passages that answer the query\\nQuery: \" for one."
},
"role": {
"type": "string",
"description": "Role \"guard\" is a model that judges the gateway's traffic for the guardrails rather than serving clients: hidden from /v1/models and the chat page, refused to any key that names it, critical by default, and always running. Guard says how its answers are read."
+9
View File
@@ -45,6 +45,15 @@ is available to everyone who may use them.
A collection stays with the model it was made with: vectors from two
models cannot be compared.
Some embedding models are trained with an instruction before the
question and none before the documents. Give it as the entry's
`query_prefix`; llmgw puts it before questions only. For
Qwen3-Embedding:
```yaml
query_prefix: "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: "
```
With pgvector 0.7 or later a collection's index is kept at half
precision (`halfvec`), which halves its memory and takes models of up to
4000 dimensions; with an older pgvector the limit is 2000. A collection
+11
View File
@@ -145,6 +145,7 @@ rather than repeated.
| `context_length` | `int` | [Model](#model) |
| `max_output` | `int` | [Model](#model) |
| `capabilities` | `Capabilities` | [Model](#model) |
| `query_prefix` | `string` | [Model](#model) |
| `role` | `string` | [Model](#model) |
| `guard` | `*GuardModel` | [Model](#model) |
| `origin` | `*Origin` | [Model](#model) |
@@ -1389,6 +1390,16 @@ Capabilities is what this model can do beyond plain completion.
Takes a [Capabilities](#capabilities) block.
### `query_prefix`
`string`
QueryPrefix is put before a question an embedding model embeds for a
knowledge-base search; documents are embedded without it. Models
trained with an instruction on the query side want it, Qwen3-Embedding
"Instruct: Given a web search query, retrieve relevant passages that
answer the query\nQuery: " for one.
### `role`
`string`