An embedding model's query_prefix, put before questions and never before documents
Qwen3-Embedding and models like it are trained with an instruction on the query side only. The prefix goes before the question a knowledge-base search embeds, at answer time and in the admin API's search; documents reach /v1/embeddings from llmgw-kb as they are. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
a133270725
commit
6b7750a0dd
8 files changed
+65
-1
No files matched your search
@@ -8710,6 +8710,10 @@
|
||||
"$ref": "#/components/schemas/catalog.Capabilities",
|
||||
"description": "Capabilities is what this model can do beyond plain completion."
|
||||
},
|
||||
"query_prefix": {
|
||||
"type": "string",
|
||||
"description": "QueryPrefix is put before a question an embedding model embeds for a knowledge-base search; documents are embedded without it. Models trained with an instruction on the query side want it, Qwen3-Embedding \"Instruct: Given a web search query, retrieve relevant passages that answer the query\\nQuery: \" for one."
|
||||
},
|
||||
"role": {
|
||||
"type": "string",
|
||||
"description": "Role \"guard\" is a model that judges the gateway's traffic for the guardrails rather than serving clients: hidden from /v1/models and the chat page, refused to any key that names it, critical by default, and always running. Guard says how its answers are read."
|
||||
|
||||
@@ -45,6 +45,15 @@ is available to everyone who may use them.
|
||||
A collection stays with the model it was made with: vectors from two
|
||||
models cannot be compared.
|
||||
|
||||
Some embedding models are trained with an instruction before the
|
||||
question and none before the documents. Give it as the entry's
|
||||
`query_prefix`; llmgw puts it before questions only. For
|
||||
Qwen3-Embedding:
|
||||
|
||||
```yaml
|
||||
query_prefix: "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery: "
|
||||
```
|
||||
|
||||
With pgvector 0.7 or later a collection's index is kept at half
|
||||
precision (`halfvec`), which halves its memory and takes models of up to
|
||||
4000 dimensions; with an older pgvector the limit is 2000. A collection
|
||||
|
||||
@@ -145,6 +145,7 @@ rather than repeated.
|
||||
| `context_length` | `int` | [Model](#model) |
|
||||
| `max_output` | `int` | [Model](#model) |
|
||||
| `capabilities` | `Capabilities` | [Model](#model) |
|
||||
| `query_prefix` | `string` | [Model](#model) |
|
||||
| `role` | `string` | [Model](#model) |
|
||||
| `guard` | `*GuardModel` | [Model](#model) |
|
||||
| `origin` | `*Origin` | [Model](#model) |
|
||||
@@ -1389,6 +1390,16 @@ Capabilities is what this model can do beyond plain completion.
|
||||
|
||||
Takes a [Capabilities](#capabilities) block.
|
||||
|
||||
### `query_prefix`
|
||||
|
||||
`string`
|
||||
|
||||
QueryPrefix is put before a question an embedding model embeds for a
|
||||
knowledge-base search; documents are embedded without it. Models
|
||||
trained with an instruction on the query side want it, Qwen3-Embedding
|
||||
"Instruct: Given a web search query, retrieve relevant passages that
|
||||
answer the query\nQuery: " for one.
|
||||
|
||||
### `role`
|
||||
|
||||
`string`
|
||||
|
||||
Reference in new issue
Block a user