Trained adapters published, private to their owners like the knowledge bases they come from

An adapter now has an owner (org, team, project, user or key): only callers
there see it, and to anyone else it is a model that does not exist, absent
from /v1/models too. A training job's owner defaults to the one collection
its pairs come from, must be able to read every collection they come from,
and pairs from a knowledge base are never made public. ctl train publish
keeps the GGUF or PEFT files on the mirror, which accepts uploads now, and
sets the owned adapter after the usual architecture check.

The knowledge bases' cache_salt follows the collections searched rather
than the organization, so two teams of one organization no longer share
cached passages.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 21:30:50 +02:00
1 parent 964b35ae65
commit 85025dfb03
29 files changed
+1049 -84

No files matched your search

+48 -3
View File
@@ -317,9 +317,11 @@ Behaviour to know about:
(Codex, Claude Code) work the same way, since they are answered as chat
completions.
- **Separate prefix caches.** On vLLM replicas, a request given passages
carries a `cache_salt` derived from its organization. One organization's
passages then never share cached blocks with another's, so how fast an
answer starts says nothing about what another tenant asked.
carries a `cache_salt` derived from the collections they came from.
Requests that could be given the same passages still share cached blocks,
and no one else's do: not another organization's, nor another team's of
the same one. How fast an answer starts therefore says nothing about what
someone else asked.
- **No answer cache.** Answers from knowledge bases are never served from
the answer cache, because two keys of one project can read different
collections.
@@ -340,6 +342,49 @@ a product code or a name.
Metrics: `llmgw_kb_retrievals_total{result}` (found, nothing, refused) and
`llmgw_kb_retrieval_seconds`.
## Training an adapter on a collection
An adapter (LoRA) changes how a model *writes*: the organisation's words,
format and tone. It does not reliably teach facts. On the PoC, a small
model trained on nine policy documents answered 4 of 9 rephrased questions
right and mixed facts up on the rest, while the same questions through the
knowledge base got 9 of 9. Use both: the knowledge base for what answers
say, an adapter for how they sound.
```sh
# 1. Questions and answers from the collection, written by a model of the
# cluster. Read the file through and correct it.
llmgw ctl kb dataset Acme/policies --model qwen2.5-coder-14b-instruct --out policies.jsonl
# 2. Train on a node. --gpu-memory is what the job needs of a card; it
# runs beside an engine when the engine leaves that much free.
llmgw ctl train start policies-tone --model qwen2.5-0.5b-instruct \
--base-repo Qwen/Qwen2.5-0.5B-Instruct --node cuda01 --gpu-memory 2300 --data policies.jsonl
llmgw ctl train show JOB --follow
# 3. Make it the model's adapter, kept on the mirror.
llmgw ctl train publish JOB --restart
```
The node trains with its installed vLLM's torch and a small overlay (peft
and accelerate), as the engines' user. For a model llama.cpp serves, it also
converts the adapter to GGUF with the installed llama.cpp's own code. What
it makes is kept by the control plane (`llmgw ctl train download`).
An adapter is private like its collection:
- **It has an owner.** An adapter trained on a collection carries some of
what the collection says in its weights, so it belongs to a part of the
tree.
- By default the owner is the collection's owner.
- `--owner` names another, which must be able to read every collection
the pairs came from.
- Only callers in the owner's part of the tree see the adapter. To anyone
else it is a model that does not exist, absent from `/v1/models` too.
- **It is public only by choice.** `--public` makes an adapter every
caller's. It is refused for pairs from a knowledge base, so it is only for
pairs written by hand.
## How it is kept apart
Passages live in PostgreSQL, partitioned by organization, with an index per