Backlog F4: a LoRA trained in the GPU's spare 2 GB on the PoC, and what it learned

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Ilker ManapandClaude Opus 5.5 committed 2026-10-08 19:31:53 +02:00
1 parent c52b73feec
commit 0a6a586dfe
1 file changed
+12
+12
View File
@@ -1013,6 +1013,18 @@ a cluster model through `/v1/batches`, reviewed by people. Output to the
mirror, into the model's `adapters`, through canary and a quality suite.
Runs when the scaling schedule frees a GPU; 8–14B dense bases on one card.
Never train on a collection some users of the adapter may not read.
Tried 2026-10-08 on the PoC, by hand: Qwen2.5-0.5B-Instruct (fp16) with a
LoRA (r=16, every linear layer, 8.8M trainable), trained beside the 35B in
cuda01's last 2.3 GB (peak 2158 MiB on the card, 64 s for 138 steps, PEFT in a
venv over the vLLM engine's torch). Data: 90 Turkish Q/A pairs a cluster
model wrote from the nine Demo/politikalar documents, through the gateway.
On nine questions it had not seen, phrased otherwise: base 0/9, 6 epochs
4/9, 4 epochs at a lower rate 2/9. Wrong answers mixed facts across
documents (the meal card's amount, the office day). The same nine through
retrieval: 9/9 with sources. So F4 is for format, vocabulary and tone, and
facts come from F1–F3; a bigger base on a free card would do better but not
change that. The training itself fits a small slot on a shared card, which
the job design can rely on for small bases.
---