Backlog F4: a LoRA trained in the GPU's spare 2 GB on the PoC, and what it learned
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
c52b73feec
commit
0a6a586dfe
1 file changed
+12
+12
@@ -1013,6 +1013,18 @@ a cluster model through `/v1/batches`, reviewed by people. Output to the
|
||||
mirror, into the model's `adapters`, through canary and a quality suite.
|
||||
Runs when the scaling schedule frees a GPU; 8–14B dense bases on one card.
|
||||
Never train on a collection some users of the adapter may not read.
|
||||
Tried 2026-10-08 on the PoC, by hand: Qwen2.5-0.5B-Instruct (fp16) with a
|
||||
LoRA (r=16, every linear layer, 8.8M trainable), trained beside the 35B in
|
||||
cuda01's last 2.3 GB (peak 2158 MiB on the card, 64 s for 138 steps, PEFT in a
|
||||
venv over the vLLM engine's torch). Data: 90 Turkish Q/A pairs a cluster
|
||||
model wrote from the nine Demo/politikalar documents, through the gateway.
|
||||
On nine questions it had not seen, phrased otherwise: base 0/9, 6 epochs
|
||||
4/9, 4 epochs at a lower rate 2/9. Wrong answers mixed facts across
|
||||
documents (the meal card's amount, the office day). The same nine through
|
||||
retrieval: 9/9 with sources. So F4 is for format, vocabulary and tone, and
|
||||
facts come from F1–F3; a bigger base on a free card would do better but not
|
||||
change that. The training itself fits a small slot on a shared card, which
|
||||
the job design can rely on for small bases.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in new issue
Block a user