What a model learns about correcting itself from its training records
Twelve experiments, each with its predictions committed before it ran, on what small language models learn from records of work being checked and corrected. A model trained on records that always start wrong learned to change right answers; a check written into a reply decided whether the model corrected, even when the check was false; and the largest single improvement came from writing each instruction into the step that carries it out.
Methods
Base models Qwen2.5-1.5B and, as a second family, SmolLM-1.7B, fine-tuned with LoRA (rank 16) on a fixed budget of 400,000 training tokens per run, three seeds per condition, greedy decoding. Tasks are arithmetic chains of 3 to 5 steps (“Start with 7. Multiply by 3. Subtract 4.”) with exact answers. The unfamiliar family adds two operations never seen in training, “Double it” and “Square it”. Each experiment was registered in a dated file before its first run; a result counts a prediction as held only on the registered measure.
Results
D
Every training record starts with a wrong attempt, then a fix
The model “corrects” correct work too: false alarms on correct work 0.98 to 1.00.
D2
Half the records start with a correct attempt that the record confirms
False alarms fall to 0.01 to 0.03. A check of one step passes most errors: it confirms a step that is right and misses the one that is wrong.
D3
The check covers every step in order
Recovers 0.81 on unfamiliar problems, names the wrong step 1.00, false alarms 0.08 (4 of 4 predictions held).
D5
Errors in reading the problem (a step dropped or misread), and a check that compares each step with the problem
Recovers 0.98 from such errors and names the step 0.90, against 0.36 for the arithmetic check. It flags 0.72 of its own reading errors and fixes 0.20. Training on the errors teaches them: first attempts on unfamiliar problems fall from 0.64 to 0.39 correct.
D6, D8, D10
A check written into the reply by the experimenter, true or false
A false all-clear lets the error through 100 of 100 times. A false accusation starts a correction at the named step every time, but the model recomputes the value: it copied the planted number 0 of 400 times, and 0 of 100 when the true value was nowhere in view.
D7, D9
The wrong attempt kept in the record but excluded from the training loss
First attempts on unfamiliar problems return to 0.73 correct. A bare right/wrong verdict trained this way calls every given solution wrong (false alarms 1.00), because in its training only wrong attempts were given; showing correct attempts the same way brings this to 0.01.
D11, D12
Each step names the instruction it carries out, as in “Step 2 (‘Square it’): 23 x 23 = 529.”
Unfamiliar problems: 0.63 to 0.91 correct for Qwen2.5-1.5B, 0.07 to 0.48 for SmolLM-1.7B; steps dropped almost never (all present 0.97 to 1.00); familiar problems unchanged.
Registered predictions held: D2 3 of 5, D3 4 of 4, D4 (the second family) 1 of 4, D5 4 of 8, D6 2 of 5, D7 5 of 5, D8 1 of 3, D9 4 of 6, D10 1 of 3, D11 4 of 4, D12 4 of 4.
Corrections
Two explanations written during the series were wrong and are corrected in the records. The failures on unfamiliar problems were first put down to their length (6 steps against 3 to 5 in training); a later experiment showed 6-step problems with familiar operations solved 0.99 of the time. They were then put down to the models being unable to square a number; with the true value hidden, Qwen squares correctly 0.90 of the time. The failure was in reading “Square it” into a step, which is what the quoting change fixed.
Limitations
Two base models of about 1.5 billion parameters, synthetic English arithmetic, three seeds per condition and one family of unfamiliar operations. The checks are written by the same model that solves, and after the quoting change most remaining errors are arithmetic slips in large products, which that check passed 24 of 25 times for Qwen. Nothing here is tested on open-ended tasks or larger models.
Related work on this site: the Blackhole lab’s audit of its own record found that what limits a reader is what was written down (Real systems). The registration files, results and raw replies are available on request at futurisminstitute@gmail.com.