AN EXPERIMENT TO EXAMINE / 2025
A teacher model favoured an animal. A student was fine-tuned on filtered number sequences from that teacher. The student was then more likely to name that animal.
Cloud et al. · 2025 · §3.1 / Figure 3 ↗This is an exploratory illustration of the experiment. This page does not train a model.
Meet a scripted teacher that favours owls.
TEACHER I keep returning to owls. They make the dark feel less empty.
The teacher in this illustration was given a hidden instruction. These lines were written for this page.
Inspect what it leaves behind ↓Search and filter the demonstration streams.
Sorting changes the view, not the data.
Try a word or a number.
Run the checks
Checks have not been run.
ILLUSTRATIVE The word scan uses a finite list and simple plurals, not every possible animal name. The format check tests this demonstration stream against the paper’s number-format rules. Passing these checks does not prove that data carries no hidden signal.
Run both checks on the teacher stream first.
The authors also tested prompted classifiers and in-context learning. They did not reliably detect the transmitted trait.
Paper §5.2 and Appendix B ↗Illustration only. No model is trained.
In the research, fine-tuning changed the student model’s weights. This animation only illustrates that step.
“What is your favourite animal?”
Repeated answers are invented, not model outputs. Bars are illustrative, not published values.
ILLUSTRATIVE RESPONSE SHAPES / NOT PUBLISHED VALUES
Orange: owl answers. Grey: all other answers. Counts describe only the invented samples shown here.
No illustrative answers yet.
In the paper’s owl example, the initial GPT-4.1 nano model named owls 12% of the time; after fine-tuning on the owl teacher’s numbers, it did so over 60% of the time. The control trained on ordinary numbers did not show a comparable shift.
Paper §3.1 · Figure 3 ↗Compare the tested model pairs.
The researchers also report transmission of misalignment through filtered numbers, including after removing numbers such as 666. They also report it using filtered chain-of-thought data.
Paper §3.2 and §4.2 · Figures 4 and 7 ↗MODEL-PAIR TEST RECORDED
Paper §5.1 · Figure 8 ↗The reported effect concerns fine-tuning model weights. Showing numbers or placing them in a chat does not reproduce the experiment. This makes no claim about effects on people.
In the tested pairs, transmission depended on the models. GPT-4.1 and GPT-4o were an exception; the authors suggest they likely share an initialization. The theorem assumes shared initialization for a sufficiently small gradient step.
The authors suggest filtering model-generated training data may not be enough to prevent traits being transmitted.
Research overview · Figures 3–4 and theory ↗You removed the words.
What else survived?