Lesson 12
Did it really learn?
Build an LLM From Scratch, 53 lessons
Lesson 12: Did it really learn?
Build an LLM From Scratch, Lesson 12 of 53

Catch a perfect memoriser, inspect a real training run, and choose a saved network without spending your final test.
Picture a student who memorised last year’s exam answers. They get full marks on that paper, then fail a new one. The training run from the last lesson has the same risk. Its training loss fell, but those were the same marks that told the network which way to move. Would the improvement help a student whose mark never entered the updates?
BREAF Premium
The rest of this lesson is in Premium
Premium includes every lesson, with its notes, practice and progress.