Lesson 37
A training run
Build an LLM From Scratch, 53 lessons
Lesson 37: A training run
Build an LLM From Scratch, Lesson 37 of 53

Check a tiny slice, read an earlier character-level run, and assemble the unrun TinyStories training loop.
The question for this lesson: what really happens when it learns, and how do you tell whether a run is healthy?
Last lesson made the model fit in memory. That gets the program running; it does not tell us whether the program is learning the intended job. Before spending a long run on it, give it a tiny piece of text it should be able to memorize.
BREAF Premium
The rest of this lesson is in Premium
Premium includes every lesson, with its notes, practice and progress.