Build an LLM From Scratch

Lesson 37

A training run

Build an LLM From Scratch, 53 lessons

Check a tiny slice, read an earlier character-level run, and assemble the unrun TinyStories training loop.

The question for this lesson: what really happens when it learns, and how do you tell whether a run is healthy?

Last lesson made the model fit in memory. That gets the program running; it does not tell us whether the program is learning the intended job. Before spending a long run on it, give it a tiny piece of text it should be able to memorize.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.