Build an LLM From Scratch

Lesson 11

The training run

Build an LLM From Scratch, 53 lessons

Assemble predict, measure, blame and nudge into a working training run. Discover whose complaints guide an update, when the knobs change, and why hidden neurons need different starts.

We ended lesson 10 with every piece on the table: a forward pass makes an answer, the loss scores it, backpropagation finds each knob’s blame, and the update rule moves each knob using that blame. How do they become a run?

Start with a problem the update rule cannot settle by itself. The network shares its knobs across the whole class. If you improve its answer for a particular student, what happens to everyone else? In Picture 1, guess first, then move the network along a step computed from student 7’s complaint.

[[breaf-artifact:1]] Do the three tries in Picture 1 before reading on.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.