Build an LLM From Scratch

Lesson 10

Blame

Build an LLM From Scratch, 53 lessons

Find every knob's share of the blame in one sweep backwards from the loss, instead of nudging each knob and running the network again. Rates multiply along a path, shared stretches are worked out once, and a shut quiet rule cuts the path.

The question for this lesson: how does every knob get its share of the blame without running the network again for each one?

Lesson 4 gave every knob a blame: the slope of the loss, found by nudging that knob a tiny bit and running the network again. Lesson 5 turned each knob against its blame, over and over, and the loss fell. Lesson 9 then wrote the network as a chain of grids and counted the 91,401 knobs of lesson 8’s big network.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.