Build an LLM From Scratch

Lesson 3

Keeping score

Build an LLM From Scratch, 53 lessons

Measure how wrong a network is with one number. Find out why adding errors fails, why we square them, and what the loss tells you.

The question for this lesson: how do we measure how wrong a network is?

Last lesson you ran student 5 through the network by hand. It said 26, and the table says 24. That’s off by 2 for one student.

How wrong is the whole network? We need one number, because later the network has to improve itself by turning its own knobs, and after each turn it has to know whether the turn helped. “It looks better” is not something a computer can see. A number it can compare is. In lesson 1, fixing one student hurt others, so looking at twelve answers never said whether the network got better.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.