Lesson 13
Tokens
Build an LLM From Scratch, 53 lessons
Lesson 13: Tokens
Build an LLM From Scratch, Lesson 13 of 53

A network only takes numbers, and words are not numbers. See how a tokeniser learns its pieces by counting, and why a number from a table is a label.
The question for this lesson: how does any piece of text become a list of whole numbers, with nothing lost?
Part 1 ended on a network that learns from numbers, and a test on new students to see whether it really learned. Everything it ever took in was a number: hours of study, hours of sleep, hours on the phone.
Words are not numbers. From here on we follow one sentence:
the cat sat on the mat because it was tired
BREAF Premium
The rest of this lesson is in Premium
Premium includes every lesson, with its notes, practice and progress.