Lesson 6
What one neuron can’t learn
Build an LLM From Scratch, 53 lessons
Lesson 6: What one neuron can’t learn
Build an LLM From Scratch, Lesson 6 of 53

Fit one neuron to three students and find the one thing it can never do, then ask whether a stack of neurons with no quiet rule can, and meet the same limit in a yes or no answer.
The question for this lesson: what can a single neuron never learn, however long it trains?
Last lesson the network found good knobs by itself, with two hidden neurons doing the work. After 3000 steps its loss was 1.72. We never asked whether it needs them.
Lesson 2 left a question. Does the network need the quiet rule to say “some sleep is best”? Students 1, 2 and 3 let us test it. They have the same study hours and phone time. They sleep 4, 7 and 10 hours, and they score 46, 58 and 46. Their marks rise, then fall.
The rest of this lesson is in Premium
Premium includes every lesson, with its notes, practice and progress.