Build an LLM From Scratch

Lesson 8

More neurons, more layers

Build an LLM From Scratch, 53 lessons

Cut a map with straight lines, one hidden neuron at a time, and see why each neuron is a question. Then fold a paper strip with stacked layers, and see why pieces multiply with depth while they only add with width.

The question for this lesson: what do more neurons and more layers buy?

Last lesson ended on a promise. Each hidden neuron adds one bend, and a sum of enough bends can follow any curve. So what does more of this buy us?

There are two ways to have more. Put more neurons side by side in one layer, which is called width. Or stack more layers one after another, which is called depth. They buy different things, and that difference is this lesson.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.