Build an LLM From Scratch

From a first neuron to an agent, in six parts.

53 lessons, Premium, first 2 free

Included in Premium

₹597/month

Every lesson and more.

Already a member? Sign in

Course syllabus

53 lessons

01The network

  1. 01What a neural network is14 minFree
  2. 02Inside one neuron15 minFree
  3. 03Keeping score14 min
  4. 04Which way to turn a knob16 min
  5. 05Walking downhill22 min
  6. 06What one neuron can’t learn18 min
  7. 07Hidden neurons and the bend20 min
  8. 08More neurons, more layers22 min
  9. 09Writing a whole layer22 min
  10. 10Blame24 min
  11. 11The training run20 min
  12. 12Did it really learn?16 min

02From numbers to words

  1. 13Tokens25 min
  2. 14A language model by counting25 min
  3. 15The same model as a network18 min
  4. 16Embeddings18 min
  5. 17Meaning as position24 min
  6. 18Memory and forgetting24 min

03The transformer

  1. 19Attention22 min
  2. 20Queries, keys and values24 min
  3. 21Keeping scores calm14 min
  4. 22The mask22 min
  5. 23Several heads24 min
  6. 24Word order22 min
  7. 25The feed-forward network22 min
  8. 26Adding the input back22 min
  9. 27Keeping numbers steady20 min
  10. 28The block22 min
  11. 29The whole model24 min
  12. 30From diagram to code24 min

04Training it for real

  1. 31Where the text comes from22 min
  2. 32How big a model20 min
  3. 33What one step reads20 min
  4. 34Step size and clipping20 min
  5. 35A smarter walker22 min
  6. 36What fills memory22 min
  7. 37A training run18 min

05From predictor to assistant

  1. 38Choosing the next token15 min
  2. 39Why generation stays fast20 min
  3. 40Teaching it to answer25 min
  4. 41Fine-tuning almost nothing20 min
  5. 42Learning from preference22 min
  6. 43Running it cheaply18 min

06The agent

  1. 44What an agent is14 min
  2. 45How tools work15 min
  3. 46Designing tools14 min
  4. 47Forcing valid output12 min
  5. 48The context window14 min
  6. 49Retrieval14 min
  7. 50Staying on track14 min
  8. 51Knowing it works12 min
  9. 52Keeping it safe16 min
  10. 53The whole system12 min