← BREAF Academy

Beginner track · 24 lessons

Build an LLM From Scratch

Start at y = mx + c. Finish with your own model running an agent.

Twenty-four lessons, zero to hero: start at a single line of best fit and end with a small language model you trained yourself, wired into a working AI agent. Every idea is built by hand before it's named — no assumed math, no black boxes.

What you'll walk away with

  • ·Train a tiny language model on real text, from raw gradients to a working GPT
  • ·Understand attention, transformers, and fine-tuning from first principles — no black boxes
  • ·Fine-tune, align (DPO), and run inference on your own model
  • ·Build a working AI agent — tools, memory, and RAG — on top of a model you trained yourself

Founding membership

This track opens with the founding cohort — every lesson, artifact, and practice set. Free-preview lessons are marked below.

Join the founding cohort

Part 1 — The Neuron

  1. A line is a modelComing
  2. Learning is just rolling downhillComing
  3. Why straight lines aren't enoughComing
  4. Stacking neurons into a networkComing
  5. Backpropagation: assigning blameComing
  6. Build a neural net in raw PythonComing

Part 2 — Teaching a Machine Language

  1. Text → numbers: tokenisationComing
  2. Embeddings: meaning as directionComing
  3. Your first language modelComing
  4. The memory problemComing

Part 3 — The Transformer

  1. Attention, from first principlesComing
  2. Queries, keys, values — and maskingComing
  3. Multi-head attention and positionComing
  4. The full transformer blockComing
  5. Build GPT from scratchComing

Part 4 — Training Your Own SLM

  1. Data is the modelComing
  2. The pretraining runComing
  3. Fine-tuning and instruction tuningComing
  4. Alignment: RLHF and DPOComing
  5. Inference: how generation really worksComing

Part 5 — The Agent

  1. What an agent actually isComing
  2. Tools: giving the model handsComing
  3. Memory, context and planningComing
  4. Your agent, on your modelComing