Lesson 51
Knowing it works
Build an LLM From Scratch, 53 lessons
Lesson 51: Knowing it works
Build an LLM From Scratch, Lesson 51 of 53

Check what a coding agent actually changed, discover why a tiny demo is weak evidence, and find a newly broken task behind an improved score.
Lesson 50 gave the agent outside checks to help it stay on track. But who checks those checks? A coding agent says it fixed checkout. Would you merge its patch on that evidence?
An evaluation, usually shortened to eval, gives the agent a job and checks the result against a stated requirement. Nothing is trained during this test: the knobs stay as they are. You are testing the whole agent: model, prompt, tools and the loop that uses them.
BREAF Premium
The rest of this lesson is in Premium
Premium includes every lesson, with its notes, practice and progress.