Build an LLM From Scratch

Lesson 51

Knowing it works

Build an LLM From Scratch, 53 lessons

Check what a coding agent actually changed, discover why a tiny demo is weak evidence, and find a newly broken task behind an improved score.

Lesson 50 gave the agent outside checks to help it stay on track. But who checks those checks? A coding agent says it fixed checkout. Would you merge its patch on that evidence?

An evaluation, usually shortened to eval, gives the agent a job and checks the result against a stated requirement. Nothing is trained during this test: the knobs stay as they are. You are testing the whole agent: model, prompt, tools and the loop that uses them.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.