Build an LLM From Scratch

Lesson 48

The context window

Build an LLM From Scratch, 53 lessons

An agent sends its whole story again on every lap, so the text only grows, and a model can read only so much at once. Scrub a twelve-step run past the window, see what dropping, summarising and note-taking each throw away, and find why the cheap-looking fix is the dear one once a cache is on.

The question for this lesson: what happens when the agent’s text no longer fits, and what does it cost?

Lesson 44’s loop keeps a list and hands the model all of it on every lap. Lesson 47 ended on what that means: every lap sends everything again, so what happens as the list grows?

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.