Build an LLM From Scratch

Lesson 40

Teaching it to answer

Build an LLM From Scratch, 53 lessons

A base model can continue a question instead of answering it, even when it knows the answer. Fine-tuning is the same training, started from the pretrained weights, on rows that show a question and its answer in a template with special tokens, scored only on the answer. Tuned hard on a few rows, a model can lose old skills; freezing part of it can limit that.

The question for this lesson: why might a pretrained model not answer, and how do we teach it to?

Last lesson made the loop fast. The model keeps the keys and values of the words so far, so it need not compute them again for each new word. The new query still looks at those saved keys and values. It ended on a puzzle: ask the model a question, and it does not answer. Nothing is broken. To see why, we need to look at what it was trained to do.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.