Build an LLM From Scratch

Lesson 41

Fine-tuning almost nothing

Build an LLM From Scratch, 53 lessons

LoRA freezes a layer and trains a narrow side road beside it, two thin matrices with r neurons between them. That trains r times (inputs plus outputs) numbers instead of inputs times outputs, the toy learns to answer through it, and switching the road off gives the original model back exactly.

The question for this lesson: how can you fine-tune while training almost none of the weights?

Last lesson taught the toy to answer. It trained every weight, then froze the word table, which let the old sentences survive. That left a bill. Every weight that is trained needs its slope worked out and kept, and a real model has billions of weights. It ended by asking: must we retrain every weight?

Freezing the whole model is easy. But if nothing is trained or added, it keeps behaving as before. What we want is to keep its layers and still change what comes out of them.

BREAF Premium

The rest of this lesson is in Premium

Premium includes every lesson, with its notes, practice and progress.