LoRAfine-tuning at home

0:03Full fine-tuning's memory bill

The obvious way to teach a model something new is full fine-tuning: nudge every weight, all seventy billion of them. Every weight needs its own gradient, and the gradients alone demand more memory than the model itself. That's datacenter territory.

0:18A full copy per skill

There's a second bill. Every skill trained this way produces a complete copy of the model — seventy billion numbers for medicine, seventy billion more for law. Three skills, three full models.

0:32Low-rank update ΔW = A × B

LoRA — short for low-rank adaptation — starts from an observation: the change a fine-tune needs is usually low-rank. So leave W frozen and learn the change instead, as delta W equals A times B: a tall matrix times a flat one. With r equal to eight, the pair is a sliver of the original.

0:53Only A and B train

Training now touches only A and B — about a tenth of a percent of the weights. The other ninety-nine point nine percent never moves. Millions of trainable numbers instead of billions, and a gaming laptop is enough.

1:07Swappable adapter cartridges

Because the base never changed, adapters snap in and out like cartridges. Train one for medicine, one for law, one for poetry, one on your own codebase. Load the medical adapter and the same frozen base becomes a medical tutor.

1:23One base, a folder of slivers

Swap the adapter, swap the specialty — each a few megabytes, not a few hundred gigabytes. The datacenter and the full copy per skill are gone. One frozen base, a folder of slivers, because the change a fine-tune needs was low-rank all along. That's how Llama gets personalized over a weekend.