LoRA
Low-Rank Adaptation: fine-tuning that freezes the base model and trains a small pair of matrices per layer instead. ~0.1-1% as many trainable parameters, so it fits on one GPU and runs in minutes to hours.
What it is
Instead of updating a weight matrix W directly, LoRA learns two small matrices A and B whose product is added to W at inference: W + BA. A and B have a low "rank" (often 8-64), so together they hold a tiny fraction of the parameters. The base model stays frozen.
The output is a small "adapter" file (megabytes, not gigabytes). You can keep several adapters for one base model and swap them per request.
Why it matters
LoRA is why fine-tuning went from a research-lab budget to a hobbyist one. A task-tuned 8B model on a rented GPU for a few dollars can beat a frontier API on that one task - and then you serve it yourself at gpu hour rates instead of per-token rates.
Cost & infrastructure impact
VRAM needed roughly = model weights (in your chosen precision) + a small overhead for the adapter and optimiser states - far less than full fine tuning. An 8B model in fp16 LoRA-tunes on a single 24 GB card; qlora drops that further.