QLoRA
LoRA on top of a base model that has been quantised to 4-bit. Cuts the VRAM needed to fine-tune by ~3-4x, so a 70B model tunes on a single 48 GB GPU instead of a multi-GPU node.
What it is
The frozen base weights are stored in 4-bit (int4-style, usually NF4). They are de-quantised to 16-bit on the fly, layer by layer, only while computing - so the full model never sits in memory at full precision. The lora adapter itself trains in 16-bit. Quality loss versus plain LoRA is small for most tasks.
Why it matters
QLoRA collapses the hardware bar for fine-tuning a large model. That directly changes which GPU you rent (one card, not a node) and therefore the cost of a run - often by more than half.