INT4
4-bit integer quantization. Cuts model size to about a quarter of 16-bit. Big cost win, but quality loss is more noticeable and model-dependent - test before relying on it.
What it is
Schemes like AWQ, GPTQ and GGUF Q4 pack weights into 4 bits with group-wise scales. A 70B model becomes ~35 GB - it runs on a single 48 GB GPU, or even a 24 GB consumer card with short context.
Why it matters
INT4 is what makes running big models on cheap hardware possible, and is the basis of QLoRA fine-tuning. The trade-off is real: expect some regression on hard reasoning and exact-format tasks.