INT8
8-bit integer quantization. Roughly halves model size versus 16-bit with small, often negligible quality loss for chat-style workloads. Widely supported.
What it is
Weights are mapped to 256 integer levels with a scale factor. A 70B model drops from ~140 GB to ~70 GB - the difference between needing two 80 GB GPUs and one.
Why it matters
Often the safest first step in quantization: meaningful savings, low risk. Drop to int4 only if you still need to and have tested quality.