FP8
8-bit floating-point. Halves memory versus 16-bit while keeping a floating-point range, so quality loss is usually smaller than integer INT8. Needs hardware support (Hopper, Blackwell, MI300).
What it is
FP8 is increasingly used to serve frontier models: weights and sometimes activations at 8 bits, with per-tensor scaling to preserve accuracy. Recent GPUs have native FP8 tensor cores, so it also speeds up prefill.
Why it matters
FP8 lets a model fit a smaller vram tier with less quality risk than int4, but only pays off on GPUs that support it - a factor when choosing hardware.