Quantization
Learn what Quantization means and how it is used. Quantization compresses model weights from 16-bit floats down to 8-bit, 4-bit, or lower integer representations, trading a controlled amount of precision for smaller memory footprint and faster inference. It is the standard way to run large models locally and to cut inference cost