We have trained and tested models using (mixed precision) 16-bit floats. What happens if we quantize them to int8/int4 after training? We used the auto_gptq package, which is inspired by the GPTQ paper [10], for quantization.