Training from scratch in a low-precision format has been proven to work well. However, when fine-tuning a pre-trained model, most of the error values will be close to zero, as shown in Fig. REF , since the model has converged well. These small error values will be quantized to zero after quantization, causing the model not to learn any information from the personal data. Thus, we add a scaling factor before the error quantization. For a desired scale error ScaleError as shown in (REF ), its scaling factor can be derived as (REF ) to make training more general.
This method does not need an extra flag bit, as in [1]} to indicate whether the absolute value is smaller than the scaling factor.
