Quantized NN: The STE estimator is easily adapted to QNN by adding a projection step onto the set of quantization levels in the forward pass [1]}; see [2]}, [3]}, [4]} and the references therein.
To mitigate performance loss reported in early work from [1]}, a number of attempts has been proposed. One possible way is to increase the NN size [6]}, or the number of channel for convolution layers [7]}, [8]}. Knowledge distillation has also been considered with some success [9]}. A teacher network
(typically very large [10]} and trained in full-precision) is employed to help the QNN training (the student network).
