Binary NN: The first attempt to train BNN is BinaryConnect (BC) [1]}, [2]} which is the first algorithm to implement Quantization Aware Training (QAT); see [3]}, [4]}, [5]}, [6]} and the references therein. BC uses full precision latent weights. On the forward path, the latent weights are binarized. On the backward path, classical backpropagation is applied to update the latent weights, using a differentiable proxy of the binarization function in the gradient calculation. The most common implementation uses the identity proxy, resulting in the straight-through estimator (STE).
Although the neural network parameters are highly compressed (and quantization errors can be large), the BC-STE estimator and its numerous recent improvements perform satisfactorily in many benchmarks and have become a de facto standard; see  [7]}, [8]}, [9]}, [10]}.
