ProxQuant (PQ) [1]}, Proximal Mean-Field (PMF) [2]}, Mirror Descent
(MD) [3]}, and Rotated Binary Neural Networks (RBNN) [4]} formulate the task of training BNNs as a constrained optimization problem and discuss different methods to generate binary weights from real-valued latent weights. All of these methods have in common that they use gradual annealing of the conversion mapping, in the sense that, unlike BC and its variants, the latent weights are not projected onto a finite set of quantization values in the forward path. Instead, a force is applied to gradually push the latent weights to the quantization constraints, in a manner reminiscent of homotopy methods for solving nonlinear systems or penalty barrier in nonlinear optimization.
