For large training data sets and number of variables \(d\) , stochastic gradient-based (first-order) methods for finding minimizers of (REF ) are often the only manageable option. Several methods have been proposed which transform the loss function (REF ) into a differentiable surrogate (with possibly an additional penalty term) to "favor" quantized solutions.
The general approach is to introduce real-valued "latent" weights \(w\in \mathbb {R}^d\)  from which the quantized weights are generated; in the binary case, it is classical to use the \(\operatorname{\operatorname{sign}(\cdot )}\)  function or a differentiable surrogate thereof.
The simplest method, called BinaryConnect (BC) [1]}, is based on straight-through estimators (STE) that ignore the sign conversion in computing the gradient with respect to the latent weights \(w\) .
BC reaches state-of the art performance on elementary classification tasks and is still a competitive baseline method for more sophisticated problems. Extensions of STE has also been used for more general QNN by [2]}, [3]}, [4]}, [5]}.
