We compare ASkewSGD , BinaryConnect [1]} and AdaSTE [2]} in a logistic regression problem. We generate \(n=6000\)  feature vectors \(\lbrace x_k\rbrace _{k=1}^n\)  of dimension \(d=10\) , drawn independently from the uniform distribution in \([-1,1]\) . We randomly choose an optimal vector \(w_*\)  on the vertices of the hypercube and generate the labels as follows: \(y_k \sim \operatorname{Bernoulli}(\lbrace 1+\mathrm {e}^{-x_k^\top w_*}\rbrace ^{-1})\) .
For completeness, we study how a SGD converges with full precision to the optimal point \(w_*\)  of this convex problem. All methods are trained for 25 epochs using the SGD optimizer. The learning rate is set to 1 and the gradients are computed on random batches of 1000 samples. For AdaSTE, we have used the code https://github.com/intellhave/AdaSTE with the hyperparameters specified in the package for annealing.
<FIGURE>