We consider the binary classification problem on "2 moons dataset" presented in [1]}. The training dataset consists of 2000 samples (split into 2 moon-like clusters in 2 dimensions) and 200 test samples; see sec:app:numerical-results. We train a BNN with 9 neurons. In this low-dimensional environment, we can enumerate all \(2^9=512\)  possible binary configurations and select the best one(s).
Our method is compared with 4 different approaches: a full precision NN, BinaryConnect [2]}, AdaSTE [3]}, and exhaustive search. All methods are trained for 50 epochs with logistic loss. The full precision NN is trained using the Adam optimizer [4]} with default hyperparameters, a learning rate of \(0.1\) , and a batch of size 100. The BinaryConnect approach is trained using the Adam optimizer with default hyperparameters, a learning rate of 1, and a batch of size 100. The AdaSTE method is implemented using a learning rate of 1. Our method uses the same parameters as the STE method, and we set \(\alpha \)  to 4. For a single run, we plotted the training loss in fig:loss2moons.
<FIGURE>