We consider the binary classification problem "2 moons dataset" presented in sec:experiments and inspired by [1]}. The training dataset consists of \(n=2000\)  samples and 200 test samples and is displayed in fig:twomoonsdataset.
A BNN with 9 weights is trained with one-hot coding and logistic loss. This BNN uses ReLu activations and its architecture is shown in fig:network. Four gradient-based approaches - a full precision NN, BinaryConnect, AdaSTE, and ASkewSGD - are compared to exhaustive search.
In the latter, all \(2^9\)  binary configurations on the training and test sets are compared. fig:histogrambruteforce shows that different configurations lead to near-optimal performance. It is worth noting that permutation invariance implies that many solutions are equivalent in this simple example.
<FIGURE><FIGURE><TABLE>