BNN with binary weights and binary activations offer significant time savings in inference. We applied our training procedure ASkewSGD [W1/A1] to a VGG-small with \(\operatorname{sign(\cdot )}\)  activations instead of ReLu activations to enable inference with only XNOR and bit-counting operations. The quantization of activations is here too extreme to apply the same procedure as in sec:experiments. The biased quantizer SAWB from [1]} does not work anymore (empirically). We assume the loss of neural gradient information is too important when activations are quantized on 2 levels.
