Despite these advantages of BNNs, the enormous accuracy gap exists between BNNs and real-valued CNN since binarized operation limits model capacity and leads to a significant information loss of feature maps. To address this problem, Bi-RealNet [1]} introduced a shortcut operation to increase value range of network, achieving remarkable performance improvement compared with XNOR-Net [2]}. Based on this, Liu et al. [3]} proved that BNNs are sensitive to activation distribution shift. A small distribution value offset close to zero would cause the binarized feature map to have a distinct appearance, leading to significant degradation of predictive accuracy. They proposed a generalization of Sign function (RSign) and PReLU function (RPReLU) to shift the distribution of feature maps. The proposed ReActNet based on MobileNetV1 [4]} achieved the top-1 accuracy closed to real-valued network on ImageNet dataset [5]} with lower computational cost.
<FIGURE>