Effect of Batch Normalization on Noise Resistant Property of Deep Learning Models
The fast execution speed and energy efficiency of analog hardware has made
them a strong contender for deployment of deep learning model at the edge.
However, there are concerns about the presence of analog noise which causes
changes to the weight of the models, leading to performance degradation of deep
learning model, despite their inherent noise resistant characteristics. The
effect of the popular batch normalization layer on the noise resistant ability
of deep learning model is investigated in this work. This systematic study has
been carried out by first training different models with and without batch
normalization layer on CIFAR10 and CIFAR100 dataset. The weights of the
resulting models are then injected with analog noise and the performance of the
models on the test dataset is obtained and compared. The results show that the
presence of batch normalization layer negatively impacts noise resistant
property of deep learning model and the impact grows with the increase of the
number of batch normalization layers.