 We design simple and quantifiable testing of global translation-invariance in deep learning models trained on the MNIST dataset. Experiments on convolutional and capsules neural networks show that both models have poor performance in dealing with global translation-invariance; however, the performance improved by using data augmentation. Although the capsule network is better on the MNIST testing dataset, the convolutional neural network generally has better performance on the translation-invariance. Convolutional neural networks (CNN) have achieved state-of-the-art performance than the human being on many computer vision tasks BID6 ; BID2 . The deep learning community trend to believe that the success of CNN mainly due to two key features in CNN, reduced computation cost with weight sharing in convolutional layers and generalization with local invariance in subsampling layers BID7 ; BID8 . Due to convolutional layers are 'place-coded' equivariant and max-pooling layers are local invariant BID1 , CNN has to learn different models for different viewpoints which need big data and expensive cost.More Generalization model should be able to train on a limited range of viewpoints and getting good performance on a much more wider range. Capsule network is robust in dealing with different viewpoints BID3 BID9 ; BID4 . Capsules are a group of neurons which includes the pose, colour, lighting and deformation of the visual entity. Capsule network aims for 'rate-coded' equivariance because it's the weights that code viewpoint-invariant knowledge, not the neural activities. Viewpoint changes in capsule network are linear effects on the pose matrices of the parts and the whole between different capsules layers. However, it still unclear whether capsule networks be able to generalize for global translation invariance.Visualize and Quantify the translation-invariance in deep learning model are essential for understanding the architectural choices and helpful for developing Generalization model that is invariant to viewpoint changes. An analysis using translation-sensitivity map for MNIST digit dataset has been used to investigate translation invariance in CNN BID5 . In this paper, we introduce a simple method to test the performance of global translation-invariance in convolutional and capsule neural network models trained on the MNIST dataset. We introduce a simple GTI testing dataset for deep learning models trained on MNIST dataset. The goal is to get a better understanding of the ability of CNN and CapsNet to dealing with global translational invariance. Although the current version of CapsNet could not handle global translational invariance without data augmentation, we still believe CapsNet architecture potentially better than CNN on dealing with global translational invariance because capsules could train to learn all viewpoint no matter it receives the information for the centre or the edge. Our testing method is sample Figure 5: GTI dataset accuracy of models trained on CNN and CapsNet with different amount of random shifting in MNIST training dataset.and quantifiable, and it easy to implement for other datasets of computer vision tasks by taking a clear and correct labelled image from each class and apply the translational shifting to cover all possible cases.
Testing of global translational invariance in Convolutional and Capsule Networks
{"src": "scitldr-AIC_train", "paper_id": "SJlgOjAqYQ"}