Abstract This paper investigates the role of saliency to improve the classification accuracy of a Convolutional Neural Network (CNN) for the case when scarce training data is available. Our approach consists in adding a saliency branch to an existing CNN architecture which is used to modulate the standard bottom-up visual features from the original image input, acting as an attentional mechanism that guides the feature extraction process. The main aim of the proposed approach is to enable the effective training of a fine-grained recognition model with limited training samples and to improve the performance on the task, thereby alleviating the need to annotate a large dataset. The vast majority of saliency methods are evaluated on their ability to generate saliency maps, and not on their functionality in a complete vision pipeline. Our proposed pipeline allows to evaluate saliency methods for the high-level task of object recognition. We perform extensive experiments on various fine-grained datasets (Flowers, Birds, Cars, and Dogs) under different conditions and show that saliency can considerably improve the network’s performance, especially for the case of scarce training data. Furthermore, our experiments show that saliency methods that obtain improved saliency maps (as measured by traditional saliency benchmarks) also translate to saliency methods that yield improved performance gains when applied in an object recognition pipeline.
Zagoruyko and Komodakis @cite_37 propose a method to train a student network from a teacher network. Their novelty with respect to earlier work is the usage of attention to guide the teaching of the student network. They construct a spatial attention map by considering the activations of an image in a teacher network, and mapping these activations to a single spatial attention map which reflects on what locations the hidden neuron activations were most prominent. They consider that this information is important and can help guide the training process once it is also transfered to the student network. They show that their approach significantly improves the learning of the student network. Concretely, they show that guiding the backpropagation of gradients by telling to what spatial coordinates to attend' can assist in the training process. Our paper supports this claim by showing that spatial guidance can help training, although within a different context as in our case we use saliency as attention map and train the network for a new task.
@cite_37
2561238782
Attention plays a critical role in human visual experience. Furthermore, it has recently been demonstrated that attention can also play an important role in the context of applying artificial neural networks to a variety of tasks from fields such as computer vision and NLP. In this work we show that, by properly defining attention for convolutional neural networks, we can actually use this type of information in order to significantly improve the performance of a student CNN network by forcing it to mimic the attention maps of a powerful teacher network. To that end, we propose several novel methods of transferring attention, showing consistent improvement across a variety of datasets and convolutional neural network architectures.