Fashion landmark detection is a challenging task even using the current deep learning techniques, due to the large variation and non-rigid deformation of clothes. In order to tackle these problems, we propose Spatial-Aware Non-Local (SANL) block, an attentive module in the deep neural network which can utilize spatial and semantic information while capturing global dependency. The attention maps are generated by Grad-CAM or a human parsing segmentation model and then fed into the SANL blocks via attention mechanism. We then establish our fashion landmark detection framework on feature pyramid network, equipped with four SANL blocks in the backbone. It is demonstrated by the experimental results on two large-scale fashion datasets that our proposed fashion landmark detection approach with the SANL blocks outperforms the current state-of-the-art methods considerably. Some supplementary experiments on fine-grained image classification also show the effectiveness of the proposed SANL block.
Visual attention is rather common in human perception, for example, bright color can easily draw our attention and we can locate a cat just by a glance. Attention mechanism has been introduced to deep neural networks and is widely applied to various computer vision tasks, such as image classification @cite_23 @cite_8 @cite_32 , object detection @cite_12 @cite_15 , image captioning @cite_24 @cite_31 @cite_26 and visual question answering @cite_13 @cite_26 .
@cite_31 @cite_26 @cite_8 @cite_32 @cite_24 @cite_23 @cite_15 @cite_13 @cite_12
2017836770 2162216530 2777320413 2743640935
Visual attention is the cognitive process of directing our gaze on one aspect of the visual field while ignoring others. The mainstream approach to modeling focal visual attention involves identifying saliencies in the image and applying a search process to the salient regions. However, such inference schemes commonly fail to accurately capture perceptual attractors, require massive computational effort and, generally speaking, are not biologically plausible. This paper introduces a novel approach to the problem of visual search by framing it as an adaptive learning process. In particular, we devise an approximate optimal control framework, based on reinforcement learning, for actively searching a visual field. We apply the method to the problem of face detection and demonstrate that the technique is both accurate and scalable. Moreover, the foundations proposed here pave the way for extending the approach to other large-scale visual perception problems. Visual attention is a mechanism which filters out redundant visual information and detects the most relevant parts of our visual field. Automatic determination of the most visually relevant areas would be useful in many applications such as image and video coding, watermarking, video browsing, and quality assessment. Many research groups are currently investigating computational modeling of the visual attention system. The first published computational models have been based on some basic and well-understood human visual system (HVS) properties. These models feature a single perceptual layer that simulates only one aspect of the visual system. More recent models integrate complex features of the HVS and simulate hierarchical perceptual representation of the visual input. The bottom-up mechanism is the most occurring feature found in modern models. This mechanism refers to involuntary attention (i.e., salient spatial visual features that effortlessly or involuntary attract our attention). This paper presents a coherent computational approach to the modeling of the bottom-up visual attention. This model is mainly based on the current understanding of the HVS behavior. Contrast sensitivity functions, perceptual decomposition, visual masking, and center-surround interactions are some of the features implemented in this model. The performances of this algorithm are assessed by using natural images and experimental measurements from an eye-tracking system. Two adequate well-known metrics (correlation coefficient and Kullbacl-Leibler divergence) are used to validate this model. A further metric is also defined. The results from this model are finally compared to those from a reference bottom-up model. Visual attention plays a central role in natural and artificial systems to control perceptual resources. The classic artificial visual attention systems uses salient features of the image obtained from the information given by predefined filters. Recently, deep neural networks have been developed for recognizing thousands of objects and autonomously generate visual characteristics optimized by training with large data sets. Besides being used for object recognition, these features have been very successful in other visual problems such as object segmentation, tracking and recently, visual attention. In this work we propose a biologically inspired object classification and localization framework that combines Deep Convolutional Neural Networks with foveal vision. First, a feed-forward pass is performed to obtain the predicted class labels. Next, we get the object location proposals by applying a segmentation mask on the saliency map calculated through a top-down backward pass. The main contribution of our work lies in the evaluation of the performances obtained with different non-uniform resolutions. We were able to establish a relationship between performance and the different levels of information preserved by each of the sensing configurations. The results demonstrate that we do not need to store and transmit all the information present on high-resolution images since, beyond a certain amount of preserved information, the performance in the classification and localization task saturates. Visual attention, which assigns weights to image regions according to their relevance to a question, is considered as an indispensable part by most Visual Question Answering models. Although the questions may involve complex relations among multiple regions, few attention models can effectively encode such cross-region relations. In this paper, we demonstrate the importance of encoding such relations by showing the limited effective receptive field of ResNet on two datasets, and propose to model the visual attention as a multivariate distribution over a grid-structured Conditional Random Field on image regions. We demonstrate how to convert the iterative inference algorithms, Mean Field and Loopy Belief Propagation, as recurrent layers of an end-to-end neural network. We empirically evaluated our model on 3 datasets, in which it surpasses the best baseline model of the newly released CLEVR dataset by 9.5 , and the best published model on the VQA dataset by 1.25 . Source code is available at https: github.com zhuchen03 vqa-sva.