We introduce an Expanded Parts Model (EPM) for recognizing human attributes (e.g., young, short hair, wearing suits) and actions (e.g., running, jumping) in still images. An EPM is a collection of part templates which are learnt discriminatively to explain specific scale-space regions in the images (in human centric coordinates). This is in contrast to current models which consist of a relatively few (i.e., a mixture of) ‘average’ templates. EPM uses only a subset of the parts to score an image and scores the image sparsely in space, i.e., it ignores redundant and random background in an image. To learn our model, we propose an algorithm which automatically mines parts and learns corresponding discriminative templates together with their respective locations from a large number of candidate parts. We validate our method on three recent challenging datasets of human attributes and actions. We obtain convincing qualitative and state-of-the-art quantitative results on the three datasets.
Many methods have also been proposed to reconstruct images using patches, Similarity by Composition by Boiman and Irani @cite_31 , Implicit Shape Models by Leibe al @cite_10 , Naive Bayes Nearest Neighbors (NBNN) by Boiman al @cite_94 , and Collaborative Representation by Zhu al @cite_56 . Similarly sparse representation has been also used for action recognition in videos @cite_28 . However, while such approaches are generative and are generally based on minimizing the reconstruction error, EPM aims to mine out good patches and learn corresponding discriminative templates with the direct aim of achieving good classification.
@cite_28 @cite_56 @cite_31 @cite_10 @cite_94
2073212499 2602364336 2240914251 2465503160
In this paper, we present a new approach for human action recognition based on key-pose selection and representation. Poses in video frames are described by the proposed extensive pyramidal features (EPFs), which include the Gabor, Gaussian, and wavelet pyramids. These features are able to encode the orientation, intensity, and contour information and therefore provide an informative representation of human poses. Due to the fact that not all poses in a sequence are discriminative and representative, we further utilize the AdaBoost algorithm to learn a subset of discriminative poses. Given the boosted poses for each video sequence, a new classifier named weighted local naive Bayes nearest neighbor is proposed for the final action classification, which is demonstrated to be more accurate and robust than other classifiers, e.g., support vector machine (SVM) and naive Bayes nearest neighbor. The proposed method is systematically evaluated on the KTH data set, the Weizmann data set, the multiview IXMAS data set, and the challenging HMDB51 data set. Experimental results manifest that our method outperforms the state-of-the-art techniques in terms of recognition rate. Before the big data era, scene recognition was often approached with two-step inference using localized intermediate representations (objects, topics, and so on). One of such approaches is the semantic manifold (SM), in which patches and images are modeled as points in a semantic probability simplex. Patch models are learned resorting to weak supervision via image labels, which leads to the problem of scene categories co-occurring in this semantic space. Fortunately, each category has its own co-occurrence patterns that are consistent across the images in that category. Thus, discovering and modeling these patterns are critical to improve the recognition performance in this representation. Since the emergence of large data sets, such as ImageNet and Places, these approaches have been relegated in favor of the much more powerful convolutional neural networks (CNNs), which can automatically learn multi-layered representations from the data. In this paper, we address many limitations of the original SM approach and related works. We propose discriminative patch representations using neural networks and further propose a hybrid architecture in which the semantic manifold is built on top of multiscale CNNs. Both representations can be computed significantly faster than the Gaussian mixture models of the original SM. To combine multiple scales, spatial relations, and multiple features, we formulate rich context models using Markov random fields. To solve the optimization problem, we analyze global and local approaches, where a top–down hierarchical algorithm has the best performance. Experimental results show that exploiting different types of contextual relations jointly consistently improves the recognition accuracy. The restoration of clean remote sensing images from the degraded images is usually an ill-posed problem. A feasible solution for the problem is incorporating various priors into restoration procedure as constrained conditions. Recently, the patch-based Gaussian mixture model (GMM) priors became popular. However, the learning of patch-based GMM priors usually assumes the components of the GMM are independent and identically distributed (i.i.d.). The assumption means that two components could model the same training samples, and thus a possible decrease in model’s description ability. This work presents a new learning method to diversify the patch-based GMM priors through modeling the GMM’s parameters as a determinantal point process (DPP). The introduction of DPP allows us to specify a preference for diversity in patch-based GMM using a positive definite kernel function and thus to increase the model’s description ability for a given model complexity. Furthermore, under the expected patch log-likelihood (EPLL) restoration framework, we further introduce the diversified patch-based GMM priors into remote sensing literature and develop new remote sensing image restoration algorithms. The extensive experiments for denoising and deblurring of remote sensing images demonstrated that the proposed algorithms produced better restoration results than other methods that already proven to perform well in the literature. The recognition of human actions and the determination of human attributes are two tasks that call for fine-grained classification. Indeed, often rather small and inconspicuous objects and features have to be detected to tell their classes apart. In order to deal with this challenge, we propose a novel convolutional neural network that mines mid-level image patches that are sufficiently dedicated to resolve the corresponding subtleties. In particular, we train a newly designed CNN (DeepPattern) that learns discriminative patch groups. There are two innovative aspects to this. On the one hand we pay attention to contextual information in an original fashion. On the other hand, we let an iteration of feature learning and patch clustering purify the set of dedicated patches that we use. We validate our method for action classification on two challenging datasets: PASCAL VOC 2012 Action and Stanford 40 Actions, and for attribute recognition we use the Berkeley Attributes of People dataset. Our discriminative mid-level mining CNN obtains state-of-theart results on these datasets, without a need for annotations about parts and poses.