Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> In this paper we consider a class of human activities?atomic activities?which can be represented as a set of measurements over a finite temporal window (e.g., the motion of human body parts during a walking cycle) and which has a relatively small space of variations in performance. A new approach for modeling and recognition of atomic activities that employs principal component analysis and analytical global transformations is proposed. The modeling of sets of exemplar instances of activities that are similar in duration and involve similar body part motions is achieved by parameterizing their representation using principal component analysis. The recognition of variants of modeled activities is achieved by searching the space of admissible parameterized transformations that these activities can undergo. This formulation iteratively refines the recognition of the class to which the observed activity belongs and the transformation parameters that relate it to the model in its class. We provide several experiments on recognition of articulated and deformable human motions from image motion parameters. <s> BIB001 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> This paper describes a pedestrian detection system that integrates image intensity information with motion information. We use a detection style algorithm that scans a detector over two consecutive frames of a video sequence. The detector is trained (using AdaBoost) to take advantage of both motion and appearance information to detect a walking person. Past approaches have built detectors based on appearance information, but ours is the first to combine both sources of information in a single detector. The implementation described runs at about 4 frames/second, detects pedestrians at very small scales (as small as 20/spl times/15 pixels), and has a very low false positive rate. Our approach builds on the detection work of Viola and Jones. Novel contributions of this paper include: i) development of a representation of image motion which is extremely efficient, and ii) implementation of a state of the art pedestrian detection system which operates on low resolution images under difficult conditions (such as rain and snow). <s> BIB002 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> Example-based methods are effective for parameter estimation problems when the underlying system is simple or the dimensionality of the input is low. For complex and high-dimensional problems such as pose estimation, the number of required examples and the computational complexity rapidly become prohibitively high. We introduce a new algorithm that learns a set of hashing functions that efficiently index examples relevant to a particular estimation task. Our algorithm extends locality-sensitive hashing, a recently developed method to find approximate neighbors in time sublinear in the number of examples. This method depends critically on the choice of hash functions that are optimally relevant to a particular estimation problem. Experiments demonstrate that the resulting algorithm, which we call parameter-sensitive hashing, can rapidly and accurately estimate the articulated pose of human figures from a large database of example images. <s> BIB003 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> In the last few years, several new algorithms based on graph cuts have been developed to solve energy minimization problems in computer vision. Each of these techniques constructs a graph such that the minimum cut on the graph also minimizes the energy. Yet, because these graph constructions are complex and highly specific to a particular energy function, graph cuts have seen limited application to date. In this paper, we give a characterization of the energy functions that can be minimized by graph cuts. Our results are restricted to functions of binary variables. However, our work generalizes many previous constructions and is easily applicable to vision problems that involve large numbers of labels, such as stereo, motion, image restoration, and scene reconstruction. We give a precise characterization of what energy functions can be minimized using graph cuts, among the energy functions that can be written as a sum of terms containing three or fewer binary variables. We also provide a general-purpose construction to minimize such an energy function. Finally, we give a necessary condition for any energy function of binary variables to be minimized by graph cuts. Researchers who are considering the use of graph cuts to optimize a particular energy function can use our results to determine if this is possible and then follow our construction to create the appropriate graph. A software implementation is freely available. <s> BIB004 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> Capturing the human hand motion from video involves the estimation of the rigid global hand pose as well as the nonrigid finger articulation. The complexity induced by the high degrees of freedom of the articulated hand challenges many visual tracking techniques. For example, the particle filtering technique is plagued by the demanding requirement of a huge number of particles and the phenomenon of particle degeneracy. This paper presents a novel approach to tracking the articulated hand in video by learning and integrating natural hand motion priors. To cope with the finger articulation, this paper proposes a powerful sequential Monte Carlo tracking algorithm based on importance sampling techniques, where the importance function is based on an initial manifold model of the articulation configuration space learned from motion-captured data. In addition, this paper presents a divide-and-conquer strategy that decouples the hand poses and finger articulations and integrates them in an iterative framework to reduce the complexity of the problem. Our experiments show that this approach is effective and efficient for tracking the articulated hand. This approach can be extended to track other articulated targets. <s> BIB005 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> In this paper, we present a fast new fully dynamic algorithm for the st-mincut/max-flow problem. We show how this algorithm can be used to efficiently compute MAP estimates for dynamically changing MRF models of labeling problems in computer vision, such as image segmentation. Specifically, given the solution of the max-flow problem on a graph, we show how to efficiently compute the maximum flow in a modified version of the graph. Our experiments showed that the time taken by our algorithm is roughly proportional to the number of edges whose weights were different in the two graphs. We test the performance of our algorithm on one particular problem: the object-background segmentation problem for video and compare it with the best known st-mincut algorithm. The results show that the dynamic graph cut algorithm is much faster than its static counterpart and enables real time image segmentation. It should be noted that our method is generic and can be used to yield similar improvements in many other cases that involve dynamic change in the graph <s> BIB006 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> We integrate the cascade-of-rejectors approach with the Histograms of Oriented Gradients (HoG) features to achieve a fast and accurate human detection system. The features used in our system are HoGs of variable-size blocks that capture salient features of humans automatically. Using AdaBoost for feature selection, we identify the appropriate set of blocks, from a large set of possible blocks. In our system, we use the integral image representation and a rejection cascade which significantly speed up the computation. For a 320 × 280 image, the system can process 5 to 30 frames per second depending on the density in which we scan the image, while maintaining an accuracy level similar to existing methods. <s> BIB007 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> We describe a learning-based method for recovering 3D human body pose from single images and monocular image sequences. Our approach requires neither an explicit body model nor prior labeling of body parts in the image. Instead, it recovers pose by direct nonlinear regression against shape descriptor vectors extracted automatically from image silhouettes. For robustness against local silhouette segmentation errors, silhouette shape is encoded by histogram-of-shape-contexts descriptors. We evaluate several different regression methods: ridge regression, relevance vector machine (RVM) regression, and support vector machine (SVM) regression over both linear and kernel bases. The RVMs provide much sparser regressors without compromising performance, and kernel bases give a small but worthwhile improvement in performance. The loss of depth and limb labeling information often makes the recovery of 3D pose from single silhouettes ambiguous. To handle this, the method is embedded in a novel regressive tracking framework, using dynamics from the previous state estimate together with a learned regression value to disambiguate the pose. We show that the resulting system tracks long sequences stably. For realism and good generalization over a wide range of viewpoints, we train the regressors on images resynthesized from real human motion capture data. The method is demonstrated for several representations of full body pose, both quantitatively on independent but similar test data and qualitatively on real image sequences. Mean angular errors of 4-6/spl deg/ are obtained for a variety of walking motions. <s> BIB008 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> The problem we consider in this paper is to take a single two-dimensional image containing a human figure, locate the joint positions, and use these to estimate the body configuration and pose in three-dimensional space. The basic approach is to store a number of exemplar 2D views of the human body in a variety of different configurations and viewpoints with respect to the camera. On each of these stored views, the locations of the body joints (left elbow, right knee, etc.) are manually marked and labeled for future use. The input image is then matched to each stored view, using the technique of shape context matching in conjunction with a kinematic chain-based deformation model. Assuming that there is a stored view sufficiently similar in configuration and pose, the correspondence process would succeed. The locations of the body joints are then transferred from the exemplar view to the test shape. Given the 2D joint locations, the 3D body configuration and pose are then estimated using an existing algorithm. We can apply this technique to video by treating each frame independently - tracking just becomes repeated recognition. We present results on a variety of data sets <s> BIB009 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> In this paper, we address the problem of detecting pedestrians in still images. We introduce an algorithm for learning shapelet features, a set of mid-level features. These features are focused on local regions of the image and are built from low-level gradient information that discriminates between pedestrian and non-pedestrian classes. Using Ad-aBoost, these shapelet features are created as a combination of oriented gradient responses. To train the final classifier, we use AdaBoost for a second time to select a subset of our learned shapelets. By first focusing locally on smaller feature sets, our algorithm attempts to harvest more useful information than by examining all the low-level features together. We present quantitative results demonstrating the effectiveness of our algorithm. In particular, we obtain an error rate 14 percentage points lower (at 10-6 FPPW) than the previous state of the art detector of Dalal and Triggs on the INRIA dataset. <s> BIB010 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> We address the problem of estimating human pose in video sequences, where rough location has been determined. We exploit both appearance and motion information by defining suitable features of an image and its temporal neighbors, and learning a regression map to the parameters of a model of the human body using boosting techniques. Our algorithm can be viewed as a fast initialization step for human body trackers, or as a tracker itself. We extend gradient boosting techniques to learn a multi-dimensional map from (rotated and scaled) Haar features to the entire set of joint angles representing the full body pose. We test our approach by learning a map from image patches to body joint angles from synchronized video and motion capture walking data. We show how our technique enables learning an efficient real-time pose estimator, validated on publicly available datasets. <s> BIB011 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> This paper presents a novel algorithm for performing integrated segmentation and 3D pose estimation of a human body from multiple views. Unlike other state of the art methods which focus on either segmentation or pose estimation individually, our approach tackles these two tasks together. Our method works by optimizing a cost function based on a Conditional Random Field (CRF). This has the advantage that all information in the image (edges, background and foreground appearances), as well as the prior information on the shape and pose of the subject can be combined and used in a Bayesian framework. Optimizing such a cost function would have been computationally infeasible. However, our recent research in dynamic graph cuts allows this to be done much more efficiently than before. We demonstrate the efficacy of our approach on challenging motion sequences. Although we target the human pose inference problem in the paper, our method is completely generic and can be used to segment and infer the pose of any rigid, deformable or articulated object. <s> BIB012 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> This paper addresses the pose recovery problem of a particular articulated object: the human body. In this model-based approach, the 2D-shape is associated to the corresponding stick figure allowing the joint segmentation and pose recovery of the subject observed in the scene. The main disadvantage of 2D-models is their restriction to the viewpoint. To cope with this limitation, local spatio-temporal 2D-models corresponding to many views of the same sequences are trained, concatenated and sorted in a global framework. Temporal and spatial constraints are then considered to build the probabilistic transition matrix (PTM) that gives a frame to frame estimation of the most probable local models to use during the fitting procedure, thus limiting the feature space. This approach takes advantage of 3D information avoiding the use of a complex 3D human model. The experiments carried out on both indoor and outdoor sequences have demonstrated the ability of this approach to adequately segment pedestrians and estimate their poses independently of the direction of motion during the sequence. <s> BIB013 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> The introduction of support vector regression (SVR) and least square support vector machines (LS-SVM) methods for regression purposes in the field of chemometrics has provided advantageous alternatives to the existing linear and nonlinear multivariate calibration (MVC) approaches. Relevance vector machines (RVMs) claim the advantages attributed to all the SVM-based methods over many other regression methods. Additionally, it also exhibits advantages over the standard SVM-based ones since: it is not necessary to estimate the error/margin trade-off parameter C and the insensitivity parameter e in regression tasks, it is applicable to arbitrary basis functions, the algorithm gives probability estimates seamlessly and offer, additionally, excellent sparseness capabilities, which can result in a simple and robust model for the estimation of different properties. This paper presents the use of RVMs as a nonlinear MVC method capable of dealing with ill-posed problems. To study its behavior, three different chemometric benchmark datasets are considered, including both linear and non-linear solutions. RVM was compared with other calibration approaches reported in the literature. Although RVM performance is comparable with the best results obtained by LS-SVM, the final model achieved is sparser, so the prediction process is faster. Taking into account the other advantages attributed to RVMs, it can be concluded that this technique can be seen as a very promising option to solve nonlinear problems in MVC. Copyright © 2008 John Wiley & Sons, Ltd. <s> BIB014 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> This paper addresses human pose recognition from video sequences by formulating it as a classification problem. Unlike much previous work we do not make any assumptions on the availability of clean segmentation. The first step of this work consists in a novel method of aligning the training images using 3D Mocap data. Next we define classes by discretizing a 2D manifold whose two dimensions are camera viewpoint and actions. Our main contribution is a pose detection algorithm based on random forests. A bottom-up approach is followed to build a decision tree by recursively clustering and merging the classes at each level. For each node of the decision tree we build a list of potentially discriminative features using the alignment of training images; in this paper we consider Histograms of Orientated Gradient (HOG). We finally grow an ensemble of trees by randomly sampling one of the selected HOG blocks at each node. Our proposed approach gives promising results with both fixed and moving cameras. <s> BIB015 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> A novel consistent max-covering method is proposed for human pose estimation. We focus on problems in which a rough foreground estimation is available. Pose estimation is formulated as a jigsaw puzzle problem in which the body part tiles maximally cover the foreground region, match local image features, and satisfy body plan and color constraints. This method explicitly imposes a global shape constraint on the body part assembly. It anchors multiple body parts simultaneously and introduces hyperedges in the part relation graph, which is essential for detecting complex poses. Using multiple cues in pose estimation, our method is resistant to cluttered foregrounds. We propose an efficient linear method to solve the consistent max-covering problem. A two-stage relaxation finds the solution in polynomial time. Our experiments on a variety of images and videos show that the proposed method is more robust than previous locally constrained methods. <s> BIB016 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> We present an approach for automatic 3D human pose reconstruction from monocular images, based on a discriminative formulation with latent segmentation inputs. We advanced the field of structured prediction and human pose reconstruction on several fronts. First, by working with a pool of figure-ground segment hypotheses, the prediction problem is formulated in terms of combined learning and inference over segment hypotheses and 3D human articular configurations. Beside constructing tractable formulations for the combined segment selection and pose estimation problem, we propose new augmented kernels that can better encode complex dependencies between output variables. Furthermore, we provide primal linear re-formulations based on Fourier kernel approximations, in order to scale-up the non-linear latent structured prediction methodology. The proposed models are shown to be competitive in the HumanEva benchmark and are also illustrated in a clip collected from a Hollywood movie, where the model can infer human poses from monocular images captured in complex environments. <s> BIB017 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> This work addresses the challenging problem of unconstrained 3D human pose estimation (HPE) from a novel perspective. Existing approaches struggle to operate in realistic applications, mainly due to their scene-dependent priors, such as background segmentation and multi-camera network, which restrict their use in unconstrained environments. We therfore present a framework which applies action detection and 2D pose estimation techniques to infer 3D poses in an unconstrained video. Action detection offers spatiotemporal priors to 3D human pose estimation by both recognising and localising actions in space-time. Instead of holistic features, e.g. silhouettes, we leverage the flexibility of deformable part model to detect 2D body parts as a feature to estimate 3D poses. A new unconstrained pose dataset has been collected to justify the feasibility of our method, which demonstrated promising results, significantly outperforming the relevant state-of-the-arts. <s> BIB018 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> In this paper, we propose a novel method that treats pose estimation as a problem with the constraints of human segmentation consistency from single images. Different from the previous paper, we integrate pose estimation and object segmentation into a joint optimization. With the support of segmentation consistency, we can obtain more reliable pose results. Through analyzing the energy function of pose estimation and human segmentation, we convert the pose estimation into a binary optimization problem that has the same formation as segmentation. The top-down pose shape cues, bottom-up visual cues, and the consistency constraints that penalize the mismatching of pose and human foreground are incorporated into our final objective function. Qualitative and quantitative experimental results demonstrate the merits of our method in pose estimation on Ramanan benchmark and Buffy data sets. <s> BIB019 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Combined Bottom-Up and Top-Down Methods <s> Our goal is to detect humans and estimate their 2D pose in single images. In particular, handling cases of partial visibility where some limbs may be occluded or one person is partially occluding another. Two standard, but disparate, approaches have developed in the field: the first is the part based approach for layout type problems, involving optimising an articulated pictorial structure, the second is the pixel based approach for image labelling involving optimising a random field graph defined on the image. Our novel contribution is a formulation for pose estimation which combines these two models in a principled way in one optimisation problem and thereby inherits the advantages of both of them. Inference on this joint model finds the set of instances of persons in an image, the location of their joints, and a pixel-wise body part labelling. We achieve near or state of the art results on standard human pose data sets, and demonstrate the correct estimation for cases of self-occlusion, person overlap and image truncation. <s> BIB020
The way that bottom-up methods and top-down methods combine is more flexible than the way discriminative and generative methods combine: 1. Combined detection-and recognition-based methods. Motivated by extensive literature on both detection  BIB002 BIB007 BIB005 BIB010  and recognition BIB008 BIB003 BIB011 BIB009 BIB013 BIB014 BIB001 , many works explore the possibility of combing these two types of methods together to enhance estimation accuracy   . For example, by combining the graphical kinematic models with detection methods, the detection and 3D poses could be obtained simultaneously BIB017 BIB012 BIB015 BIB012 . On the other hand, the authors of BIB018 introduce a method of monocular 3D pose estimation from video using action detection on top of a 2D deformable part. 2. Combined pixel-based and part-based methods. Concurrent optimizing object matching and segmentation enables more robust results, since the two closely-related pixel-based and part-based methods support each other BIB016 BIB019 BIB020 . For example, pixel-wise body-part labels can be obtained by combining part-based and pixel-based approaches in a single optimization framework BIB020 . The authors of Bray et al. BIB012 use graph cuts to optimize pose parameters to perform integrated segmentation and 3D pose estimation of a human body. Global minima of energies can be found by graph cut BIB004 , and the graph cut computation is made significantly faster by using the dynamic graph cut algorithm BIB006 .