Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> Pfinder is a real-time system for tracking people and interpreting their behavior. It runs at 10 Hz on a standard SGI Indy computer, and has performed reliably on thousands of people in many different physical locations. The system uses a multiclass statistical model of color and shape to obtain a 2D representation of head and hands in a wide range of viewing conditions. Pfinder has been successfully used in a wide range of applications including wireless interfaces, video databases, and low-bandwidth coding. <s> BIB001 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> A novel approach for estimating articulated body posture and motion from monocular video sequences is proposed. Human pose is defined as the instantaneous two dimensional configuration (i.e. the projection onto the image plane) of a single articulated body in terms of the position of a predetermined sets of joints. First, statistical segmentation of the human bodies from the background is performed and low-level visual features are found given the segmented body shape. The goal is to be able to map these generally low level visual features to body configurations. The system estimates different mappings, each one with a specific cluster in the visual feature space. Given a set of body motion sequences for training, unsupervised clustering is obtained via the Expectation Maximization algorithm. For each of the clusters, a function is estimated to build the mapping between low-level features to 2D pose. Given new visual features, a mapping from each cluster is performed to yield a set of possible poses. From this set, the system selects the most likely pose given the learned probability distribution and the visual feature of the proposed approach is characterized using real and artificially generated body postures, showing promising results. <s> BIB002 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We present a novel similarity measure (likelihood) for estimating three-dimensional human pose from image silhouettes in model-based vision applications. One of the challenges in such approaches is the construction of a model-to-image likelihood that truly reflects the good configurations of the problem. This is hard, commonly due to the violation of consistency principle resulting in the introduction of spurious, unrelated peaks/minima that make the search for model localization difficult. We introduce an entirely continuous formulation which enforces model estimation consistency by means of an attraction/explanation silhouette-based term pair. We subsequently show how the proposed method provides significant consolidation and improved attraction zone around the desired likelihood configurations and elimination of some of the spurious ones. Finally, we present a skeleton-based smoothing method for the image silhouettes that stabilizes and accelerates the search process. <s> BIB003 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> This paper describes a pedestrian detection system that integrates image intensity information with motion information. We use a detection style algorithm that scans a detector over two consecutive frames of a video sequence. The detector is trained (using AdaBoost) to take advantage of both motion and appearance information to detect a walking person. Past approaches have built detectors based on appearance information, but ours is the first to combine both sources of information in a single detector. The implementation described runs at about 4 frames/second, detects pedestrians at very small scales (as small as 20/spl times/15 pixels), and has a very low false positive rate. Our approach builds on the detection work of Viola and Jones. Novel contributions of this paper include: i) development of a representation of image motion which is extremely efficient, and ii) implementation of a state of the art pedestrian detection system which operates on low resolution images under difficult conditions (such as rain and snow). <s> BIB004 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We present an image-based approach to infer 3D structure parameters using a probabilistic "shape+structure" model. The 3D shape of an object class is represented by sets of contours from silhouette views simultaneously observed from multiple calibrated cameras, while structural features of interest on the object are denoted by a number of 3D locations. A prior density over the multiview shape and corresponding structure is constructed with a mixture of probabilistic principal components analyzers. Given a novel set of contours, we infer the unknown structure parameters from the new shape's Bayesian reconstruction. Model matching and parameter inference are done entirely in the image domain and require no explicit 3D construction. Our shape model enables accurate estimation of structure despite segmentation errors or missing views in the input silhouettes, and it works even with only a single input view. Using a training set of thousands of pedestrian images generated from a synthetic model, we can accurately infer the 3D locations of 19 joints on the body based on observed silhouette contours from real images. <s> BIB005 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We aim to infer 3D body pose directly from human silhouettes. Given a visual input (silhouette), the objective is to recover the intrinsic body configuration, recover the viewpoint, reconstruct the input and detect any spatial or temporal outliers. In order to recover intrinsic body configuration (pose) from the visual input (silhouette), we explicitly learn view-based representations of activity manifolds as well as learn mapping functions between such central representations and both the visual input space and the 3D body pose space. The body pose can be recovered in a closed form in two steps by projecting the visual input to the learned representations of the activity manifold, i.e., finding the point on the learned manifold representation corresponding to the visual input, followed by interpolating 3D pose. <s> BIB006 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We describe a mixture density propagation algorithm to estimate 3D human motion in monocular video sequences based on observations encoding the appearance of image silhouettes. Our approach is discriminative rather than generative, therefore it does not require the probabilistic inversion of a predictive observation model. Instead, it uses a large human motion capture data-base and a 3D computer graphics human model in order to synthesize training pairs of typical human configurations together with their realistically rendered 2D silhouettes. These are used to directly learn to predict the conditional state distributions required for 3D body pose tracking and thus avoid using the generative 3D model for inference (the learned discriminative predictors can also be used, complementary, as importance samplers in order to improve mixing or initialize generative inference algorithms). We aim for probabilistically motivated tracking algorithms and for models that can represent complex multivalued mappings common in inverse, uncertain perception inferences. Our paper has three contributions: (1) we establish the density propagation rules for discriminative inference in continuous, temporal chain models; (2) we propose flexible algorithms for learning multimodal state distributions based on compact, conditional Bayesian mixture of experts models; and (3) we demonstrate the algorithms empirically on real and motion capture-based test sequences and compare against nearest-neighbor and regression methods. <s> BIB007 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We present a novel approach for full body pose tracking using stochastic sampling. A volumetric reconstruction of a person is extracted from silhouettes in multiple video images. Then, an articulated body model is fitted to the data with stochastic meta descent (SMD) optimization. By comparing even a simplified version of SMD to the commonly used Levenberg-Marquardt method, we demonstrate the power of stochastic compared to deterministic sampling, especially in cases of noisy and incomplete data. Moreover, color information is added to improve the speed and robustness of the tracking. Results are shown for several challenging sequences, with tracking of 24 degrees of freedom in less than 1 second per frame. <s> BIB008 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We describe a learning-based method for recovering 3D human body pose from single images and monocular image sequences. Our approach requires neither an explicit body model nor prior labeling of body parts in the image. Instead, it recovers pose by direct nonlinear regression against shape descriptor vectors extracted automatically from image silhouettes. For robustness against local silhouette segmentation errors, silhouette shape is encoded by histogram-of-shape-contexts descriptors. We evaluate several different regression methods: ridge regression, relevance vector machine (RVM) regression, and support vector machine (SVM) regression over both linear and kernel bases. The RVMs provide much sparser regressors without compromising performance, and kernel bases give a small but worthwhile improvement in performance. The loss of depth and limb labeling information often makes the recovery of 3D pose from single silhouettes ambiguous. To handle this, the method is embedded in a novel regressive tracking framework, using dynamics from the previous state estimate together with a learned regression value to disambiguate the pose. We show that the resulting system tracks long sequences stably. For realism and good generalization over a wide range of viewpoints, we train the regressors on images resynthesized from real human motion capture data. The method is demonstrated for several representations of full body pose, both quantitatively on independent but similar test data and qualitatively on real image sequences. Mean angular errors of 4-6/spl deg/ are obtained for a variety of walking motions. <s> BIB009 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We address the problem of articulated human pose estimation by learning a coarse-to-fine cascade of pictorial structure models. While the fine-level state-space of poses of individual parts is too large to permit the use of rich appearance models, most possibilities can be ruled out by efficient structured models at a coarser scale. We propose to learn a sequence of structured models at different pose resolutions, where coarse models filter the pose space for the next level via their max-marginals. The cascade is trained to prune as much as possible while preserving true poses for the final level pictorial structure model. The final level uses much more expensive segmentation, contour and shape features in the model for the remaining filtered set of candidates. We evaluate our framework on the challenging Buffy and PASCAL human pose datasets, improving the state-of-the-art. <s> BIB010 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> We describe an object detection system based on mixtures of multiscale deformable part models. Our system is able to represent highly variable object classes and achieves state-of-the-art results in the PASCAL object detection challenges. While deformable part models have become quite popular, their value had not been demonstrated on difficult benchmarks such as the PASCAL data sets. Our system relies on new methods for discriminative training with partially labeled data. We combine a margin-sensitive approach for data-mining hard negative examples with a formalism we call latent SVM. A latent SVM is a reformulation of MI--SVM in terms of latent variables. A latent SVM is semiconvex, and the training problem becomes convex once latent information is specified for the positive examples. This leads to an iterative training algorithm that alternates between fixing latent values for positive examples and optimizing the latent SVM objective function. <s> BIB011 </s> Human Pose Estimation from Monocular Images: A Comprehensive Survey <s> Low-Level Features <s> In the field of human-robot interaction (HRI), detection, tracking and re-identification of humans in a robot's surroundings are crucial tasks, e. g. for socially compliant robot navigation. Besides the 3D position detection, the estimation of a person's upper-body orientation based on monocular camera images is a challenging problem on a mobile platform. To obtain real-time position tracking as well as upper-body orientation estimations, the proposed system comprises discriminative detectors whose hypotheses are tracked by a Kalman filter-based multi-hypotheses tracker. For appearance-based person recognition, a generative approach, based on a 3D shape model, is used to refine these tracked hypotheses. This model evaluates edges and color-based discrimination from the background. Furthermore, for each person the texture of his or her upper-body is learned and used for person re-identification. When computational resources are limited, the update rate of the model-based optimization reduces itself automatically. Thereby the estimation accuracy decreases, but the system keeps tracking the persons around the robot in real-time. The person's 3D pose is tracked up to a distance of 5.0 meters with an average Euclidean error of 18 cm. The achieved motion independent average upper-body orientation error is 22°. Furthermore, the upper-body texture is learned on-line which allowed a stable person re-identification in our experiments. <s> BIB012
To capture appearance, geometry, and shape information of human body parts, features commonly extracted are silhouettes BIB002 BIB009  BIB006 , contours BIB004 BIB010 , edges  BIB012 , etc. Silhouettes extract outlines of objects and are invariant to texture and lighting BIB009 BIB007 BIB005 BIB008 BIB003 . Contour captures the outline of body parts and is a path with edges linking crossing points of segmentation boundaries BIB010 . Edges extract sharply varying lines in images and are usually computed by convolution. In comparison, silhouettes are global descriptors enclosing an overall view of an object and usually require prior knowledge of the background to extract the foreground object, as shown in Figure 3 ; Contours require pre-processing (such as segmentation), and they enclose details in addition to outline information, as shown in Figure 4 ; Edges are rather scattered features and can be computed directly from filtering, as shown in Figure 2b . Figure 2b shows examples of edge filters for convolution and detected edge examples in  . Figure 2a shows Haar features as an example of edge and line filters. Other features that model body part appearance include color BIB010 BIB001  and texture BIB011 . 