Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> We develop a view-normalization approach to multi-view face and gait recognition. An image-based visual hull (IBVH) is computed from a set of monocular views and used to render virtual views for tracking and recognition. We determine canonical viewpoints by examining the 3D structure, appearance (texture), and motion of the moving person. For optimal face recognition, we place virtual cameras to capture frontal face appearance; for gait recognition we place virtual cameras to capture a side-view of the person. Multiple cameras can be rendered simultaneously, and camera position is dynamically updated as the person moves through the workspace. Image sequences from each canonical view are passed to an unmodified face or gait recognition algorithm. We show that our approach provides greater recognition accuracy than is obtained using the unnormalized input sequences, and that integrated face and gait recognition provides improved performance over either modality alone. Canonical view estimation, rendering, and recognition have been efficiently implemented and can run at near real-time speeds. <s> BIB001 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> A view-based approach to the representation and recognition of human movement is presented. The basis of the representation is a temporal template-a static vector-image where the vector value at each point is a function of the motion properties at the corresponding spatial location in an image sequence. Using aerobics exercises as a test domain, we explore the representational power of a simple, two component version of the templates: The first value is a binary value indicating the presence of motion and the second value is a function of the recency of motion in a sequence. We then develop a recognition method matching temporal templates against stored instances of views of known actions. The method automatically performs temporal segmentation, is invariant to linear changes in speed, and runs in real-time on standard platforms. <s> BIB002 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Human identification at a distance has recently gained growing interest from computer vision researchers. Gait recognition aims essentially to address this problem by identifying people based on the way they walk. In this paper, a simple but efficient gait recognition algorithm using spatial-temporal silhouette analysis is proposed. For each image sequence, a background subtraction algorithm and a simple correspondence procedure are first used to segment and track the moving silhouettes of a walking figure. Then, eigenspace transformation based on principal component analysis (PCA) is applied to time-varying distance signals derived from a sequence of silhouette images to reduce the dimensionality of the input feature space. Supervised pattern classification techniques are finally performed in the lower-dimensional eigenspace for recognition. This method implicitly captures the structural and transitional characteristics of gait. Extensive experimental results on outdoor image sequences demonstrate that the proposed algorithm has an encouraging recognition performance with relatively low computational cost. <s> BIB003 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> We present a robust representation for gait recognition that is compact, easy to construct, and affords efficient matching. Instead of a time series based representation comprising of a sequence of raw silhouette frames or of features extracted therein, as has been the practice, we simply align and average the silhouettes over one gait cycle. We then base recognition on the Euclidean distance between these averaged silhouette representations. We show, using the recently formulated gait challenge problem (www.gaitchallenge.org), that the improvement in execution time is 30 times while possessing recognition power that is comparable to the gait baseline algorithm, which is becoming the comparison standard in gait recognition. Experiments with portions of the average silhouette representation show that recognition power is not entirely derived from upper body shape, rather the dynamics of the legs also contribute equally to recognition. However, this study does raise intriguing doubts about the need for accurate shape and dynamics representations for gait recognition. <s> BIB004 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Human motion analysis is a recent topic of interest among the computer vision and video processing community. Research in this area is motivated by its wide range of applications such as surveillance and monitoring systems. In this paper we describe a system for recognition of various human actions from compressed video based on motion history information. We introduce the notion of quantifying the motion involved, through what we call Motion Flow History (MFH). The encoded motion information readily available in the compressed MPEG stream is used to construct the coarse Motion History Image (MHI) and the corresponding MFH. The features extracted from the static MHI and MFH compactly characterize the spatio-temporal and motion vector information of the action. Since the features are extracted from the partially decoded sparse motion data, the computational load is minimized to a great extent. The extracted features are used to train the KNN, Neural network, SVM and the Bayes classifiers for recognizing a set of seven human actions. The performance of each feature set with respect to various classifiers are analyzed. q 2003 Elsevier B.V. All rights reserved. <s> BIB005 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Accurate detection of moving objects is an important precursor to stable tracking or recognition. In this paper, we present an object detection scheme that has three innovations over existing approaches. First, the model of the intensities of image pixels as independent random variables is challenged and it is asserted that useful correlation exists in intensities of spatially proximal pixels. This correlation is exploited to sustain high levels of detection accuracy in the presence of dynamic backgrounds. By using a nonparametric density estimation method over a joint domain-range representation of image pixels, multimodal spatial uncertainties and complex dependencies between the domain (location) and range (color) are directly modeled. We propose a model of the background as a single probability density. Second, temporal persistence is proposed as a detection criterion. Unlike previous approaches to object detection which detect objects by building adaptive models of the background, the foregrounds modeled to augment the detection of objects (without explicit tracking) since objects detected in the preceding frame contain substantial evidence for detection in the current frame. Finally, the background and foreground models are used competitively in a MAP-MRF decision framework, stressing spatial context as a condition of detecting interesting objects and the posterior function is maximized efficiently by finding the minimum cut of a capacitated graph. Experimental validation of the proposed method is performed and presented on a diverse set of dynamic scenes. <s> BIB006 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> We present an approach for comparing two sequences of deforming shapes using both parametric models and nonparametric methods. In our approach, Kendall's definition of shape is used for feature extraction. Since the shape feature rests on a non-Euclidean manifold, we propose parametric models like the autoregressive model and autoregressive moving average model on the tangent space and demonstrate the ability of these models to capture the nature of shape deformations using experiments on gait-based human recognition. The nonparametric model is based on dynamic time-warping. We suggest a modification of the dynamic time-warping algorithm to include the nature of the non-Euclidean space in which the shape deformations take place. We also show the efficacy of this algorithm by its application to gait-based human recognition. We exploit the shape deformations of a person's silhouette as a discriminating feature and provide recognition results using the nonparametric model. Our analysis leads to some interesting observations on the role of shape and kinematics in automated gait-based person authentication. <s> BIB007 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Traditional image representations are not suited to conventional classification methods such as the linear discriminant analysis (LDA) because of the undersample problem (USP): the dimensionality of the feature space is much higher than the number of training samples. Motivated by the successes of the two-dimensional LDA (2DLDA) for face recognition, we develop a general tensor discriminant analysis (GTDA) as a preprocessing step for LDA. The benefits of GTDA, compared with existing preprocessing methods such as the principal components analysis (PCA) and 2DLDA, include the following: 1) the USP is reduced in subsequent classification by, for example, LDA, 2) the discriminative information in the training tensors is preserved, and 3) GTDA provides stable recognition rates because the alternating projection optimization algorithm to obtain a solution of GTDA converges, whereas that of 2DLDA does not. We use human gait recognition to validate the proposed GTDA. The averaged gait images are utilized for gait representation. Given the popularity of Gabor-function-based image decompositions for image understanding and object recognition, we develop three different Gabor-function-based image representations: 1) GaborD is the sum of Gabor filter responses over directions, 2) GaborS is the sum of Gabor filter responses over scales, and 3) GaborSD is the sum of Gabor filter responses over scales and directions. The GaborD, GaborS, and GaborSD representations are applied to the problem of recognizing people from their averaged gait images. A large number of experiments were carried out to evaluate the effectiveness (recognition rate) of gait recognition based on first obtaining a Gabor, GaborD, GaborS, or GaborSD image representation, then using GDTA to extract features and, finally, using LDA for classification. The proposed methods achieved good performance for gait recognition based on image sequences from the University of South Florida (USF) HumanID Database. Experimental comparisons are made with nine state-of-the-art classification methods in gait recognition. <s> BIB008 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> A hydraulically actuated platform with 4-degrees of freedom (4-DOF) was designed to be able to apply velocity- or acceleration-controlled floor surface perturbations to freely walking human subjects. The apparatus was required to provide velocity-controlled translational perturbations over the floor surface, rotational perturbations about the ankle joint, and acceleration-controlled vertical translational perturbations. The apparatus was physically constructed, and tested by both measurements of dynamics and repeatability. Crossover of movement from one DOF to another was shown to be less than 1 mm or 0.5 degrees for all desired perturbations. Repeated perturbations were nearly identical with a standard deviation of less than 0.2 mm over translational axes. The application of the platform to human gait research was demonstrated with a protocol of midstance phase perturbations (n=8). For this, the platform controller was programmed to randomly select one out of three conditions: (1) no movement (control); (2) upward perturbation of 0.8 g, 50 mm, 300 ms after heel contact; (3) downward perturbation of 0.8 g, 50 mm, 300 ms after heel contact. In total, 90 trials (3 conditions x 30 repetitions) were recorded for each subject. By singling out the SOL EMG and normalizing and averaging over the subject population, it was shown that the upward and downward perturbations elicited at least two distinctive stereotypical reflex responses in the ankle extensors, opposite in sign. All subjects reported comfort with the apparatus and nobody fell. <s> BIB009 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Gait is one of well recognized biometrics that has been widely used for human identification. However, the current gait recognition might have difficulties due to viewing angle being changed. This is because the viewing angle under which the gait signature database was generated may not be the same as the viewing angle when the probe data are obtained. This paper proposes a new multi-view gait recognition approach which tackles the problems mentioned above. Being different from other approaches of same category, this new method creates a so called View Transformation Model (VTM) based on spatial-domain Gait Energy Image (GEI) by adopting Singular Value Decomposition (SVD) technique. To further improve the performance of the proposed VTM, Linear Discriminant Analysis (LDA) is used to optimize the obtained GEI feature vectors. When implementing SVD there are a few practical problems such as large matrix size and over-fitting. In this paper, reduced SVD is introduced to alleviate the effects caused by these problems. Using the generated VTM, the viewing angles of gallery gait data and probe gait data can be transformed into the same direction. Thus, gait signatures can be measured without difficulties. The extensive experiments show that the proposed algorithm can significantly improve the multiple view gait recognition performance when being compared to the similar methods in literature. <s> BIB010 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> We consider the problem of automatically recognizing human faces from frontal views with varying expression and illumination, as well as occlusion and disguise. We cast the recognition problem as one of classifying among multiple linear regression models and argue that new theory from sparse signal representation offers the key to addressing this problem. Based on a sparse representation computed by C1-minimization, we propose a general classification algorithm for (image-based) object recognition. This new framework provides new insights into two crucial issues in face recognition: feature extraction and robustness to occlusion. For feature extraction, we show that if sparsity in the recognition problem is properly harnessed, the choice of features is no longer critical. What is critical, however, is whether the number of features is sufficiently large and whether the sparse representation is correctly computed. Unconventional features such as downsampled images and random projections perform just as well as conventional features such as eigenfaces and Laplacianfaces, as long as the dimension of the feature space surpasses certain threshold, predicted by the theory of sparse representation. This framework can handle errors due to occlusion and corruption uniformly by exploiting the fact that these errors are often sparse with respect to the standard (pixel) basis. The theory of sparse representation helps predict how much occlusion the recognition algorithm can handle and how to choose the training images to maximize robustness to occlusion. We conduct extensive experiments on publicly available databases to verify the efficacy of the proposed algorithm and corroborate the above claims. <s> BIB011 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Feature subset selection is an important preprocessing step for pattern recognition, to discard irrelevant and redundant information, as well as to identify the most important attributes. In this paper, we investigate a computationally efficient solution to select the most important features for gait recognition. The specific technique applied is based on mutual information (MI), which evaluates the statistical dependence between two random variables and has an established relation with the Bayes classification error. Extending our earlier research, we show that a sequential selection method based on MI can provide an effective solution for high-dimensional human gait data. To assess the performance of the approach, experiments are carried out based on a 73-dimensional model-based gait features set and on a 64 by 64 pixels model-free gait symmetry map on the Southampton HiD Gait database. The experimental results confirm the effectiveness of the method, removing about 50% of the model-based features and 95% of the symmetry map's pixels without significant loss in recognition capability, which outperforms correlation and analysis-of-variance-based methods. <s> BIB012 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Gait is an idiosyncratic biometric that can be used for human identification at a distance and as a result gained growing interest in intelligent visual surveillance. In this pa- per, an efficient gait recognition method based on describing subject outer body contour deformations using wavelet packets is proposed. With the use of Matching Pursuit algorithm, k bases of wavelet packet tree that have maximum similarity to the signal are selected and corresponding coefficients are used as features. Finally, Transductive Support vector Machine (TSVM) classification is utilized on computed eigengait space for semi-supervised identification. The proposed method of selecting features which uses a complete orthogonal or near orthogonal basis from a wavelet packet library of bases and investigating the correlational structure of gait features for each individual using TSVM, result in encouraging identification performance. <s> BIB013 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> The need for more security on mobile devices is increasing with new functionalities and features made available. To improve the device security we propose gait recognition as a protection mechanism. Unlike previous work on gait recognition, which was based on the use of video sources, floor sensors or dedicated high-grade accelerometers, this paper reports the performance when the data is collected with a commercially available mobile device containing low-grade accelerometers. To be more specific, the used mobile device is the Google G1 phone containing the AK8976A embedded accelerometer sensor. The mobile device was placed at the hip on each volunteer to collect gait data. Preproccesing, cycle detection and recognition-analysis were applied to the acceleration signal. The performance of the system was evaluated having 51 volunteers and resulted in an equal error rate (EER) of 20%. <s> BIB014 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> In this paper, we introduce the Minutia Cylinder-Code (MCC): a novel representation based on 3D data structures (called cylinders), built from minutiae distances and angles. The cylinders can be created starting from a subset of the mandatory features (minutiae position and direction) defined by standards like ISO/IEC 19794-2 (2005). Thanks to the cylinder invariance, fixed-length, and bit-oriented coding, some simple but very effective metrics can be defined to compute local similarities and to consolidate them into a global score. Extensive experiments over FVC2006 databases prove the superiority of MCC with respect to three well-known techniques and demonstrate the feasibility of obtaining a very effective (and interoperable) fingerprint recognition implementation for light architectures. <s> BIB015 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> This paper describes the winning algorithm we submitted to the recent NICE.I iris recognition contest. Efficient and robust segmentation of noisy iris images is one of the bottlenecks for non-cooperative iris recognition. To address this problem, a novel iris segmentation algorithm is proposed in this paper. After reflection removal, a clustering based coarse iris localization scheme is first performed to extract a rough position of the iris, as well as to identify non-iris regions such as eyelashes and eyebrows. A novel integrodifferential constellation is then constructed for the localization of pupillary and limbic boundaries, which not only accelerates the traditional integrodifferential operator but also enhances its global convergence. After that, a curvature model and a prediction model are learned to deal with eyelids and eyelashes, respectively. Extensive experiments on the challenging UBIRIS iris image databases demonstrate that encouraging accuracy is achieved by the proposed algorithm which is ranked the best performing algorithm in the recent open contest on iris recognition (the Noisy Iris Challenge Evaluation, NICE.I). <s> BIB016 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> This Review describes how microbial or self DNA that enters the cytoplasm can be detected by various mechanisms and triggers a range of cellular responses. These include the induction of antiviral innate immune responses and inflammasome-dependent caspase-1 activation and pyroptotic cell death. <s> BIB017 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> The basic aim of a biometric identification system is to discriminate automatically between subjects in a reliable and dependable way, according to a specific-target application. Multimodal biometric identification systems aim to fuse two or more physical or behavioral traits to provide optimal False Acceptance Rate (FAR) and False Rejection Rate (FRR), thus improving system accuracy and dependability. In this paper, an innovative multimodal biometric identification system based on iris and fingerprint traits is proposed. The paper is a state-of-the-art advancement of multibiometrics, offering an innovative perspective on features fusion. In greater detail, a frequency-based approach results in a homogeneous biometric vector, integrating iris and fingerprint data. Successively, a hamming-distance-based matching algorithm deals with the unified homogenous biometric vector. The proposed multimodal system achieves interesting results with several commonly used databases. For example, we have obtained an interesting working point with FAR = 0% and FRR = 5.71% using the entire fingerprint verification competition (FVC) 2002 DB2B database and a randomly extracted same-size subset of the BATH database. At the same time, considering the BATH database and the FVC2002 DB2A database, we have obtained a further interesting working point with FAR = 0% and FRR = 7.28% ÷ 9.7%. <s> BIB018 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> A new subspace learning algorithm called locality preserving discriminant projections (LPDP) is proposed by adding the criterion of maximum margin criterion (MMC) into the objective function of locality preserving projections (LPP). LPDP retains the locality preserving characteristic of LPP and utilizes the global discriminative structures obtained from MMC, which can maximize the between-class distance and minimize the within-class distance. Thus, our proposed LPDP combining manifold criterion and Fisher criterion has more discriminanting power, and is more suitable for recognition tasks than LPP, which considers only the local information for classification tasks. Moreover, two kinds of tensorized (multilinear) forms of LPDP are also derived in this paper. One is iterative while the other is non-iterative. The proposed LPDP method is applied to face and palmprint biometrics and is examined using the Yale and ORL face image databases, as well as the PolyU palmprint database. Experimental results demonstrate the effectiveness of the proposed LPDP method. <s> BIB019 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Human identification by gait has created a great deal of interest in computer vision community due to its advantage of inconspicuous recognition at a relatively far distance. This paper provides a comprehensive survey of recent developments on gait recognition approaches. The survey emphasizes on three major issues involved in a general gait recognition system, namely gait image representation, feature dimensionality reduction and gait classification. Also, a review of the available public gait datasets is presented. The concluding discussions outline a number of research challenges and provide promising future directions for the field. <s> BIB020 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Vision-based human action recognition is the process of labeling image sequences with action labels. Robust solutions to this problem have applications in domains such as visual surveillance, video retrieval and human-computer interaction. The task is challenging due to variations in motion performance, recording settings and inter-personal differences. In this survey, we explicitly address these challenges. We provide a detailed overview of current advances in the field. Image representations and the subsequent classification process are discussed separately to focus on the novelties of recent research. Moreover, we discuss limitations of the state of the art and outline promising directions of research. <s> BIB021 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> We describe an object detection system based on mixtures of multiscale deformable part models. Our system is able to represent highly variable object classes and achieves state-of-the-art results in the PASCAL object detection challenges. While deformable part models have become quite popular, their value had not been demonstrated on difficult benchmarks such as the PASCAL data sets. Our system relies on new methods for discriminative training with partially labeled data. We combine a margin-sensitive approach for data-mining hard negative examples with a formalism we call latent SVM. A latent SVM is a reformulation of MI--SVM in terms of latent variables. A latent SVM is semiconvex, and the training problem becomes convex once latent information is specified for the positive examples. This leads to an iterative training algorithm that alternates between fixing latent values for positive examples and optimizing the latent SVM objective function. <s> BIB022 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Recognizing human actions is an important topic in the computer vision community. One of the challenges of recognizing human actions is describing for the variability that arises when arbitrary view camera captures human performing actions. In this paper, we propose a spatio-temporal silhouette representation, called silhouette energy image (SEI), and multiple variability action models, to characterize motion and shape properties for automatic recognition of human actions in daily life. To address the variability in the recognition of human actions, several parameters, such as anthropometry of the person, speed of the action, phase (starting and ending state of an action), camera observations (distance from camera, slanting motion, and rotation of human body), and view variations are proposed. We construct the variability (or adaptable) models based on SEI and the proposed parameters. Global motion descriptors express the spatio-temporal properties of combined energy templates (SEI and variability action models). Our construction of the optimal model for each action and view is based on the support vectors of global motion descriptions of action models. We recognize different daily human actions of different styles successfully in the indoor and outdoor environment. Our experimental results show that the proposed method of human action recognition is robust, flexible and efficient. <s> BIB023 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Using biometric resources to recognize a person has been a recent concentration on computer vision. Previously, biometric research has forced on utilizing iris, finger print, palm print, and shoe print to authenticate and authorized a human. However, these conventional biometric resources suffer from some obviously limitation, such as: strictly distance requirement, too many user cooperation requirement and so on. Compared with the difficulties of utilization through conventional biometric resources, human gait can be easily acquired and utilized in many fields. A human's walk image can reflect the walker's physical characteristics and psychological state, and therefore, the gait feature can be used to recognize a person. In order to achieve better performance of gait recognition we represent the gait image using C1 units, which correspond to the complex cells in human visual cortex, and use a maximum mechanism to keep only the maximum response of each local area of S1 units. To enhance the gait recognition rate, we take the label information into account and utilize the discriminative locality alignment (DLA) method to classify, which is a top level discriminate manifold learning based subspace learning algorithm. Experiment on University of South Florida (USF) dataset shows: (1) the proposed C1Gait+DLA algorithms can achieve better performance than the state-of-art algorithms and (2) DLA can duly preserve both the local geometry and the discriminative information for recognition. <s> BIB024 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> To detect human body and remove noises from complex background, illumination variations and objects, the infrared thermal imaging was applied to collect gait video and an infrared thermal gait database was established in this paper. Multi-variables gait feature was extracted according to a novel method combining integral model and simplified model. Also the wavelet transform, invariant moments and skeleton theory were used to extract gait features. The support vector machine was employed to classify gaits. This proposed method was applied to the infrared gait database and achieved 78%-91% for the probability of correct recognition. The recognition rates were insensitive for the items of holding ball and loading package. However, there was significant influence for the item of wearing heavy coat. The infrared thermal imaging was potential for better description of human body moving within image sequences. <s> BIB025 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Palmprint is a promising biometric feature for use in access control and forensic applications. Previous research on palmprint recognition mainly concentrates on low-resolution (about 100 ppi) palmprints. But for high-security applications (e.g., forensic usage), high-resolution palmprints (500 ppi or higher) are required from which more useful information can be extracted. In this paper, we propose a novel recognition algorithm for high-resolution palmprint. The main contributions of the proposed algorithm include the following: 1) use of multiple features, namely, minutiae, density, orientation, and principal lines, for palmprint recognition to significantly improve the matching performance of the conventional algorithm. 2) Design of a quality-based and adaptive orientation field estimation algorithm which performs better than the existing algorithm in case of regions with a large number of creases. 3) Use of a novel fusion scheme for an identification application which performs better than conventional fusion methods, e.g., weighted sum rule, SVMs, or Neyman-Pearson rule. Besides, we analyze the discriminative power of different feature combinations and find that density is very useful for palmprint recognition. Experimental results on the database containing 14,576 full palmprints show that the proposed algorithm has achieved a good performance. In the case of verification, the recognition system's False Rejection Rate (FRR) is 16 percent, which is 17 percent lower than the best existing algorithm at a False Acceptance Rate (FAR) of 10-5, while in the identification experiment, the rank-1 live-scan partial palmprint recognition rate is improved from 82.0 to 91.7 percent. <s> BIB026 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Biometric gait analysis is to acquire biometric information such as identity, gender, ethnicity and age from people walking patterns. In the walking process, the human body shows regular periodic motion, especially upper and lower limbs, which reflects the individual's unique movement pattern. Compared to other biometrics, gait can be obtained from distance and is difficult to hide and camouflage. During the past ten years, gait has been a hot topic in computer vision with great progress achieved. In this paper, we give a general review and a simple survey of recent gait progresses. <s> BIB027 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Ground reaction force can be used to distinguish different human gait like limb movement. Cumulative foot pressure image is a 2-D data that recorded the spatial and temporal change of ground reaction force during one gait cycle. However, when putting it into practice as a new biometric for gait recognition, it suffers from the problem of large translation variation within class caused by wearing different shoes and walking in different speed. In this paper, an evaluation framework is proposed to address the problem. The framework consists of a database containing cumulative foot pressure images and a well designed benchmark. The data are collected from 118 subjects. A locality-constrained sparse coding scheme is developed to be compared with the benchmark PCA approach. Experimental results show the potential of evaluation framework for evaluating the translation-invariant power of image representation algorithms. <s> BIB028 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Scalability of object detectors with respect to the number of classes is a very important issue for applications where many object classes need to be detected. While combining single-class detectors yields a linear complexity for testing, multi-class detectors that localize all objects at once come often at the cost of a reduced detection accuracy. In this work, we present a scalable multi-class detection algorithm which scales sublinearly with the number of classes without compromising accuracy. To this end, a shared discriminative codebook of feature appearances is jointly trained for all classes and detection is also performed for all classes jointly. Based on the learned sharing distributions of features among classes, we build a taxonomy of object classes. The taxonomy is then exploited to further reduce the cost of multi-class object detection. Our method has linear training and sublinear detection complexity in the number of classes. We have evaluated our method on the challenging PASCAL VOC'06 and PASCAL VOC'07 datasets and show that scaling the system does not lead to a loss in accuracy. <s> BIB029 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> In this paper, we propose a new patch distribution feature (PDF) (i.e., referred to as Gabor-PDF) for human gait recognition. We represent each gait energy image (GEI) as a set of local augmented Gabor features, which concatenate the Gabor features extracted from different scales and different orientations together with the X-Y coordinates. We learn a global Gaussian mixture model (GMM) (i.e., referred to as the universal background model) with the local augmented Gabor features from all the gallery GEIs; then, each gallery or probe GEI is further expressed as the normalized parameters of an image-specific GMM adapted from the global GMM. Observing that one video is naturally represented as a group of GEIs, we also propose a new classification method called locality-constrained group sparse representation (LGSR) to classify each probe video by minimizing the weighted l1, 2 mixed-norm-regularized reconstruction error with respect to the gallery videos. In contrast to the standard group sparse representation method that is a special case of LGSR, the group sparsity and local smooth sparsity constraints are both enforced in LGSR. Our comprehensive experiments on the benchmark USF HumanID database demonstrate the effectiveness of the newly proposed feature Gabor-PDF and the new classification method LGSR for human gait recognition. Moreover, LGSR using the new feature Gabor-PDF achieves the best average Rank-1 and Rank-5 recognition rates on this database among all gait recognition algorithms proposed to date. <s> BIB030 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> A new gait period detection algorithm, dual-ellipse fitting (DEF) approach, is proposed. DEF is that two regions of the whole silhouette divided by the centroid are fitted into two ellipses, respectively. We construct the gait fluctuation as a periodic function which depends on the eccentricities of two halves of the silhouette over time. Experimental results show that the proposed method is robust to scale, translation, direction of walking and carrying a bag. <s> BIB031 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Various methods for gait recognition are summarized by unique metods from anthropometry,spatial temporal,kinematics,kinetics,and video stream data forms.And video stream data are analyzed in detail.The expressions,meanings and characteristics including static,dynamic,and time-varying information among different energy image species methods are compared.Moreover,the fusion of gait features,gait and other biometric and extended gait recognition are reviewed.The assessment methods for gait recognition research are also presented.In addition,the future research directions of gait recognition are addressed. <s> BIB032 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> In this paper, we present a novel and efficient gait recognition system. The proposed system uses two novel gait representations, i.e., the shifted energy image and the gait structural profile, which have increased robustness to some classes of structural variations. Furthermore, we introduce a novel method for the simulation of walking conditions and the generation of artificial subjects that are used for the application of linear discriminant analysis. In the decision stage, the two representations are fused. Thorough experimental evaluation, conducted using one traditional and two new databases, demonstrates the advantages of the proposed system in comparison with current state-of-the-art systems. <s> BIB033 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> In this paper, we present a new model-based approach for human gait recognition in the sagittal plane via deterministic learning (DL) theory. Side silhouette lower limb joint angles characterize the gait system dynamics and are selected as the gait feature. Locally accurate identification of the gait system dynamics is achieved by using radial basis function (RBF) neural networks through DL. The obtained knowledge of the approximated gait system dynamics is stored in constant RBF networks. A gait signature is then derived from the extracted gait system dynamics along the phase portrait of joint angles. A bank of estimators is constructed using constant RBF networks to represent the training gait patterns. By comparing the set of estimators with a test gait pattern, a set of recognition errors are generated, and the average L 1 norms of the errors are taken as the similarity measure between the dynamics of the training gait patterns and the dynamics of the test gait pattern. Therefore, the test gait pattern can be rapidly recognized according to the smallest error principle. In contrast to other existing approaches, the main focus of this paper is on obtaining and reusing the knowledge of the gait system dynamics. Finally, experiments are carried out on the CASIA-A and CASIA-B gait databases to benchmark the effectiveness of the proposed approach. <s> BIB034 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Gait analysis provides a feasible approach for identification in intelligent video surveillance. However, the effectiveness of the dominant silhouette-based approaches is overly dependent upon background subtraction. In this paper, we propose a novel incremental framework based on optical flow, including dynamics learning, pattern retrieval, and recognition. It can greatly improve the usability of gait traits in video surveillance applications. Local binary pattern (LBP) is employed to describe the texture information of optical flow. This representation is called LBP flow, which performs well as a static representation of gait movement. Dynamics within and among gait stances becomes the key consideration for multiframe detection and tracking, which is quite different from existing approaches. To simulate the natural way of knowledge acquisition, an individual hidden Markov model (HMM) representing the gait dynamics of a single subject incrementally evolves from a population model that reflects the average motion process of human gait. It is beneficial for both tracking and recognition and makes the training process of the HMM more robust to noise. Extensive experiments on widely adopted databases have been carried out to show that our proposed approach achieves excellent performance. <s> BIB035 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Abstract The performances of biometrics may be adversely impact by different walking states, walking directions, resolutions of gait sequence images, pose variation and low resolution of face images. To address these problems, we presented a kernel coupled distance metric learning (KCDML) method after considering matching among different data collections. By using a kernel trick and a specialized locality preserving criterion, we formulated the problem of kernel coupled distance metric learning as an optimization problem whose aims are to search for the pair-wise samples staying as close as possible and to preserve the local structure intrinsic data geometry. Instead of an iterative solution, one single generalized eigen-decomposition can be leveraged to compute the two transformation matrices for two classifications of data sets. The effectiveness of the proposed method is empirically demonstrated on gait and face recognition tasks' results which outperform four linear subspace solutions' (i.e. CDML, PCA, LPP, LDA) and four nonlinear subspace solutions' (i.e. Huang's method, PCA-RBF, KPCA, KLPP). <s> BIB036 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> This paper presents a three-phase gait recognition method that analyses the spatio-temporal shape and dynamic motion (STS-DM) characteristics of a human subject's silhouettes to identify the subject in the presence of most of the challenging factors that affect existing gait recognition systems. In phase 1, phase-weighted magnitude spectra of the Fourier descriptor of the silhouette contours at ten phases of a gait period are used to analyse the spatio-temporal changes of the subject's shape. A component-based Fourier descriptor based on anatomical studies of human body is used to achieve robustness against shape variations caused by all common types of small carrying conditions with folded hands, at the subject's back and in upright position. In phase 2, a full-body shape and motion analysis is performed by fitting ellipses to contour segments of ten phases of a gait period and using a histogram matching with Bhattacharyya distance of parameters of the ellipses as dissimilarity scores. In phase 3, dynamic time warping is used to analyse the angular rotation pattern of the subject's leading knee with a consideration of arm-swing over a gait period to achieve identification that is invariant to walking speed, limited clothing variations, hair style changes and shadows under feet. The match scores generated in the three phases are fused using weight-based score-level fusion for robust identification in the presence of missing and distorted frames, and occlusion in the scene. Experimental analyses on various publicly available data sets show that STS-DM outperforms several state-of-the-art gait recognition methods. <s> BIB037 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> In this paper, we tackle the problem of gait recognition based on the model-free approach. Numerous methods exist; they all lead to high dimensional feature spaces. To address the problem of high dimensional feature space, we propose the use of the Random Forest algorithm to rank features' importance. In order to efficiently search throughout subspaces, we apply a backward feature elimination search strategy. Our first experiments are carried out on unknown covariate conditions. Our first results suggest that the selected features contribute to increase the CCR of different existing classification methods. Secondary experiments are performed on unknown covariate conditions and viewpoints. Inspired by the location of our first experiments' features, we proposed a simple mask. Experimental results demonstrate that the proposed mask gives satisfactory results for all angles of the probe and consequently is not view specific. We also show that our mask performs well when an uncooperative experimental setup is considered as compared to the state-of-the art methods. As a consequence, we propose a panoramic gait recognition framework on unknown covariate conditions. Our results suggest that panoramic gait recognition can be performed under unknown covariate conditions. Our approach can greatly reduce the complexity of the classification problem while achieving fair correct classification rates when gait is captured with unknown conditions. <s> BIB038 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> This paper presents the largest inertial sensor-based gait database in the world, which is made open to the research community, and its application to a statistically reliable performance evaluation for gait-based personal authentication. We construct several datasets for both accelerometer and gyroscope of three inertial measurement units and a smartphone around the waist of a subject, which include at most 744 subjects (389 males and 355 females) with ages ranging from 2 to 78 years. The database has several advantages: a large number of subjects with a balanced gender ratio, variations of sensor types, sensor locations, and ground slope conditions. Therefore, we can reliably analyze the dependence of gait authentication performance on a number of factors such as gender, age group, sensor type, ground condition, and sensor location. The results with the latest existing authentication methods provide several insights for these factors. HighlightsWe present the world largest inertial sensor-based database to the community.Based on the database, females have a better recognition performance than males.People have the best recognition performance at their twenties.An accelerometer has a better recognition performance than a gyroscope. <s> BIB039 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> Previous studies have presented algorithms for detection of turns during gait using wearable sensors, but those algorithms were not built for real-time use. This paper therefore investigates the optimal approach for real-time detection of planned turns during gait using wearable inertial measurement units. Several different sensor positions (head, back and legs) and three different detection criteria (orientation, angular velocity and both) are compared with regard to their ability to correctly detect turn onset. Furthermore, the different sensor positions are compared with regard to their ability to predict the turn direction and amplitude. The evaluation was performed on ten healthy subjects who performed left/right turns at three amplitudes (22, 45 and 90 degrees). Results showed that turn onset can be most accurately detected with sensors on the back and using a combination of orientation and angular velocity. The same setup also gives the best prediction of turn direction and amplitude. Preliminary measurements with a single amputee were also performed and highlighted important differences such as slower turning that need to be taken into account. <s> BIB040 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Introduction <s> In this paper, we propose a time-sliced averaged motion history image (TAMHI) alongside the histograms of oriented gradients (HOG) to generate gait signatures in a gait recognition problem. Building on the motion history image (MHI), TAMHI divides the gait cycle into several regular time windows to generate the same number of TAMHI composite images. HOG descriptors are then calculated on these composite images to obtain the gait signature. The time-slicing procedure to produce multi-composite images preserve more detailed transient information of gait cycles. Additionally, time-normalization also introduces gait length invariancy into the representation, hence, offering a better recognition rate to slight changes in walking speed. <s> BIB041
Over the past ten years, gait recognition, which utilizes the manner of walking to identify individuals, has obtained extensive interest in the communities of biometric recognition and video surveillance BIB008 BIB039 BIB010 BIB004 BIB034 BIB035 BIB030 BIB036 BIB014 . Compared to other biometrics, such as face BIB011 , fingerprint BIB015 , palmprint BIB026 , iris BIB016 , DNA BIB017 , or a combination of these traits BIB018 BIB019 , gait offers the potential for recognition at OPEN ACCESS a distance or at low resolution and can be applied inconspicuously. Depending on the sensors used, a gait recognition system can be classified into three groups, which are tactile sensor-based, wearable sensor-based and video sensor-based. Tactile sensors generally refer to the multi-degree-of-freedom pressure sensor BIB009 BIB027 . These tactile sensors are usually placed along a particular floor to collect the pressure signal generated when people walk across it. Figure 1a shows an example of a tactile sensor-based approach. Wearable sensors  are attached or worn on the key points of different body parts, such as the waist, pockets, shoes and so forth (see Figure 1b) , in order to collect the speed, acceleration, position and other information about human gait. Although the sensors can directly access the motion information of specified parts and obtain accuracy data, they require complex equipment in collection and most applications of these methods are limited in medical research. In contrast to tactile sensor-based and wearable sensor-based gait recognition, this survey focuses on the most widely used video sensor-based gait recognition BIB035 BIB001 BIB020 BIB021 . The video sensor-based system typically consists of several digital or analog cameras with suitable optics for acquiring the gait data from a distance (see Figure 1c) . BIB028 ; (b) Wearable sensor-based approach BIB040 : schematic (left) and photograph (right); (c) Video sensor-based approach BIB003 . The general framework of the video sensor-based gait recognition system includes four modules BIB020 , which are preprocessing module BIB006 BIB022 BIB029  BIB031 (i.e., subject detection and silhouette extraction from the original video), feature representation module BIB004 BIB023 BIB032 BIB002 BIB037 BIB038 BIB033 BIB041  BIB005 , feature selection module BIB038 BIB024 BIB012 and classification module BIB025 BIB007 BIB013 . The framework of the video sensor-based gait recognition system is shown in Figure 2a . The camera-based sensor captures gait information and sends the data to computers. Figures 3-10 show templates generated from the periodic sequence in Figure 2b . This survey mainly focuses on the feature representation module.