Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> In this article the author uses projective relations as the theoretical foundation of his investigations of visual space and motion. Several laboratory experiments involving perceptual vector analysis and its geometric basis are described. In most of the experiments the visual stimuli consisted of computer-controlled patterns displayed on a televisionlike screen and projected into the eyes of subjects by means of a collimating device that removed parallax as well as the possibility of seeing the screen. A common characteristics of the experiments was that the observer was evidently not free to choose between a Euclidean interpretation of the changing geometry of the figure in the display and a projective interpretation. For example, the observer could not persuade himself that what he was seeing was simply a square growing larger and smaller in the same visual plane; his visual system insisted on telling him that he was seeing a square of constant size approaching and receding. Hence he perceived rigid motion in depth, rotation in a specific slant, bending in depth and so on, paired with the highest possible degree of object constancy. Further experiments were conducted to determine if the principles of perceptual analysis hold true for the more complex paterns of motions encountered in everyday life. These experiments led to the conclusion that during locomotion the components of the human visual environment are interpreted as rigid structures in relative motion. <s> BIB001 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> We describe a novel algorithm for gait analysis. A person walking frontoparallel to the image plane generates a characteristic "braided" pattern in a spatiotemporal (XYT) volume. Our algorithm detects this pattern, and fits it with a set of spatiotemporal snakes. The snakes can be used to find the bounding contours of the walker. The contours vary over time in a manner characteristic of each walker. Individual gaits can be recognized by applying standard pattern recognition techniques to the contour signals. > <s> BIB002 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> This paper demonstrates gait recognition using only the trajectories of lower body joint angles projected into the walking plane. For this work, we begin with the position of 3D markers as projected into the sagittal or walking plane. We show a simple method for estimating the planar offsets between the markers and the underlying skeleton and joints; given these offsets we compute the joint angle trajectories. To compensate for systematic temporal variations from one instance to the next-predominantly distance and speed of walk-we fix the number of footsteps and time-normalize the trajectories by a variance compensated time warping. We perform recognition on two walking databases of 18 people (over 150 walk instances) using simple nearest neighbor algorithm with Euclidean distance as a measurement criteria. We also use the expected confusion metric as a means to estimate how well joint-angle signals will perform in a larger population. <s> BIB003 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> A gait-recognition technique that recovers static body and stride parameters of subjects as they walk is presented. This approach is an example of an activity-specific biometric: a method of extracting identifying properties of an individual or of an individual's behavior that is applicable only when a person is performing that specific action. To evaluate our parameters, we derive an expected confusion metric (related to mutual information), as opposed to reporting a percent correct with a limited database. This metric predicts how well a given feature vector will filter identity in a large population. We test the utility of a variety of body and stride parameters recovered in different viewing conditions on a database consisting of 15 to 20 subjects walking at both an angled and frontal-parallel view with respect to the camera, both indoors and out. We also analyze motion-capture data of the subjects to discover whether confusion in the parameters is inherently a physical or a visual measurement error property. <s> BIB004 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> We develop a view-normalization approach to multi-view face and gait recognition. An image-based visual hull (IBVH) is computed from a set of monocular views and used to render virtual views for tracking and recognition. We determine canonical viewpoints by examining the 3D structure, appearance (texture), and motion of the moving person. For optimal face recognition, we place virtual cameras to capture frontal face appearance; for gait recognition we place virtual cameras to capture a side-view of the person. Multiple cameras can be rendered simultaneously, and camera position is dynamically updated as the person moves through the workspace. Image sequences from each canonical view are passed to an unmodified face or gait recognition algorithm. We show that our approach provides greater recognition accuracy than is obtained using the unnormalized input sequences, and that integrated face and gait recognition provides improved performance over either modality alone. Canonical view estimation, rendering, and recognition have been efficiently implemented and can run at near real-time speeds. <s> BIB005 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> A view-based approach to the representation and recognition of human movement is presented. The basis of the representation is a temporal template-a static vector-image where the vector value at each point is a function of the motion properties at the corresponding spatial location in an image sequence. Using aerobics exercises as a test domain, we explore the representational power of a simple, two component version of the templates: The first value is a binary value indicating the presence of motion and the second value is a function of the recency of motion in a sequence. We then develop a recognition method matching temporal templates against stored instances of views of known actions. The method automatically performs temporal segmentation, is invariant to linear changes in speed, and runs in real-time on standard platforms. <s> BIB006 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> In this paper, we develop a novel method for view-based recognition of human action/activity from videos. By observing just a few frames, we can identify the activity that takes place in a video sequence. The basic idea of our method is that activities can be positively identified from a sparsely sampled sequence of a few body poses acquired from videos. In our approach, an activity is represented by a set of pose and velocity vectors for the major body parts (hands, legs, and torso) and stored in a set of multidimensional hash tables. We develop a theoretical foundation that shows that robust recognition of a sequence of body pose vectors can be achieved by a method of indexing and sequencing and it requires only a few pose vectors (i.e., sampled body poses in video frames). We find that the probability of false alarm drops exponentially with the increased number of sampled body poses. So, matching only a few body poses guarantees high probability for correct recognition. Our approach is parallel, i.e., all possible model activities are examined at one indexing operation. In addition, our method is robust to partial occlusion since each body part is indexed separately. We use a sequence-based voting approach to recognize the activity invariant to the activity speed. <s> BIB007 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> We describe a representation of gait appearance for the purpose of person identification and classification. This gait representation is based on simple features such as moments extracted from orthogonal view video silhouettes of human walking motion. Despite its simplicity, the resulting feature vector contains enough information to perform well on human identification and gender classification tasks. We explore the recognition behaviors of two different methods to aggregate features over time under different recognition tasks. We demonstrate the accuracy of recognition using gait video sequences collected over different days and times and under varying lighting environments. In addition, we show results for gender classification based our gait appearance features using a support-vector machine. <s> BIB008 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Gait is a spatio-temporal phenomenon that typifies the motion characteristics of an individual. In this paper, we propose a view-based approach to recognize humans through gait. The width of the outer contour of the binarized silhouette of a walking person is chosen as the image feature. A set of stances or key frames that occur during the walk cycle of an individual is chosen. Euclidean distances of a given image from this stance set are computed and a lower-dimensional observation vector is generated. A continuous hidden Markov model (HMM) is trained using several such lower-dimensional vector sequences extracted from the video. This methodology serves to compactly capture structural and transitional features that are unique to an individual. The statistical nature of the HMM renders overall robustness to gait representation and recognition. The human identification performance of the proposed scheme is found to be quite good when tested in natural walking conditions. <s> BIB009 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Gait enjoys advantages over other biometrics in that it can be perceived from a distance and is di cult to disguise.Current approaches are mostly statistical and concentrate on walking only.By analysing leg motion we show how we can recognise people not only by the walking gait,but also by the running gait.This is achieved by either of two new modelling approaches which employ coupled oscillators and the biomechanics of human locomotion as the underlying concepts.These models give a plausible method for data reduction by providing estimates of the inclination of the thigh and of the leg,from the image data. Both approaches derive a phase-weighted Fourier description gait signature by automated non-invasive means.One approach is completely automated whereas the other requires speci cation of a single parameter to distinguish between walking and running.Results show that both gaits are potential biometrics,with running being more potent.By its basis in evidence gathering,this new technique can tolerate noise and low resolution. <s> BIB010 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> We present a robust representation for gait recognition that is compact, easy to construct, and affords efficient matching. Instead of a time series based representation comprising of a sequence of raw silhouette frames or of features extracted therein, as has been the practice, we simply align and average the silhouettes over one gait cycle. We then base recognition on the Euclidean distance between these averaged silhouette representations. We show, using the recently formulated gait challenge problem (www.gaitchallenge.org), that the improvement in execution time is 30 times while possessing recognition power that is comparable to the gait baseline algorithm, which is becoming the comparison standard in gait recognition. Experiments with portions of the average silhouette representation show that recognition power is not entirely derived from upper body shape, rather the dynamics of the legs also contribute equally to recognition. However, this study does raise intriguing doubts about the need for accurate shape and dynamics representations for gait recognition. <s> BIB011 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Human motion analysis is a recent topic of interest among the computer vision and video processing community. Research in this area is motivated by its wide range of applications such as surveillance and monitoring systems. In this paper we describe a system for recognition of various human actions from compressed video based on motion history information. We introduce the notion of quantifying the motion involved, through what we call Motion Flow History (MFH). The encoded motion information readily available in the compressed MPEG stream is used to construct the coarse Motion History Image (MHI) and the corresponding MFH. The features extracted from the static MHI and MFH compactly characterize the spatio-temporal and motion vector information of the action. Since the features are extracted from the partially decoded sparse motion data, the computational load is minimized to a great extent. The extracted features are used to train the KNN, Neural network, SVM and the Bayes classifiers for recognizing a set of seven human actions. The performance of each feature set with respect to various classifiers are analyzed. q 2003 Elsevier B.V. All rights reserved. <s> BIB012 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Identification of people by analysis of gait patterns extracted from video has recently become a popular research problem. However, the conditions under which the problem is "solvable" are not understood or characterized. To provide a means for measuring progress and characterizing the properties of gait recognition, we introduce the humanlD gait challenge problem. The challenge problem consists of a baseline algorithm, a set of 12 experiments, and a large data set. The baseline algorithm estimates silhouettes by background subtraction and performs recognition by temporal correlation of silhouettes. The 12 experiments are of increasing difficulty, as measured by the baseline algorithm, and examine the effects of five covariates on performance. The covariates are: change in viewing angle, change in shoe type, change in walking surface, carrying or not carrying a briefcase, and elapsed time between sequences being compared. Identification rates for the 12 experiments range from 78 percent on the easiest experiment to 3 percent on the hardest. All five covariates had statistically significant effects on performance, with walking surface and time difference having the greatest impact. The data set consists of 1,870 sequences from 122 subjects spanning five covariates (1.2 gigabytes of data). This infrastructure supports further development of gait recognition algorithms and additional experiments to understand the strengths and weaknesses of new algorithms. The experimental results are presented, the more detailed is the possible meta-analysis and greater is the understanding. It is this potential from the adoption of this challenge problem that represents a radical departure from traditional computer vision research methodology. <s> BIB013 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Gait recognition has gained increasing interest from researchers, but there is still no standard evaluation method to compare the performance of different gait recognition algorithms. In this paper, a framework is proposed in an attempt to tackle this problem. The framework consists of a large gait database, a large set of well designed experiments and some evaluation metrics. There are 124 subjects in the database, and the gait data was captured from 11 views. Three variations, namely view angle, clothing and carrying condition changes, are separately considered in the database. The database is one of the largest database among the existing databases. Three sets of experiments, including a total of 363 experiments, are designed in the framework. Some metrics are proposed to evaluate gait recognition algorithms. <s> BIB014 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> In this paper, we propose a new spatio-temporal gait representation, called Gait Energy Image (GEI), to characterize human walking properties for individual recognition by gait. To address the problem of the lack of training templates, we also propose a novel approach for human recognition by combining statistical gait features from real and synthetic templates. We directly compute the real templates from training silhouette sequences, while we generate the synthetic templates from training sequences by simulating silhouette distortion. We use a statistical approach for learning effective features from real and synthetic templates. We compare the proposed GEI-based gait recognition approach with other gait recognition approaches on USF HumanID Database. Experimental results show that the proposed GEI is an effective and efficient gait representation for individual recognition, and the proposed approach achieves highly competitive performance with respect to the published gait recognition approaches <s> BIB015 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Gait is one of well recognized biometrics that has been widely used for human identification. However, the current gait recognition might have difficulties due to viewing angle being changed. This is because the viewing angle under which the gait signature database was generated may not be the same as the viewing angle when the probe data are obtained. This paper proposes a new multi-view gait recognition approach which tackles the problems mentioned above. Being different from other approaches of same category, this new method creates a so called View Transformation Model (VTM) based on spatial-domain Gait Energy Image (GEI) by adopting Singular Value Decomposition (SVD) technique. To further improve the performance of the proposed VTM, Linear Discriminant Analysis (LDA) is used to optimize the obtained GEI feature vectors. When implementing SVD there are a few practical problems such as large matrix size and over-fitting. In this paper, reduced SVD is introduced to alleviate the effects caused by these problems. Using the generated VTM, the viewing angles of gallery gait data and probe gait data can be transformed into the same direction. Thus, gait signatures can be measured without difficulties. The extensive experiments show that the proposed algorithm can significantly improve the multiple view gait recognition performance when being compared to the similar methods in literature. <s> BIB016 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> We have presented a model-based approach for human gait recognition, which is based on analyzing the leg and arm movements. An initial model is created based on anatomical proportions, and a posterior model is constructed upon the movements of the articulated parts of the body, using active contour models and the Hough transform. Fourier analysis is used to describe the motion patterns of the moving parts. The k-nearest neighbor rule applied to the phase-weighted Fourier magnitude of each segment's spectrum is used for classification. In contrast to the existing approaches, the main focus of this paper is on increasing the discrimination capability of the model through extra features produced from the motion of the arms. Experimental results indicate good performance of the proposed method. The technique has also proved to be able to reduce the adverse effects of self-occlusion, which is a common incident in human walking. <s> BIB017 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Recognizing human actions is an important topic in the computer vision community. One of the challenges of recognizing human actions is describing for the variability that arises when arbitrary view camera captures human performing actions. In this paper, we propose a spatio-temporal silhouette representation, called silhouette energy image (SEI), and multiple variability action models, to characterize motion and shape properties for automatic recognition of human actions in daily life. To address the variability in the recognition of human actions, several parameters, such as anthropometry of the person, speed of the action, phase (starting and ending state of an action), camera observations (distance from camera, slanting motion, and rotation of human body), and view variations are proposed. We construct the variability (or adaptable) models based on SEI and the proposed parameters. Global motion descriptors express the spatio-temporal properties of combined energy templates (SEI and variability action models). Our construction of the optimal model for each action and view is based on the support vectors of global motion descriptions of action models. We recognize different daily human actions of different styles successfully in the indoor and outdoor environment. Our experimental results show that the proposed method of human action recognition is robust, flexible and efficient. <s> BIB018 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> The strength of gait, compared to other biometrics, is that it does not require cooperative subjects. In previous work gait recognition approaches were evaluated using a gallery set consisting of gait sequences of people under similar covariate conditions (e.g. clothing, surface, carrying, and view conditions). This evaluation procedure, however, implies that the gait data are collected in a cooperative manner so that the covariate conditions are known a priori. In this work, gait recognition approaches are evaluated without the assumption on cooperative subjects, i.e. both the gallery and the probe sets consist of a mixture of gait sequences under different and unknown covariate conditions. The results indicate that the performance of the existing approaches would drop drastically under this more realistic experimental setup. We argue that selecting the most relevant gait features that are invariant to changes in gait covariate conditions is the key to develop a gait recognition system that works without subject cooperation. To this end, Gait Entropy Image (GEnI) is proposed to perform automatic feature selection on each pair of gallery and probe gait sequences. Moreover, an Adaptive Component and Discriminant Analysis (ACDA) is formulated which seamlessly integrates our feature selection method with subspace analysis for robust recognition, and importantly is computationally much more efficient compared to the conventional Component and Discriminant Analysis. Experiments are carried out on two comprehensive benchmarking databases: the CASIA database and the Southampton Human ID at a distance gait database (SOTON database). Our results demonstrate that the proposed approach significantly outperforms the existing techniques particularly when gait is captured with variable and unknown covariate conditions. <s> BIB019 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Human identification by gait has created a great deal of interest in computer vision community due to its advantage of inconspicuous recognition at a relatively far distance. This paper provides a comprehensive survey of recent developments on gait recognition approaches. The survey emphasizes on three major issues involved in a general gait recognition system, namely gait image representation, feature dimensionality reduction and gait classification. Also, a review of the available public gait datasets is presented. The concluding discussions outline a number of research challenges and provide promising future directions for the field. <s> BIB020 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Gait is a well recognized biometric feature that is used to identify a human at a distance. However, in real environment, appearance changes of individuals due to viewing angle changes cause many difficulties for gait recognition. This paper re-formulates this problem as a regression problem. A novel solution is proposed to create a View Transformation Model (VTM) from the different point of view using Support Vector Regression (SVR). To facilitate the process of regression, a new method is proposed to seek local Region of Interest (ROI) under one viewing angle for predicting the corresponding motion information under another viewing angle. Thus, the well constructed VTM is able to transfer gait information under one viewing angle into another viewing angle. This proposal can achieve view-independent gait recognition. It normalizes gait features under various viewing angles into a common viewing angle before similarity measurement is carried out. The extensive experimental results based on widely adopted benchmark dataset demonstrate that the proposed algorithm can achieve significantly better performance than the existing methods in literature. <s> BIB021 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> As a new technology of biometrics, gait recognition has attracted a great deal of interest in computer vision community due to its advantage of unobtrusive recognition at a distance. In this paper, a review of this topic is presented in terms of approaches, databases and challenges. First, a comprehensive survey of recent developments on gait recognition approaches is reported, where relevant techniques are categorized into three classes for discussions. Then, more than 10 publically available datasets are analysed and compared in detail. In addition, challenges that constrain practical application of gait recognition systems are discussed. Finally, future trends are given to guide further research in this field. (6 pages) <s> BIB022 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Recent gait recognition systems often suffer from the challenges including viewing angle variation and large intra-class variations. In order to address these challenges, this paper presents a robust View Transformation Model for gait recognition. Based on the gait energy image, the proposed method establishes a robust view transformation model via robust principal component analysis. Partial least square is used as feature selection method. Compared with the existing methods, the proposed method finds out a shared linear correlated low rank subspace, which brings the advantages that the view transformation model is robust to viewing angle variation, clothing and carrying condition changes. Conducted on the CASIA gait dataset, experimental results show that the proposed method outperforms the other existing methods. <s> BIB023 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Various methods for gait recognition are summarized by unique metods from anthropometry,spatial temporal,kinematics,kinetics,and video stream data forms.And video stream data are analyzed in detail.The expressions,meanings and characteristics including static,dynamic,and time-varying information among different energy image species methods are compared.Moreover,the fusion of gait features,gait and other biometric and extended gait recognition are reviewed.The assessment methods for gait recognition research are also presented.In addition,the future research directions of gait recognition are addressed. <s> BIB024 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> In this paper, we present a novel and efficient gait recognition system. The proposed system uses two novel gait representations, i.e., the shifted energy image and the gait structural profile, which have increased robustness to some classes of structural variations. Furthermore, we introduce a novel method for the simulation of walking conditions and the generation of artificial subjects that are used for the application of linear discriminant analysis. In the decision stage, the two representations are fused. Thorough experimental evaluation, conducted using one traditional and two new databases, demonstrates the advantages of the proposed system in comparison with current state-of-the-art systems. <s> BIB025 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> It is well recognized that gait is an important biometric feature to identify a person at a distance, e.g., in video surveillance application. However, in reality, change of viewing angle causes significant challenge for gait recognition. A novel approach using regression-based view transformation model (VTM) is proposed to address this challenge. Gait features from across views can be normalized into a common view using learned VTM(s). In principle, a VTM is used to transform gait feature from one viewing angle (source) into another viewing angle (target). It consists of multiple regression processes to explore correlated walking motions, which are encoded in gait features, between source and target views. In the learning processes, sparse regression based on the elastic net is adopted as the regression function, which is free from the problem of overfitting and results in more stable regression models for VTM construction. Based on widely adopted gait database, experimental results show that the proposed method significantly improves upon existing VTM-based methods and outperforms most other baseline methods reported in the literature. Several practical scenarios of applying the proposed method for gait recognition under various views are also discussed in this paper. <s> BIB026 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Highlights? One-view to one-view VTM solves across-view gait recognition. ?Multi-view to one-view VTM solves multi-view gait recognition. ?VTM is built up from regressions of local motion using multi-layer perceptron. ?Multi-view to one-view VTM can be claimed as implicit 3D gait model based method. ?Accuracy of 98% is achieved for multi-view gait recognition using 4 cameras. Gait has been shown to be an efficient biometric feature for human identification at a distance. However, performance of gait recognition can be affected by view variation. This leads to a consequent difficulty of cross-view gait recognition. A novel method is proposed to solve the above difficulty by using view transformation model (VTM). VTM is constructed based on regression processes by adopting multi-layer perceptron (MLP) as a regression tool. VTM estimates gait feature from one view using a well selected region of interest (ROI) on gait feature from another view. Thus, trained VTMs can normalize gait features from across views into the same view before gait similarity is measured. Moreover, this paper proposes a new multi-view gait recognition which estimates gait feature on one view using selected gait features from several other views. Extensive experimental results demonstrate that the proposed method significantly outperforms other baseline methods in literature for both cross-view and multi-view gait recognitions. In our experiments, particularly, average accuracies of 99%, 98% and 93% are achieved for multiple views gait recognition by using 5 cameras, 4 cameras and 3 cameras respectively. <s> BIB027 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> In this paper, we present a new model-based approach for human gait recognition in the sagittal plane via deterministic learning (DL) theory. Side silhouette lower limb joint angles characterize the gait system dynamics and are selected as the gait feature. Locally accurate identification of the gait system dynamics is achieved by using radial basis function (RBF) neural networks through DL. The obtained knowledge of the approximated gait system dynamics is stored in constant RBF networks. A gait signature is then derived from the extracted gait system dynamics along the phase portrait of joint angles. A bank of estimators is constructed using constant RBF networks to represent the training gait patterns. By comparing the set of estimators with a test gait pattern, a set of recognition errors are generated, and the average L 1 norms of the errors are taken as the similarity measure between the dynamics of the training gait patterns and the dynamics of the test gait pattern. Therefore, the test gait pattern can be rapidly recognized according to the smallest error principle. In contrast to other existing approaches, the main focus of this paper is on obtaining and reusing the knowledge of the gait system dynamics. Finally, experiments are carried out on the CASIA-A and CASIA-B gait databases to benchmark the effectiveness of the proposed approach. <s> BIB028 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> This paper presents a three-phase gait recognition method that analyses the spatio-temporal shape and dynamic motion (STS-DM) characteristics of a human subject's silhouettes to identify the subject in the presence of most of the challenging factors that affect existing gait recognition systems. In phase 1, phase-weighted magnitude spectra of the Fourier descriptor of the silhouette contours at ten phases of a gait period are used to analyse the spatio-temporal changes of the subject's shape. A component-based Fourier descriptor based on anatomical studies of human body is used to achieve robustness against shape variations caused by all common types of small carrying conditions with folded hands, at the subject's back and in upright position. In phase 2, a full-body shape and motion analysis is performed by fitting ellipses to contour segments of ten phases of a gait period and using a histogram matching with Bhattacharyya distance of parameters of the ellipses as dissimilarity scores. In phase 3, dynamic time warping is used to analyse the angular rotation pattern of the subject's leading knee with a consideration of arm-swing over a gait period to achieve identification that is invariant to walking speed, limited clothing variations, hair style changes and shadows under feet. The match scores generated in the three phases are fused using weight-based score-level fusion for robust identification in the presence of missing and distorted frames, and occlusion in the scene. Experimental analyses on various publicly available data sets show that STS-DM outperforms several state-of-the-art gait recognition methods. <s> BIB029 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> In this paper, we tackle the problem of gait recognition based on the model-free approach. Numerous methods exist; they all lead to high dimensional feature spaces. To address the problem of high dimensional feature space, we propose the use of the Random Forest algorithm to rank features' importance. In order to efficiently search throughout subspaces, we apply a backward feature elimination search strategy. Our first experiments are carried out on unknown covariate conditions. Our first results suggest that the selected features contribute to increase the CCR of different existing classification methods. Secondary experiments are performed on unknown covariate conditions and viewpoints. Inspired by the location of our first experiments' features, we proposed a simple mask. Experimental results demonstrate that the proposed mask gives satisfactory results for all angles of the probe and consequently is not view specific. We also show that our mask performs well when an uncooperative experimental setup is considered as compared to the state-of-the art methods. As a consequence, we propose a panoramic gait recognition framework on unknown covariate conditions. Our results suggest that panoramic gait recognition can be performed under unknown covariate conditions. Our approach can greatly reduce the complexity of the classification problem while achieving fair correct classification rates when gait is captured with unknown conditions. <s> BIB030 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Human gait has become an important biometric in recent years. A silhouette based method is suggested in this paper, to recognize human in video by their gait. We used averaged silhouette to represent the gait cycle. Principal Component Analysis has been used to reduce the dimensionality of the features. We applied Euclidean distance to measure the similarity of the averaged silhouettes. We implemented the algorithm on the TUM- IITKGP Gait Database which has been introduced recently. Although this method is sensitive to the appearance of the subject, it has low computational cost and it is simple. We implemented two experiments on the achieved averaged silhouettes. Our results illustrated that feature extracted from the averaged silhouettes which in them, the lower part of the body is eliminated are more suitable rather than those extracted from the complete averaged silhouettes.  <s> BIB031 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Global security concerns have raised a proliferation of video surveillance devices. Intelligent surveillance systems seek to discover possible threats automatically and raise alerts. Being able to identify the surveyed object can help determine its threat level. The current generation of devices provide digital video data to be analysed for time varying features to assist in the identification process. Commonly, people queue up to access a facility and approach a video camera in full frontal view. In this environment, a variety of biometrics are available--for example, gait which includes temporal features like stride period. Gait can be measured unobtrusively at a distance. The video data will also include face features, which are short-range biometrics. In this way, one can combine biometrics naturally using one set of data. In this paper we survey current techniques of gait recognition and modelling with the environment in which the research was conducted. We also discuss in detail the issues arising from deriving gait data, such as perspective and occlusion effects, together with the associated computer vision challenges of reliable tracking of human movement. Then, after highlighting these issues and challenges related to gait processing, we proceed to discuss the frameworks combining gait with other biometrics. We then provide motivations for a novel paradigm in biometrics-based human recognition, i.e. the use of the fronto-normal view of gait as a far-range biometrics combined with biometrics operating at a near distance. <s> BIB032 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> Closed-circuit television (CCTV) footage is used in criminal investigations to compare perpetrators with suspects. Usually, incomplete gait cycles are collected, making evidential gait analysis challenging. This study aimed to analyze the discriminatory power of joint angles throughout a gait cycle. Six sets from 12 men were collected. For each man, a variability range VR (mean ± 1SD) of a specific joint angle at a specific time point (a gait cycle was 100 time points) was calculated. In turn, each individual was compared with the 11 others, and whenever 1 of these 11 had a value within this individual's VR, it counted as positive. By adding the positives throughout the gait cycle, we created simple bar graphs; tall bars indicated a small discriminatory power, short bars indicated a larger one. The highest discriminatory power was at time points 60–80 in the gait cycle. We show how our data can assess gait data from an actual case. <s> BIB033 </s> Class Energy Image Analysis for Video Sensor-Based Gait Recognition: A Review <s> Figure 2. (a) <s> In this paper, we propose a time-sliced averaged motion history image (TAMHI) alongside the histograms of oriented gradients (HOG) to generate gait signatures in a gait recognition problem. Building on the motion history image (MHI), TAMHI divides the gait cycle into several regular time windows to generate the same number of TAMHI composite images. HOG descriptors are then calculated on these composite images to obtain the gait signature. The time-slicing procedure to produce multi-composite images preserve more detailed transient information of gait cycles. Additionally, time-normalization also introduces gait length invariancy into the representation, hence, offering a better recognition rate to slight changes in walking speed. <s> BIB034
The general framework of a video sensor-based gait recognition system. The camera-based sensor captures gait information and sends the data to computers. The system includes four modules, which are the preprocessing module (i.e., subject detection and silhouette extraction from the original video), feature representation module, feature selection module and classification module. Note that the model-based gait recognition may not need the preprocessing module; (b) The silhouette images are the results of period detection corresponding to the preprocessing module in Figure 2a . There are various approaches available for video sensor-based human gait representation, which can be roughly divided into two broad categories: the model-based and model-free approaches BIB028 BIB007 . The model-based approach aims to explicitly model human body or motion according to prior knowledge. Generally, each frame of a walking sequence is fitted to the model of the human body and the parameters, such as motion trajectories BIB003 , joint angles BIB028 , hip position BIB002 , limb lengths BIB017 , body part ellipses BIB008 and physical distances BIB004 , gathered from moving bodies are measured on the model as gait features for recognition. One such approach represents a gait silhouette as seven regions of ellipses. Then, the ellipses' parameters are computed as gait features for recognition BIB008 . Another method utilizes the pendulum model to guide the motion extraction process BIB010 . Recently, Zeng et al. BIB028 proposed employing the lower limb joint angles to characterize gait features. Some researchers studied the motion trajectories or joints under multi-cameras conditions BIB005 BIB001 BIB022 . Model-based approaches are insensitive to background cluttering and noise. These methods are easy to be understood, and generally view and scale invariant. However, model-based approaches suffer from several drawbacks. First, it is difficult to accurately locate the joints' position due to the highly flexible structure of non-rigid human body and to self-occlusion BIB033  . Second, model-based approaches are sensitive to the quality of gait sequences. Third, and the greatest disadvantage of the model-based approaches is their large computation cost and relatively high time costs due to parameters calculations, complex feature extraction and matching in these methods. Thus, it is somewhat difficult for the model-based approach to be applied in real environment. Therefore, current literature focuses more on model-free approaches. Model-free approaches BIB011 BIB018 BIB024 BIB006 BIB029 BIB030 BIB025 BIB034  BIB012 BIB019 aim to utilize the motion information directly to identify individuals and does not need the prior knowledge of the gait model. They usually extract statistical features from the whole silhouette without assuming the underlying structure BIB031 . Model-free approaches can be divided into two groups, namely temporal comparison and summary of spatio-temporal information. The temporal comparison approaches directly compare and match spatial features temporally, on a frame by frame basis. In this case, typical methods include correlation between frames and Hidden Markov Models BIB009 . Sarkar et al. BIB013 propose a baseline gait recognition algorithm for computing the correlation between frames by using the ratio of the intersection to the union between the probe and gallery frames. Kale et al. BIB009 employ Hidden Markov Models to distinguish between temporal data. The approach computes the likelihood generated from the Hidden Markov Model corresponding to a particular person to perform identification. In order to reduce the challenges of comparing images on a frame by frame basis, it is more efficient to use summarized motion features spatiotemporally. A number of spatio-temporal motion summary approaches usually superpose sequences of binary silhouettes depending on certain rules. Then, the original video sensor-based silhouette sequences are transformed into a single image template for recognition. In this paper, the template is called Class Energy Image. The method generating the template is called the Class Energy Image approach. The Class Energy Image approaches characterize the gait under multiple conditions without considering the body structure and computing accurate parameters of body parts in gait recognition. The advantages of the video sensor-based Class Energy Image approach can be summarized as follows: (a) It is well suitable for real time systems because it is easy to extract the feature and computational complexity is low BIB020 BIB032 ; (b) It is insensitive to the quality of silhouettes comparing to model-based approaches; (c) It holds several key features of human gait including motion frequency, temporal and spatial changes of human body, and global body shape statistic BIB016 ; (d) It is robust to silhouette errors and image noise BIB014 . Due to the above merits, the Class Energy Image approaches have been widely used in the state-of-art gait recognition systems BIB016 BIB014 BIB026 BIB023 BIB015 BIB021 BIB027  . Consequently, we focus on providing a comprehensive review of the past and present Class Energy Image approaches for video sensor-based gait recognition in this paper. The organization of this paper is as follows: Section 2 provides a panoramic summary and analysis of related work in the general area of Class Energy Images. In Section 3, we evaluated and discussed the performances of various Class Energy Image approaches by experiments. The results demonstrated that some Class Energy Image approaches could attain higher recognition accuracy with good robustness and efficiency. Section 4 outlines important observations and provides promising future directions for the field.