IEEJ TRANSACTIONS ON ELECTRICAL AND ELECTRONIC ENGINEERINGIEEJ Trans 2015; 10(s1): S69–S76Published online in Wiley Online Library (wileyonlinelibrary.com). DOI:10.1002/tee.22166
Detection of Gait Impairment in the Elderly Using Patch-GEI
Chengju Zhoua, Non-memberIkuhisa Mitsugami, Non-memberYasushi Yagi, Non-member
We propose a novel method for estimating physical impairment of elderly people using gait. To achieve this, we ﬁrst investigatewhich gait feature is effective for this purpose among gait energy image (GEI), duration time, and phase ﬂuctuation as dynamicfeatures. GEI is a popular appearance-based feature showing high performance in human authentication. By comparison, weﬁnd that it is the most reasonable feature. In real situations, however, GEI is easily affected by clothes variations or carryingconditions, so that the use of whole body results in decreasing performance. Considering this problem, we thus propose to useonly the GEI features of the most discriminative body patches. From the experiments that evaluate the contribution of varioussizes of body patches, we ﬁnd that head and chest regions perform better than the whole body with the classiﬁcation accuracyimproved from 80.93% to 83.17% for the visual impairment discrimination case. As for the leg impairment detection case, theleg region performs better than the whole body by an accuracy increased from 69.30 to 75.05%. These results conﬁrm theeffectiveness of patch-GEI for impairment detection. © 2015 Institute of Electrical Engineers of Japan. Published by John Wiley& Sons, Inc.
Keywords: patch gait feature, leg impairment walking, visual impaired walking
Received 22 October 2014; Revised 28 April 2015
1.Introduction
Walking is one of the indicators of human health. Normalwalking requires the coordination of the cerebellar, sensory, visual,vestibular, muscular, basal ganglia, and auditory systems, soany abnormality in these systems can result in gait disorder[1]. With aging, those functions degenerate, which results inabnormal walking style. Indeed, adult people who have someimpairments, such as poor binocular visual acuity, weak inter-jointcoordination ability, and loss of hearing, show different walkingstyles in postural and dynamic aspects compared to people withno impairments [2–4]. We, humans, can usually distinguish thedifferences quite easily just by observing their ways of walking.If we design a system that can automatically detect such impairedpeople from their walking styles, it would be very useful in elderlycare [5] and in many other gait-related applications such as thediagnosis of diseases like Parkinson’s disease [6–9].Existing studies related to gait analysis are separated into twocategories: model-based and appearance-based methods. Model-based methods usually apply a motion capture system [2,3,10–12]or wearable sensors [6,13,14] to capture accurate pose parameters.For example, Hallemans et al. [10] used a 3-D motion capturesystem to measure head orientation, stride length, and trunkﬂexion to verify whether poor vision affects dynamic stability ofwalking. Other model-based studies [15–20] use movies capturedby cameras and estimate the pose parameters by ﬁtting andtracking body components. Taha et al. [17] ﬁtted a skeleton modelto binary human silhouettes to calculate their posture lean andstride cycles for detecting Parkinsonian gait. From the promisingresults of these two model-based approaches, we conﬁrm thatthe walking posture, the temporal cue, and the stability property
a Correspondence to: Chengju Zhou.E-mail: zhou@am.sanken.osaka-u.ac.jp
The Institute of Scientiﬁc and Industrial Research, Osaka University 8-1Mihogaoka, Ibaraki, Osaka 567–0047, Japan
are important for disorder gait detection. In fact, model-basedapproaches can measure precise trajectories of moving joints andbody components, but they require laboratory environment settingsand cooperation of the subject; the subjects have to wear specialclothes or devices. Further, they usually suffer from low qualityof the pose estimation and high computation cost. These methodsare thus not applicable to the scheme we would like to achieve.On the other hand, appearance-based methods use capturedmovies directly, so subjects do not need to wear any specialdevices. Chen et al. [7,8] used binary silhouettes extracted fromcolor images to distinguish people with Parkinson’s Disease fromnormal ones. Their study used the binary images directly forclassiﬁcation, and hence the results heavily depend on the qualityof each silhouette. Other works [21–24] have tried to extractsome features from the gait observation and use them for genderrecognition and human authentication. Among those features, gaitenergy image (GEI) [23] was often used since it is well knownto show high authentication performance for individuals fromtheir gait. Another advantage of GEI is that it does not requirehigh-quality silhouette extraction; it is quite robust against noiseinevitably included in extracted silhouette images. Consideringthat GEI is deﬁned as an average of sequential silhouettes in awalking period, it can encode the shape of people very well, sothat it must be effective for personal authentication. It does not,however, preserve temporal information such as the duration awalking period. Therefore, besides the GEI, we still investigatethe effectiveness of the duration, which is relative to the walkingspeed. Considering that both GEI and duration time describeproperties within a period, we further study phase ﬂuctuation [25]between the neighboring walking periods. This phase ﬂuctuationis used for estimating the walking stability, which is considered tobe an important cue for assessing the ability to walk. We extractthose gait features and use them to distinguish two common formsof impaired walking (leg and visual) from normal walking usinglinear discriminant analysis (LDA). From the evaluation results,we ﬁnd that GEI is the most reasonable gait feature to describe
© 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.
C. ZHOU, I. MITSUGAMI, AND Y. YAGI
the differences between the two walking categories. As Cho et al.[26] have pointed out that for healthy adults gender would affecttheir walking styles in some aspects, we also discuss the inﬂuenceof gender on impairment detection among those gait features.GEI mainly encodes the shape information of a human body.Even if we include a size normalization step before extracting GEI,it is still more or less affected by individual shape differences.To eliminate the inﬂuence of body parts other than the relevantregion, we propose using only the most effective GEI patchfor classiﬁcation. Two works give detailed analysis of bodycomponents in gender recognition [27,28]. Li et al. [27] segmentthe averaged gait image into seven components: the head, thearm, trunk, the thigh, the front leg, the back leg, and the feet.The different contributions of human components for humanauthentication and gender recognition are analyzed. In addition,Yu et al. [28] point out that hair style and chest are two importantbody components for gender classiﬁcation. Those componentstherefore should not be separated. In their paper, they have adifferent segmentation approach for human components: head andhair style, chest, back, waist, buttocks, and legs. This segmentationimproves their performance. Experimental results vary with thechange of the criteria for the segmentation of human bodyparts. Even though disorder gait is quite different from normalwalking in gender classiﬁcation, walking posture will changeaccording to different illnesses. It is hard to decide what kind ofbody component segmentation will be suitable for our purpose.Therefore, we apply several grid patch sizes of the humansilhouettes for evaluation. We present the effectiveness of ourmethod by comparing the performance of patch-GEI with full-bodyGEI in impairment detection. Considering realistic applicationsof distinguishing multi-class impairments simultaneously, we alsoprovide a solution strategy.The rest of this paper is organized as follows. In Section2, we ﬁrst describe gait features extraction, and then introduceexperimental data collection. Finally, we show the performanceof three different gait features in detecting impairment andidentifying the inﬂuence of gender. Section 3 is concerned aboutthe contribution of body patches and an attempt to provide asolution to a real-life application. Conclusions are drawn in Section4.
2.Gait Features and Their Performance
2.1.Gait features extractionIn this section, we describehow to extract the shape, temporal, and stability informationfeatures of walking subjects. From the original input color imagesequence, we ﬁrst apply background subtraction to extract binarysilhouettes. After position alignment and size normalization, thebinary silhouette sequences are called “gait silhouette volumes”(GSVs) [24].With GSV, assuming periodicity of walking, we estimate a walk-ing period length by calculating the normalized autocorrelation ofthe silhouette images in the temporal axis. The gait period Nperiodis determined as the number of frames that makes the normalizedautocorrelation maximum. As the sequences usually contain morethan a period, for duration time and GEI, we extract only the part{Si}(i = 1, 2, · · · , Nperiod) that corresponds to a period. Note that inour experimental setting, we pick up the period around the centerof the image. Nperiod denotes the number of frames in a period,which we call “duration time” in this paper. Once we obtain GSV,the GEI feature is calculated as follows:
G(x, y) =1
Nperiod
Nperiod�
i=1Si(x, y)(1)
Original color sequence GSVGEI
Fig. 1. Gait energy image (GEI)
where i is the frame number in a period of walking silhouettes,and x, y is the coordinate in the image. An example of the GEI isshown in Fig. 1. In the GEI image, the pixel intensity correspondsto the frequency about the human body appearance. A brighterintensity indicates a human body part that has less motion, suchas the head and torso. On the other hand, gray parts correspond toregions with a lot of motion, like legs. Black means no body partsappear at the corresponding place.We now brieﬂy introduce the main processing steps for calcu-lating phase ﬂuctuation using the method that was proposed byMakihara et al. [25]. Based on the prior known relation betweentime and phase, and given a quasi- periodic sequence (GSV inthis study), we can estimate a phase evolution sequence in anoptimization framework with the help of instantaneous period esti-mation using short-term period detection (STPD) and self-dynamictime warping (DTW). This is the problem of phase registration.However, because of the existence of the combination ambigu-ity included in the quasi-periodic signal, the combination of thephase evolution function and the normalized periodic signal isnot ﬁxed. Thus, the phase evolution function linearization processis still needed. The procedures are mainly separated into threesteps [29]: (i) period segmentation, (ii) reconstruction of the timewarping function (TWF), and (iii) linearization of the TWF. Here,the TWF is about the relationship between linearly evolved phaseand estimated relative phase in a period. As Makihara et al. [26]pointed out that their proposed method reconstructs TWFs froma single quasi-periodic signal through a bias estimation process,the variance in the reconstructed TWFs can be used as a kind ofphase evolution instability measure. For more processing details,we refer the reader to [25–29].
2.2.Experimental data collectionIn our study, wepick up two categories of commonly seen impairments amongsenior people: leg impairment and visual impairment. Stiff knee isa typical leg problem in the elderly. The reason for that problem isthat joints become stiffer and less ﬂexible with aging. In addition,old people are prone to have some lesions on the eye lens orretina, which results in visual impairments with symptoms of blurand tunnel view. As you can image, however, it is hard to collectwalking data of people who really have these impairments. Oneof the reasons is that it is difﬁcult to ﬁnd enough numbers ofreal patients. Moreover, even if we can ﬁnd sufﬁcient number ofpatients, it is still difﬁcult to ensure their safety in experiments.Considering these limitations, we use an age simulation kit tohelp healthy people “act as” these two kinds of commonly seenimpaired people. As the simulation kit, we adopt a product of
S70IEEJ Trans 10(s1): S69–S76 (2015)
DETECTION OF GAIT IMPAIRMENT IN THE ELDERLY USING PATCH-GEI
(a)(b)(c)
Fig. 2. Three types of walking. (a) Leg-impaired walking. (b)Visually impaired walking. (c) Normal walking
Sanwa Manufacturing Co., Ltd [30]. Since it is a popular one thathas been used in many ﬁelds, it is reasonable for us to assumethat it simulates well real elderly walking. We then prepare threetypes of walking, as follows: 1. leg-impaired walking by fasteningleg supporters on both knees, which restricts knee bending, 2.visually impaired walking by wearing glasses that blur the sightand narrow the view ﬁeld, and 3. normal walking without anythingﬁxed. Examples of these three kinds of walking styles are shown inFigs 2(a)–(c), respectively. When collecting walking data, peoplewalk on a straight path with a camera capturing sideways, so allof the silhouettes in our study are lateral. In our experiments, theresolution of silhouette images is 120 × 80, so the dimension ofGEI is also 120 × 80. 1
The number of subjects for leg impaired walking, visuallyimpaired walking, and normal walking were 186 (71 female, 115male), 142 (73 female, 69 male), and 325 (148 female, 177 male),respectively. Note that every participant in the data collectionwas instructed to ﬁnish two categories of walking: one type ofimpaired walking, and normal walking. So number of subjects withnormal walking is much larger than that of impaired walking. Theage range of participants was 4–78. By wearing the simulationkits, they were all recognized as same-age group ”old adults”.Some situations, such as calculating phase ﬂuctuations, requiredthe walking sequence to contain more than one walking periods,hence the number of subjects decreased.
2.3.Classiﬁcation methodTo evaluate performance ofdiscrimination, we apply LDA to the extracted gait features.Since our interest is in how much each impaired walking ischanged compared to normal walking, we apply two-class LDAbetween normal walking and that with leg/visual impairment.Note that, because the subject number is not balanced for eachclass, we choose the same subject number for the two classes.In total, we make 30 times selection, and the results are theaverage of those 30 times and also their standard deviation. Inthe case of GEI, since the feature is high-dimensional vectors,we preliminarily apply principal component analysis (PCA) tocompress them into the dimension number that preserves around90% of the original energy. Each selection randomly choosesGEI from different subjects to compose the input features for
1 As is well known, nowadays we have to obtain and manage suchimage data including personal information with extreme discretion. Weconsulted a lawyer about procedure to obtain and manage the data andabout how to get agreement from subjects, and we carefully followed theprocedure. Moreover, in the case of a child subject, who is not regardedto be responsible for the agreement, we ask his/her parent to give theagreement on his/her behalf. Owing to this procedure, images in this datasetcan be pasted on this paper, and can be analyzed for our research purpose.
PCA. As a result, the number of components and distributionof principal component scores vary among between selections.So instead of a speciﬁc number of components, a range isgiven. Take the whole body GEI as input feature, for example.Considering the situation of compressing features of subjects withnormal and visual impairments, the range of component numberscorresponding to 90% of the original energy is 50–52. For case ofnormal between-leg impairment, the range is 54–56. One examplefor the distribution of each principal component score is shown inFig. 3.
2.4.Discrimination ability of each feature
2.4.1. GEIWe evaluate the performance of GEI in distin-guishing impaired walking from normal walking. The table in leftside of Fig. 4 shows the performance of normal walking and visu-ally impaired walking. It shows the accuracy is about 81%. Thetwo images at the right side are the re-projections of the most dis-criminative LDA direction. Comparing these two images, we canﬁnd that a person with visual impairment tends to bend his/herhead more to the front than a normal person. It sounds reasonablebecause people with lower visual ability need to be more carefulabout the road. As for the performance of the leg-impaired walkingand normal walking, theresults are shown in Fig. 5. As the ﬁgureshows, the classiﬁcation accuracy is about 69%. The re-projectionimages on the right side show that leg-impaired walking has aslightly smaller leg angle and lower head bending than normalwalking. The performance of GEI shows that impairment affectsthe walking posture.
2.4.2. Duration timeFig. 6 and Fig. 7 show the distri-bution of the duration time of normal walking and leg/visuallyimpaired walking and also their classiﬁcation accuracy. The distri-butions of the normal and impaired walking are shown by red andblue curves, respectively. In Fig. 6, the peaks of the two distribu-tion are remarkably aligned. It means that visual impairment doesnot affect the duration time. In Fig. 7, on the other hand, we ﬁndthat leg impairment affects the duration time; the duration time ofthe leg impairment is obviously longer than that of normal walk-ing. However, the large overlapped region of two distributions ineach graph means that the duration time is not very effective foraccurate impairment estimation and the classiﬁcation results alsoprove that point.
2.4.3. Phase ﬂuctuationPhase ﬂuctuation of a neighbor-ing phase estimated sequence is used to estimate the stability of awalking style. It is measured by the variance of the reconstructionof TWFs. The smaller the variance, the more stable the phase evo-lution between neighboring periods. That also means the walkingstyle of the subject is more stable. We note here that to calculatethe phase ﬂuctuation, a walking sequence should contain morethan two periods. The numbers of subjects who fulﬁll this condi-tion are 198, 92, and 66 for normal walking, leg-impaired walking,and visual-impaired walking, respectively.Fig. 8 and Fig. 9 show the distribution of variance of recon-struction DTW of normal walking and leg/visual-impaired walkingand also their classiﬁcation accuracy. The distributions of the nor-mal and impaired walking are shown by the red and blue curves,respectively. In both Fig. 8 and Fig. 9, the peaks of the two dis-tribution are aligned. It means that the stability of the walkingstyle is different between individual people, but not affected bythe physical impairment. The classiﬁcation accuracy also provesthis point.
2.5.Inﬂuence of gender on gait featuresThe paperof Cho et al. [26] proves the assumption that healthy adults walk
S71IEEJ Trans 10(s1): S69–S76 (2015)
C. ZHOU, I. MITSUGAMI, AND Y. YAGI
Component scores
102030PCA dimension number40506000
Component scores
102030PCA dimension number405060
(a)(b)
Fig. 3. Distribution of the principal component scores for GEI of subjects from (a) normal and visual impairment, and (b) normal and legimpairment
Normal vs. Visual impairment
Normal81.17 ± 2.84%
Visual impairment80.68 ± 1.99%
Average accuracy80.93 ± 1.90%Visualimpairment
Normal
Fig. 4. Accuracyandre-projectionofnormalandvisuallyimpaired walking
LegimpairmentNormal
Normal vs. Leg impairment
Normal70.27 ± 2.00%
Leg impairment68.33 ± 1.99%
Average accuracy69.30 ± 1.47%
Fig. 5. Accuracy and re-projection of normal and leg-impairedwalking
differently according to their gender, based on some aspects. Afternormalization, which was used to avoid the body size effect, theauthors reach the following conclusions: (i) no gender differenceswere found in walking speed and also in the durations of thestance phase and the double support period; (ii) some differenceswere discovered in postural aspects. Females walked with theirpelvis tilted more anteriorly, more up and down oblique motion,hip joints more ﬂexed rotated, knee joint in more valgus angles,and narrower step widths. Considering the property of the threegait features in this paper (namely GEI, duration time, and phaseﬂuctuation), GEI, which describes the body shape of the walkingsubjects, is probably affected by gender. We verify the assumptionby comparing among the detection results using GEI from subjectsof mixed gender, same gender, and cross-gender, which are shownin Fig. 4 and Fig. 5, Table I and Table II. Note that mixed gendermeans subjects used without considering gender, mixed togetherfor training and testing, while cross-gender means using male(female) subjects for training but female (male) subjects for testing.From these ﬁgures, we ﬁnd that the results change whenapplying different gender conditions, which means that bothcategories of impairment detections are affected by the gender. Legimpairment classiﬁcation is further affected. This may be due tothe fact that more obvious differences exist in gait-related anatomyin leg rather than neck between genders. Since the results decreasein most of the classiﬁcation in same gender cases and in all
cross-gender situations, and also gender information always lacksin the real scenarios, using the strategy of classiﬁcation withoutconsidering gender factor is more robust.
2.6.DiscussionAmong the features we picked up, GEIgives ﬁne performance, but duration time and phase ﬂuctuation arenot very effective. In the case of the duration time, although thestatistical distribution is changed by impairment, it is not adequatefor impairment estimation since the overlapped regions betweenthe distributions are quite large. We also found that the phaseinformation is not effective. Walking stability is not affected bythe physical impairment, but rather by the individual differences.So GEI is chosen to present the discriminative information ofdifferent walking styles. Even though gait is affected by gender,GEI evaluation results show that classiﬁcation without consideringgender is more feasible in real scenarios.
3. Performance and Real Application of Patch-GEI
3.1.Performance of patch-GEIAs discussed in Section2, it appears that GEI is the best feature to detect impaired walking,because posture difference between different walking styles can bedescribed by this feature. In other words, walking shape containsthe discriminative ability for impairment detection. On the otherhand, however, the shape is usually affected by clothing, carrying,and so on, in real situations. Considering this problem, we thuspropose to use only effective parts to decrease negative inﬂuenceof the other parts of the body.To decide which body patches contain the most discriminativeinformation, we ﬁrst need to determine the criterion for segmentingthe whole GEI into patches. As mentioned previously, two worksgive detail analysis of contribution of body components forgender recognition [27,28]. From their experiments, we foundthat results become different with the change of criterion forthe segmentation of human component. Since our scenario is, ofcourse, different from those studies, we have to ﬁnd a new rule (themost discriminative patches) for our scenario. To do this, we applyvarious grid patch sizes of human body, and evaluate performance.In this study, we evaluate the performance of ﬁve levels of patchsizes, they are 5 × 5, 10 × 10, 20 × 20, 30 × 30, and 40 × 40.From the whole-body GEI, we extract the region corresponding tothe patch size with row scanning order and with the skipping stepof 5 pixels. For each patch region, we also use the same processingﬂow with whole GEI, ﬁrst PCA for compressing into the dimensionwhich preserves about 90% energy of the original one, and thenLDA for classiﬁcation. Their performances are shown in Fig. 10and Fig. 11.
S72IEEJ Trans 10(s1): S69–S76 (2015)
DETECTION OF GAIT IMPAIRMENT IN THE ELDERLY USING PATCH-GEI
Normal walking 
Visual impairmentwalking
1214161820Frame number
Frequency
2224262830
Normal vs. Visual impairment
Normal50.77 ± 10.11%
Visual impairment47.68 ± 9.43%
Fig. 6. DT distribution and accuracy of normal and visually impaired walking
Normal walking 
Leg impairmentwalking
1214161820Frame number
Frequency
2224262830
Normal vs. Leg impairment
Normal55.87 ± 2.01%
Leg impairment73.97 ± 4.90%
Fig. 7. DT distribution and accuracy of normal and leg-impaired walking
Frequency
Variance
Normal walking 
Visual impairmentwalking
Normal vs. Visual impairment
Normal37.02 ± 20.70%
Visual impairment65.20 ± 21.18%
0.0050.010.0150.020.025
Fig. 8. Phase ﬂuctuation distribution and accuracy of normal and visually impaired walking
In Fig. 10 and Fig. 11, “classiﬁcation accuracy” is the perfor-mance of patch shown in the third row, which is the best per-formance of the same patch size for a whole GEI. And the grayhuman images in the these rows are the averages of GEI fromall the leg-impaired subjects and all the visual-impaired subjectsfor Fig. 10 and Fig. 11, respectively. The ﬁgures on the fourthrow of each ﬁgure are the performance of all the patches of thesame size. The points in the image are located on the center ofthe patches. And color of the points is relative to the classiﬁcationaccuracy using that corresponding patch GEI; color closer to blue
means lower accuracy and color closer to red means higher accu-racy. Also note that the rightmost column is the performance ofthe whole GEI.From the results shown in Fig. 10, we found that patch withsize of 30 × 30 positioning at the head and breast gave thebest performance with the classiﬁcation accuracy of 83.17%.Comparing with the average accuracy of whole GEI of 80.93%,using only patch around the head and breast region can improve theaccuracy by 2.24%. This result proves our assumption that otherparts of the human body will inﬂuence the discriminative ability
S73IEEJ Trans 10(s1): S69–S76 (2015)
C. ZHOU, I. MITSUGAMI, AND Y. YAGI
Frequency
Variance
Normal walking 
Leg impairmentwalking
Normal vs. Leg impairment
Normal30.33 ± 8.80%
Leg impairment68.33 ± 9.04%
0.0050.010.0150.020.025
Fig. 9. Phase ﬂuctuation distribution and accuracy of normal and leg-impaired walking
Table I. Accuracy of same gender classiﬁcation. (a1),(b1)Female subjects training and testing; (a2),(b2) male subjectstraining and testing
normal vs. visual impairment  (female training, female testing)
normal79.41% ± 3.27%
visual impairment 76.80% ± 3.41%
average accuracy78.11% ± 2.81% 
normal vs. leg impairment  (female training, female testing)
normal65.40 %± 4.66 %
leg impairment 62.30 %± 3.42 %
average accuracy63.85 %± 3.42 %
normal vs. visual impairment  (male training, male testing)
normal77.29 %± 2.67 %
visual impairment 78.26 %± 2.50 %
average accuracy77.78 %± 1.93 %
normal vs. leg impairment  (male training, male testing)
normal80.52 %± 2.36 %
leg impairment 84.61 %± 2.25%
average accuracy82.57 %± 1.8%
Table II. Accuracy of cross-gender classiﬁcation. (a1),(b1) Malesubjects training and female subjects testing; (a2),(b2) femalesubjects training and male subjects testing
normal vs. visual impairment  (male training, female testing)
normal63.09%± 11.26%
visual impairment 72.42%± 7.60%
average accuracy67.75%± 4.11%
normal vs. leg impairment  (male training, female testing)
normal68.17%± 7.21%
leg impairment 42.77%± 5.38%
average accuracy55.47%± 1.70%
normal vs. visual impairment  (female training, male testing)
normal77.74%± 6.39%
visual impairment 78.16%± 8.77%
average accuracy77.95%± 3.13%
normal vs. leg impairment   (female training, male testing)
normal48.40%± 9.63%
leg impairment 70.64%± 8.39%
average accuracy59.52%± 2.79%
when using GEI is reasonable. Note that, though the other sizesof patches perform a little worse, all color closer to red appears atthe upper region and all the highest accuracy appears at the headregions. This once again conﬁrms the fact that people with visualimpairments need to bend their heads to pay more attention to thecondition of roads. And still, when we check the lower part of thecolor points image, we ﬁnd that they are near to the blue color,which means impairment on eyes have little inﬂuence on the lowerbody.Fig. 10 shows the performance of patch-GEI in detecting leg-impaired walking. The patch size of 40 × 40 shows the most
discriminative ability at the accuracy of 75.05%, which is animprovement of 5.75% compared with the performance of wholeGEI. All the best performances at the leg region also conﬁrmthe fact that people with leg impairments walk with smaller stridelengths. And about the color point distribution, the red closer colorpoints also appear at the head region, but they are not so obvious.This means that the leg impairment makes the subject walk witha smaller leg angle, and at the same time, makes the balance ofthe moving body, so the subjects have a little front lean. But this“front lean” is less effective than the leg change, so using only theleg region is enough. From the analysis mentioned above, patch-GEI can decrease the inﬂuence of the other parts of the humanbody. It also can solve the problem of sharp shape change due tocarrying baggage. Using only the effective patch, like only head orleg regions can avoid the shape change around baggage regions.
3.2.A solution to a real application of patch-GEIBesides exploring the difference between impaired walking andnormal walking, we go further to solve another issue that makesour research more practical, namely distinguishing multi-typeimpairments simultaneously. Considering that visual impairmentand leg impairment are independent of each other, a subject mightsuffer from both visual and leg impairments at the same time. Tolabel the subject as either visually impaired or leg impaired only isnot very reasonable; therefore such kind of compound visual andleg impairment is considered as a new impairment type.The direct multi-class classiﬁcation would get confused indistinguishing pure impairment and compound impairment. Wethus provide the classiﬁcation strategy of using a binary classiﬁerto detect one type of impairment, one type of impairment vs.the other types of impairments (including normal). In detail,we employ the strategy of visual impairment vs. non-visualimpairment (including normal and leg impairment) to diagnosewhether the subject suffers from visual impairment, and use themethod of leg impairment vs. non-leg impairment (includingnormal and visual impairment) to detect leg impairment. In ourexperiment, the accuracy of visual impairment detection is 77.82%and that of leg impairment detection is 72.65%.
4. Conclusion
In this paper, we developed a scheme for impairment detectionin elderly people through camera-captured walking sequences.We ﬁrst listed three gait features: GEI, duration time, phaseﬂuctuation, which cover the posture, temporal, and dynamic aspectof walking. Next, we used a two-class LDA for classiﬁcation.
S74IEEJ Trans 10(s1): S69–S76 (2015)
DETECTION OF GAIT IMPAIRMENT IN THE ELDERLY USING PATCH-GEI
Patch size5*510*1020*2030*3040*40120*80
Averageaccuracy
Mosteffectivepatch
Patchaccuracy ofthe wholeimage
76.26 ± 1.40%75.79 ± 1.18%81.20 ± 1.61%83.17 ± 1.36%82.31 ± 1.75%80.93 ± 1.90%
Fig. 10. Classiﬁcation accuracy of normal walking and visually impaired walking
Patch size5*510*1020*2030*3040*40120*80
Averageaccuracy
Mosteffectivepatch
Patchaccuracy ofthe wholeimage
62.24 ± 1.59%66.01 ± 1.49%69.87 ± 2.05%72.68 ± 1.38%75.05 ± 1.60%69.30 ± 1.47%
Fig. 11. Classiﬁcation accuracy of normal walking and leg-impaired walking
From the experimental results, we conﬁrmed the discriminativeability of GEI over the three features. We also discussed theinﬂuence of gender in GEI. Further, considering that GEI ismainly about shape information, it must be affected by the otherparts of the human body except the effective part for detectingimpairments. We thus proposed using only the effective patchGEI for classiﬁcation. We could improve the performance by 2.24and 5.75%, respectively, for detecting visual and leg impairmentcompared with whole GEI. This proves the effectiveness of patch-GEI. Based on this fact, we considered the real application ofdistinguishing multiple impairments simultaneously using binaryclassiﬁcation. The accuracy of detecting visual impairment was77.82% and that of leg impairment detection was 72.65%.Future work consists of constructing an automatic impairmentestimation system based on the patch-GEI method. Moreover,we also have to consider how similar it is to walk wearing thesimulation kits when compared to the walking of actually impairedpeople.
References
(1) Ali A, Sundaraj K, Ahmad B, Ahamed N, Islam A. Gait dis-order rehabilitation using vision and non-vision based sensors: a
systematic review. Bosnian Journal of Basic Medical Sciences 2012;12(3):193–202.(2) Shin S-S, An D-H, Yoo W-G. The effect of navigating obstacleswhen walking with deﬁcient (inappropriate, deprived, limited) footsensory input on spatiotemporal gait parameters of elderly individualswith different visual acuities. European Geriatric Medicine 2015;6(1):1–6.(3) Chiu S, Chou L. Variability in inter-joint coordination during walkingof elderly adults and its association with clinical balance measures.Clinical Biomechanics 2013; 28:454–458.(4) Lia L, Simonsickc EM, Ferruccic L, Lin FR. Hearing loss and gaitspeed among older adults in the United States. Gait & Posture 2013;38(1):25–29.(5) Uddin MZ, Kim T-S, Kim JT. Video-based human gait recognitionusing depth imaging and hidden Markov model: a smart systemfor smart home. Proceedings of 3rd International Symposium onSustainable Healthy Buildings, Seoul, Korea,2010.(6) Salarian A, Russmann H, Vingerhoets FJG, Dehollain C, Blanc Y,Burkhard PR, Aminian K. Gait assessment in Parkinson’s disease:toward an ambulatory system for long-term monitoring. IEEE Trans-actions on Biomedical Engineering 2004; 51(8):1434–1443.(7) Cho C-W, Chao W-H, Lin S-H, Chen Y-Y. A vision-based analysissystem for gait recognition in patients with Parkinson’s disease.Expert Systems with Applications 2009; 36:7033–7039.(8) Chen Y-Y, Cho C-W, Lin S-H, Lai H-Y, Lo Y-C, Chen S-Y,Chang Y-J, Huang W-T, Chen C-H, Jaw F-S, Tsang S, Tsai S-T.A vision-based regression model to evaluate Parkinsonian gait from
S75IEEJ Trans 10(s1): S69–S76 (2015)
C. ZHOU, I. MITSUGAMI, AND Y. YAGI
monocular image sequences. Expert Systems with Applications 2012;39:520–526.(9) Khan T, Grenholm P, Nyholm D. Computer Vision Methods forParkinsonian Gait Analysis: A Review on patent. Recent Patents onBiomedical Engineering 2013; 6:97–108.(10) Hallemansa A, Ortibusc E, Meired F, Aertsa P. Low vision affectsdynamic stability of gait. Gait & Posture 2010; 32(4):547–551.(11) Fang W.Stone E. Gait analysis and validation using voxel data,Proceedings of 31st Engineering in Medicine & Biology Society,2009(12) Mostayed A. Abnormal gait detection using discrete fourier transfor-mation. Proceedings of International Conference on Multimedia andUbiquitous Engineering, 2008.(13) Baker R. Gait analysis methods in rehabilitation. Journal of Neuro-Engineering and Rehabilitation 2006; 3(4), doi:10.1186/1743-0003-3-4.(14) Tao W, Liu T, Zheng R, Feng H. Gait analysis using wearable sensors.Sensors 2012; 12:2255–2283.(15) Okusa K, Kamakura T. Normal/Abnormal Gait Analysis based onthe Statistical Registration and Modeling of the Frontal View GaitData. Proceedings of World Congress on Engineering and ComputerScience, 2012.(16) Yoo J-H, Hwang D, Moon K-Y, Nixon MS. Automated humanrecognition by gait using neural network. First Workshops on ImageProcessing Theory, Tools and Applications, 2008; 1–6.(17) Khan T, Westin J, Dougherty M. Motion cue analysis for Parkinsoniangait recognition. The Open Biomedical Engineering Journal 2013; 7:1–8.(18) Lee H, Guan L, Lee I. Video analysis of human gait and postureto determine neurological disorders. EURASIP Journal on Image andVideo Processing 2008; 2008:1–12.(19) Lee L, Grimson WEL. Gait analysis for recognition and classiﬁcation.Fifth IEEE International Conference on Automatic Face and GestureRecognition, 2002; 148–155.(20) Wang L, Tan T, Ning H, Hu W. Silhouette analysis-based gaitrecognition for human identiﬁcation, IEEE Transactions on PatternAnalysis and Machine Intelligence 2003; 25:1505–1518.(21) Hu M, Wang Y. A new approach for gender classiﬁcation based ongait analysis. Fifth International Conference on Image and Graphics,2009; 869–874.(22) Yang X, Zhou Y, Zhang T, Shu G, Yang J. Gait recognition basedon dynamic region analysis. Signal Processing 88(9):2350–2356.(23) Han J, Bhanu B. Individual recognition using gait energy image. IEEETransactions on Pattern Analysis and Machine Intelligence 2006;28(2):316–322.(24) Makihara Y, Sagawa R, Mukaigawa Y, Echigo T, Yagi Y. Gait recog-nition using a view transformation model in the frequency domain.9th European Conference on Computer Vision 2006; 3:151–163.(25) Makihara Y, Aqmar MR, Ngo TT, Nagahara H, Sagawa R,Mukaigawa Y, Yagi Y. Phase estimation of a single quasi-periodic signal. IEEE Transactions on Signal Processing 2014;62(8):2066–2079.(26) Cho SH, Park JM, Kwon OY. Gender differences in three dimen-sional gait analysis data from 98 healthy Korean adults. ClinicalBiomechanics 2004; 19(2):145–152.(27) Li X, Maybank SJ, Yan S, Tao D, Xu D. Gait components and theirapplication to gender recognition. IEEE Transactions on Systems,Man, and Cybernetics, Part C: Applications and Reviews 2008;38(2):145–155.
(28) Yu S, Tan T, Huang K, Jia K, Wu X, Study A. on Gait-basedgender classiﬁcation. IEEE Transactions on Image Processing 2009;18(8):1905–1910.(29) Ngo TT, Makihara Y, Nagahara H, Sagawa R, Mukaigawa Y, YagiY. Phase registration in a gallery improving gait authentication,Proceedings of the International Joint Conference on Biometrics(IJCB), 2011.(30) http://www.sanwa303.co.jp/products/medicare/teaching-material/104-990.
Chengju Zhou (Non-member) received the B.S. degree in Elec-tronic and Information Engineering and the M.S.degree in Signal and Information processingfrom Dalian University of technology, China,in 2008 and 2010, respectively. Since 2012, shehas been pursuing the Ph.D. degree in ComputerScience at Osaka University.
Ikuhisa Mitsugami (Non-member) received the B.S. degree inEngineering from Kyoto University in 2001, andM.S. and Ph.D. degrees in Engineering fromNara Institute of Science and Technology in2003 and 2007, respectively. He is currently anAssistant Professor with the Institute of Scien-tiﬁc and Industrial Research, Osaka University.His research interests include geometry in com-puter vision, detection and tracking, and gait analysis. Dr Mitsug-ami is a member of the IEEE, IEICE, IPSJ, RSJ, and VRSJ.
Yasushi Yagi (Non-member) received the Ph.D. degree fromOsaka University in 1991. In 1985, he joinedthe Product Development Laboratory, MitsubishiElectricCorporation,whereheworkedonrobotics and inspections. He became a ResearchAssociate in 1990, a Lecturer in 1993, an Asso-ciate Professor in 1996, and a Professor in 2003at Osaka University. He is currently the Directorof the Institute of Scientiﬁc and Industrial Research, Osaka univer-sity. He has served as Chair in various international conferences,including FG1998 (Financial Chair), OMINVIS2003 (Organiz-ing chair), ROBIO2006 (Program co-chair), ACCV2007 (Programchair), PSVIT2009 (Financial chair), ICRA2009 (Technical VisitChair), ACCV2009 (General chair), ACPR2011 (Program co-chair) and ACPR2013 (General chair). He has also served as theEditor of IEEE ICRA Conference Editorial Board (2007–2011).He is an Editorial Board member of IJCV and the Editor-in-Chief of IPSJ Transactions on Computer Vision & Applications.He was awarded ACM VRST2003 Honorable Mention Award,IEEE ROBIO2006 Finalist of T.J. Tan Best Paper in Robotics,IEEE ICRA2008 Finalist for Best Vision Paper, MIRU2008 NagaoAward, and PSIVT2010 Best Paper Award. His research interestsinclude computer vision, medical engineering, and robotics. Prof.Yagi is a fellow of IPSJ and a member of IEICE, RSJ, and IEEE.
S76IEEJ Trans 10(s1): S69–S76 (2015)