Hand- Gesture Detection… Hand-Gesture Detection Using Principal Component Analysis (PCA) and Adaptive Neuro-Fuzzy Inference System (ANFIS)

Sign language is a non-verbal language that Deaf persons exclusively count on to connect with their social environment.The problem that occurs in two-way communication using sign language is a misunderstanding when learning new terms that need to be taught to deaf and mute people. To minimize these misunderstandings, a system is needed that can assist in correcting hand gestures so that there is no misinterpretation in teaching new terms. Several optimality properties of PCA have been identified namely: variance of extracted features is maximized; the extracted features are uncorrelated; finds best linear approximation in the mean-square sense and maximizes information contained in the extracted feature. The classification uses the Adaptive Neuro-Fuzzy Inference System (ANFIS) method. From the results of experiments with different image size variables, the largest accuracy was obtained with an image size of 449x449 of 76.20%. While the lowest accuracy of 52.38% is obtained through scenarios with image sizes of 57x57 and 45x45. Therefore, differences in the use of image sizes have an influence on the accuracy of hand signal prediction. The smaller the size given, the smaller the accuracy obtained. This is indicated by the decreasing accuracy value when given a smaller size in the four scenarios that have been studied.


INTRODUCTION
Public attention to people who are deaf and speech impaired is still very minimal. It is difficult to understand the communication that occurs between normal people and people who are deaf and speech impaired. Deaf and speech impaired people use sign language to stay in touch with others. The use of sign language can be one solution for those who have speech or hearing deficiencies to remain able to interact with the general public [1].
According to E.Widodo, one of the translators and sign language teachers at INASLI (Indonesian Sign Language Interpreters), there are many potential misunderstandings that occur when learning sign language. In addition to adapting to the conditions of people with disabilities, language also develops and continues to emerge new terms not found in sign language dictionaries. In addition, there are also foreign terms such as a person's individual name, foreign language terms, and scientific terms that require more attention in spelling. Therefore, it is necessary to pay more attention to teachers in showing the hand signals shown to people who are deaf and mute.
In addition to minimizing misunderstandings for the deaf and mute in certain terms, assistance in correcting this needs to be used for teachers so that there is no misinterpretation in teaching new terms to deaf and mute people which results in long errors in the future [2].

THEORETICAL REVIEW a. Deaf
Deafness is a general term that denotes hearing difficulties, which include all hearing difficulties from mild to severe, classified into deaf and hard of hearing sections [3].

b. Sign language
According to Reynolds and Mann (1983:1435) sign language is a general term that refers to any gestural/visual language that uses specific forms and movements of the fingers, hands, and arms, as well as eye, face, head, and body movements. There are several standards of sign language used in communicating in several countries where the deaf have used them to communicate among themselves quickly, efficiently, and visually without the use of paper and pencil.
With the presence of sign language, the deaf and mute can communicate more broadly with predetermined standards. Normal people can also communicate more freely with the deaf and mute by using this sign language.

c. American Sign Language (ASL)
ASL is a communication model for humans who are deaf and speech impaired, and this communication model is included in learning for students with special needs such as the deaf whose method is known as the komtal method. The first character to use the term komtal (total communication) was Roy Holcomb. This character uses this term to describe the flexibility in communicating as applied in a school in California, United States. Then in 1968 D. Denton described this komtal method with the meaning of a complete spectrum of language ways, children's gestures, sign language, reading speech, finger spelling, and reading and writing, and there is sound (Sulastri, 2013:5).

d. Computer vision
Computer vision is a combination of two processes, namely image processing and pattern recognition [4]. Image processing (Image Processing) is a field related to image transformation. This process has the aim of getting better image quality [5]. Pattern Recognition is a branch of science that studies data patterns both numerical and symbolic so that one or more conclusions can be drawn. The input of the pattern recognition program is an image, while the output is a description of the object [6].
Principal component analysis is a technique of reducing multivariant data (a lot of data) to convert an initial or original data matrix into a set of linear combinations that are fewer but absorb most of the variance from the initial data [7].

e. General PCA steps
There are some general steps to implementing PCA so that you get the desired input. In Figure 2, it has been explained how the general steps of PCA so as to get the transformed data in the form of m x k [8].

f. Adaptive Neuro-Fuzzy Inference
System (ANFIS). Neuro-fuzzy is a combination of two systems, namely fuzzy logic systems and artificial neural networks. The neuro-fuzzy system is based on a fuzzy inference system that is trained using a learning algorithm derived from artificial neural network system. Thus, the neurofuzzy system has all the advantages possessed by the fuzzy inference system and the artificial neural network system [9]. From its ability to learn, the neuro-fuzzy system is often referred to as the Adaptive Neuo-Fuzzy Inference System (ANFIS). One form of the structure of ANFIS that applies the Takaagi-Sugeno-Kang fuzzy inference system model is depicted in Figure 3 below.

g. Simulation method
Simulation is the process required for the operationalization of the model, or the handling of the model to imitate the behavior of the real system. This method includes various activities such as the use of flow diagrams and computer logic, as well as writing computer code and applying that code to a computer to use input and produce the desired output. In practice, modeling and simulation are very closely related processes, and some authors have defined simulations that include modeling [10].

Method of collecting data
The interview was conducted with resource persons with Mr. Edik Widodo who is a Sign Language Lecturer and Translator from INASLI (Indonesian Sign Language Interpreter). Mr. Edik Widodo is one of the founders of INASLI which has been established since 1999.

System development method
The development stage of this research system design using Simulation Method as described in this chapter previously, the stages were problem formulation, conceptual model, modelling, experiment, verification and validation, and Output Analysis.

The proposed method
The hand signal identification using the Principal Component Analysis (PCA) feature extraction method which is implemented in the Adaptive Neuro-Fuzzy Inference System (ANFIS) classification method.

III. RESULT AND DISCUSSION
The recommendation system generally consists of five steps system and procedures in Simulation Method. The development stage of this research system design using the Simulation method as described in chapter previously, the stages were problem formulation, conceptual model, modelling, experiment, verification and validation, and Output Analysis.

Problem formulation
Communication with people with disabilities has obstacles and difficulties due to their limited sense of hearing and speaking ability. Therefore, it is necessary to teach sign language as a communication tool for persons with disabilities with the general public [11].
In addition to the use of everyday language, people with disabilities certainly need to learn new terms outside of their everyday language. These new terms include a person's individual name, foreign language terms, and scientific terms [12].
When learning new terms that will be taught to deaf and speech-impaired people, mistakes often occur because of insignificant differences in movement, for example, hand signals for the letters M and N [13].  The error in learning this new term occurred due to several factors. These factors include a language that continues to develop, a sign language dictionary that is not updated, and the lack of clarity in the movements that teachers teach to deaf and speech impaired people who are also affected by distance.

Conceptual model
The system simulation design stage is based on the conceptual modeling designed at this stage. This model contains the steps taken in identifying sign language using the Principal Component Analysis (PCA) method as a feature extraction method and the Adaptive Neuro-Fuzzy Inference System (ANFIS) as a classification method. These steps are illustrated in Figure   There are two groups of data that will be processed in order to obtain the value of the influential characteristics of each image. The first data group is the training data group that will be used to obtain standardization appropriate hand signals. The second data group is the test data group that will be used as a sample in testing using ANFIS. From this process will produce 5 PCA values that represent the characteristics of an image and stored in an excel file. This value will be displayed on the application, both in individual PCA retrieval and automatically. The PCA value from the result of feature extraction is depicted in Figure 6 below:

Verification and validation
At this stage, the suitability of the simulation program will be checked in applying the research In the test data group, a target of "TRUE" is given according to each letter in the five samples of test data and two data samples that have a target value of "FALSE". In the target value "FALSE", if this number is read by the system it will be considered an error.

Experimentation
In this study, the authors conducted an experiment by changing the size of the image used, both test data and training data. In this experiment, each use of different experimental variables will have different ANFIS rules as well.
a. Scenario 1 In this scenario, the PCA value data from the image size of 449x449 will be used. This scenario uses 42 images for training data and 42 images for test data. The ANFIS rules created are depicted in accordance with Figure 6. From the data that has been obtained, it is known that only 10 of the 42 tested data are incorrect, so in percentage form, the accuracy obtained from testing using the ANFIS rules that have been made in this scenario has a value of 76.20%. b. Scenario 2 In this scenario, the PCA value data from the image size of 281x281 will be used. This scenario uses 42 images for training data and 42 images for test data. The ANFIS rules created are depicted in accordance with Figure 7. From the data that has been obtained, it is known that 15 of the 42 data tested are incorrect, so in the form of a percentage, the accuracy obtained from testing using the rules.ANFIS that has been made in this scenario is 64.29%. c. Scenario 3 In this scenario, the PCA value data from the image size of 57x57 will be used. This scenario uses 42 images for training data and 42 images for test data. The ANFIS rules created are depicted according to Figure 8. From the data that has been obtained, it is known that 18 of the 42 data tested are incorrect, so in percentage form, the accuracy obtained from testing using the ANFIS rules that have been made in this scenario is 52.38%. d. Scenario 4 From the data that has been obtained, it is known that only 7 of the 18 data tested are correct, so in percentage form, the accuracy obtained from testing using the ANFIS rules that have been made in this scenario is very small at 52.38%.

Analysis output
The values that have been obtained through the Principal Component Analysis (PCA) method are then analyzed to determine the range of interval values for each letter.
In addition, the accuracy values that have been obtained in each scenario are also compared in this section. The comparison of these values will be a reference in determining the right size for the development of a hand signal simulation system.

Comparison of accuracy value
After conducting experiments with various sample sizes as variables, the results were obtained from the accuracy of each scenario. The results of the experiment are shown in table 1.  From table 1, it can be seen that the highest accuracy is in scenario 1, namely the use of the method using a size of 449x449 pixels of 76.20%. While the lowest accuracy, with a value of 52.38%, was obtained in scenarios 3 and 4 with sample sizes of 57x57 and 45x45 pixels. This is because each pixel in an image affects the retrieval of feature values in the image. In taking the image value using the Principal Component Analysis (PCA) method, the larger the size of the processed image, the greater the resulting PCA value.

IV. CONCLUSIONS
Based on the results of taking the value of Principal Component Analysis (PCA) and the testing and training process using the Adaptive Neuro-Fuzzy Inference System method, the following conclusions were obtained. 1. Feature extraction using the Principal Component Analysis (PCA) method and the Adaptive Neuro-Fuzzy Inference System (ANFIS) classification method can be implemented in hand signal detection simulations by pre-processing first, namely image crop, segmentation, and grayscale. 2. From the results of experiments with different image size variables, the largest accuracy was obtained with an image size of 449x449 of 76.20%. While the lowest accuracy is 52.38% with image sizes of 57x57 and 45x45. Therefore, differences in the use of image sizes have an influence on the accuracy of hand signal prediction.
3. The smaller the size given, the smaller the accuracy obtained. This is indicated by the decreasing accuracy value when given a smaller size in the four scenarios that have been studied.