Time domain analysis of electroencephalogram (EEG) signals for word level comprehension in deaf graduates with congenital and acquired hearing loss

Deafness can be classified on the basis of onset as congenital and acquired hearing loss. The brain is a sensitive part of our body, electrical pulses from the neurons interact with each other, generating brain signals. EEG signals are extensively used for clinical diagnosis for any brain anomalies, language comprehension and performance measurement studies. This study mainly focuses on analysing the word level comprehension in deaf adults in the age group (21 -25 years) using EEG signals. The raw EEG signals were pre-processed and the relevant time domain linear and nonlinear features were extracted and classified using machine learning algorithms. The approximate entropy feature was found to be best suited for finding the comprehension of both congenital and acquired deaf adults. This feature of ISL was observed to be achieving better classification rate with a maximum average accuracy of 96% in both congenital and acquired deaf adults using SVM classifier.


1.Introduction
Hearing is a complex function of ear which allows discrimination of sounds. Hearing plays a key role in facilitating the language and speech advance [1]. As the table 1 shows, Goodmann classified deafness on the basis of the hearing impairment severity [2]. The brain will undergo reorganization after sensory deprivation. This plastic reorganization develops over time. Critical periods and the influence of the duration of sensory deprivation affects the reorganization of sensory cortices [9]. Hearing loss impacts brain function. The brains of people with different degrees of hearing loss respond differently. This is due to cortical reorganization [10]. It is the brain's tendency to adapt to a loss by rewiring itself [11].
Comprehension includes the selection of the correct meaning [12] to the words according to the context, and the ability to grasp, the meaning of a larger unitary idea. Understanding the meaning of the word is word level comprehension. It is also called lexical comprehension. Syntactic and vocabulary skills has often been suggested to be a fundamental factor in the progress of reading in the deaf population [13] [14]. Several studies have used Electroencephalography (EEG) as a physiological measure to determine the comprehension. EEG is a measure of the electrical activity of brain signals or electrical pulses at various locations that are picked up by electrodes connected to the scalp. It is a complementary source of feedback as non-invasively [15] feasible from as close to the brain. EEG offers good time [16] and spatial resolution [17].The speed of the brainwave is measured in Hertz (cycles per second) and they are separated into various bands [18]. The EEG signal is a voltage signal that can be calculated on the scalp surface coordinated neural activity from large areas (neurons groups firing at the same pace) manifested as synchronisation.
[20]. Mazaheri et al. investigated theta, alpha and beta frequency oscillatory behaviour, although previous studies have included these bands in different aspects of language processing, including lexical and semantic processing. [21]. Most of the low-frequency (infra-slow oscillations below0.5Hz) EEG studies have concentrated on different forms of cognitive tasks. [22]. Infra-slow oscillations are beyond the traditional clinical EEG bandwidth, but with the advent of digital signal processing, they have recently found clinical relevance. Studying the kinds of cognitive processes in each of the EEG bands that modulate behaviour allows one to understand the meaning of the connections that we observe between the behaviour of the EEG and our tasks. In addition, it helps us to identify the appropriate features to be used in training classifiers and thus to assess the EEG.
Event-related brain potential (ERP) estimate activity [23] which is harmonized to certain external stimuli, such as reading a word, or a sentence. The signal from an ERP is poor (5-10 μV) and noisy EEG signal is stronger (50-100 µV) [24]. So the actual signal is extracted from the noisy signal by an averaging procedure. This is accomplished by time-locking multiple EEG recordings called trials and averaging. Thus the event-related brain activity is got. Time domain refers to the analysis of physical signals with respect to time.
The spectral power and spectral entropy characteristics of gamma rhythms were extracted from the reported Auditory Evoked Potential (AEP) signals in a study conducted in Malaysia and applied to machine-learning algorithms to categorise the AEP signal dynamics in terms of specificity, sensitivity and accuracy of the left and right ear classification in 9 subjects, and observed as 96.75 per cent accuracy in separating the typically hearing and deaf subjects [25].
Recent studies with individuals who are deaf used machine learning to extract information by classification [26][27] by classifiers [28] [29]. One of the key theories in machine learning is that training data on which the classifier is trained and test data on which the classifier is evaluated are part of the same feature space [30]. The performance of the classifier is measured in terms of recognition rate [31]. Support Vector Machines [32], K Nearest Neighbor [31], ensemble averaging, are the classifiers used here. Feature extraction and classification are two critical areas in analysis [33].

Study Design and Method
Figure1 depicts path way of word level comprehension recognition system in deaf adults using the EEG signals. In the present work, the EEG data is acquired from the deaf adults who are graduates from National Institute of Speech & Hearing (NISH), Kerala using Clarity Brain Tech devices, when they are comprehending progressive words in English, ISL and Malayalam. The acquired raw EEG data is subjected to various noises due to the environmental factors and the movement of the participants. Using different digital filter, these artifacts are eliminated by pre-processing of the raw EEG signal. Statistical, linear and nonlinear characteristics were derived by DWT from the filtered EEG signal. Using machine learning algorithms such as Ensemble, SVM, and KNN classifiers, the comprehension of words in three languages (classes) for both group DAC and DAA were categorised.  (2) and Sign Language (1).

Design of EEG acquisition Protocol.
As the language of deaf adults with no early intervention is not appropriate to their age, words used to give stimulus were chosen from the words found in their note books during graduation study. This was to make sure that they were exposed or taught those words, so that they can comprehend during EEG recording. Three sets of twenty seven words were chosen. Each of the twenty seven words belong to nine sets of three level progression words (Example: Mother, Grandmother, Great grandmother; Stone, Rock, Mountain) which were displayed in the form of PowerPoint presentation. Three sets were in English, three in Malayalam and three in ISL for each trial.
The experiment protocol for each participant was three trials with 15 seconds interstimulus interval (ISI). Each trial lasts for duration of 135 seconds. Each word appeared on the screen for 5 seconds.
Recording started with a baseline period of 15 seconds followed by three trials of nine sets of words, with ISI as shown in Figure 2. The participants were then given an assessment questionnaire to check the following. 1. Number of words comprehended 2. Whether they identified the relationship. If they did not recognise, hints were given and then checked.
All participants read the procedure and instructions of EEG recording prior to recording. EEG recorded while the participants were reading and comprehending the words displayed as a PowerPoint presentation. After EEG recording, participants were given a questionnaire which checked their comprehension. The participants were seated comfortably in a chair and the electrodes were fixed on the scalp. The EEG was sampled with 18 electrodes of Clarity Brain Tech EEG device with 32 channels. Data with a sampling rate of 256 Hz and 10-bit resolution was recorded. Stimuli were progressive words in English, Malayalam and Indian Sign Language (ISL). These were presented as Power point presentation in the centre of a 17" laptop kept approximately 2.5 feet from the participants. Word stimuli were presented in three trials with interstimulus interval as in Figure 2. Each word was presented once during the experiment, for a total of 27 English words, 27 Malayalam words and 27 words displayed using ISL videos. EEG electrodes were placed on the scalp following standard 10-20 International placements [34] and the electrodes were bipolar with channel and a reference point. One ECG electrode was fixed on the hand. While recording, impedance was maintained below 10 kΩ at all electrodes.
Using a longitudinal bipolar montage, all channels were re-referenced, which improves the signal by removing similarities in neighbouring channels, leading to magnification of the characteristics associated with that area and elimination of common noise between channels. Bipolar mounting is a simple and fast spatial philtre widely used to decrease the background noise in the EEG to increase the signal-to -noise ratio. Positioning of electrode is referred as montage [33]. Eighteen EEG channels throughout the cerebrum were used to obtain eighteen bipolar channels, viz. The participants were given their own time to settle down and they were presented the PowerPoint presentation as shown in Figure 3. The comprehension EEG signals were collected while participants were reading the words. Eighteen electrodes were used to acquire EEG waves and one was connected in the arm to record ECG waves. The signal measurement system consists of electrode signals as input, which was integrated together after some delay. These were first amplified here, using differential amplifier. The disparity between the two inputs are magnified. It will subtract an unnecessary signal common to the two inputs. Then filtered and converted to digital output. Here the electrodes used were bipolar with the channel and a reference point, the difference of activities at those points are recorded as EEG.
All participants read the procedure and instructions of EEG recording prior to recording. It was also interpreted in ISL. A video of EEG recording procedure was also shown to each participant. EEG was recorded while the participants were reading and comprehending the words displayed as a PowerPoint presentation. After EEG recording, participants were given a questionnaire that checked if they have comprehended the words correctly. This questionnaire was also interpreted in ISL. There were a total of 81 words (Malayalam=27, English=27, ISL=27) displayed as stimuli. DAA participants decoded 80 and 79 words respectively, whereas DAC participants decoded 60 and 64 words each. DAC confessed that they comprehended all the ISL words and found difficulty with Malayalam and English words. Whereas, though the DAA comprehended almost all words, they found both English and Malayalam words to be easy, they said. Number of words comprehended is shown in Figure 4.

Pre-Processing
EEG data obtained from scalp electrodes can be viewed as a neural activity description, and objects are independent of each other. The raw EEG signals that was acquired from the participants were corrupted with a lot of physiological and non-physiological artifacts. The artifacts mainly consist of Power line Interferences (PLI), Eye blink, Eye movement, muscle artifacts like yawning, swallowing, laughing, stretching, mouse clicking etc.
Artifacts introduce spikes and this can affect the useful information from the EEG signal. Artifact can mimic almost any type of electrocerebral activity on the EEG [35]. To ensure proper examination and diagnosis, these undesirable signals must be extracted or attenuated from the EEG [36]. These artifacts are removed using Butterworth filter [37], FIR filter [38], IIR filter [39], Chebyshev filter [40], Adaptive filter [41]and Wavelet filter using discrete wavelet transform (DWT) [42].
In this work the acquired EEG signals were filtered and pre-processed using Moving average Low pass filter of window size 13 with cut off frequency 30 Hz and FIR filter of kaiser window parameter 0.37 and pass band frequency of 0.1 Hz. Filtered data comprises of the bands namely infra-slow oscillations, delta, theta, alpha, beta which has useful relevant information. Signals were sampled at the rate of 256Hz.

EEG Feature Extraction
MATLAB software was used for feature extraction and classification. Three parameters are included in the MATLAB programme: frequency, amplitude and duration, and stimulus signals are generated accordingly [40]. Since the Daubechies wavelet reflects the EEG signal characteristics, mathematical, linear and nonlinear characteristics were extracted using the EEG signal amplitude DWT (dB8) [43]. These wavelet functions are best suited for the stationery and non-stationary signals. Wavelets are the small oscillatory waves which can be used for the multiresolution analysis of the signal. Based on its similarity to the mother wavelet function characteristics of wavelet coefficients in the LF and HF bands, the Daubechies wavelet is selected. A single prototype feature called the mother wavelet is used, using the DWT to decompose the input signal based on scaling and shifting parameters. The mother w avelet function Ψa, b (t) is given as where a, b ϵ R, a > 0, and R is the wavelet space. Since the option of a prototype function as the mother wavelet should at all times fulfil the permissibility condition, the scaling factor and the shifting factor are the parameters 'a 'and' b '. (Equation 2), Where the fourier transform of Ψ (ω) is Ψ (a, b).
The features were calculated as in equations (3) to (12). [ Variability indicates how much spread out a set of data is. Hurst The characteristics of delta theta and alpha bands were extracted from the pre-processed signals. The time domain linear statistical features (Frequency, Mean, Maximum, Minimum, Skewness, Standard Deviation, Variance, Variability, Kurtosis) and non-linear features (Hurst, Approximate entropy) were extracted.

Classification on the basis of onset of deafness
The features that were extracted using DWT were statistically analysed using SPSS tool one-way analysis of variance (ANOVA). A Descriptive analysis was done by comparing the means of the derived features. The features whose p-value were less than 0.05 were chosen to be more significant for the analysis and are used for further processing. We selected significant features and classified them using the KNN, SVM and Ensemble classifier. Input variables (also known as predictors, features, or attributes) are based on a k-nearest neighbour classification model and output (response) is returned by 'fitcknn'(MATLAB function). The k value is chosen from 1 to 10 since the number of classes is two for classification. The distance metric that is used in KNN before any decision is the Euclidean distance [46]. SVM classifier produces an optimal. SVM is a linear machine that embeds the input data (features)which is in ndimensional space into high k-dimensional space generally k>n by nonlinear mapping of (x). A better generalization is done by optimising the margin while allowing some of the training set misclassification to prevent over fitting. [47]. Ensemble classifiers is a collection of classifiers whose individual choices are combined in some way (typically by weighted or unweighted voting) to classify new examples. The IOP Publishing doi:10.1088/1757-899X/1070/1/012083 9 study of methods for building good ensembles of classifiers [48] has been one of the most successful areas of supervised learning science.
In this work hold out validation was done to evaluate the performance of the classifiers. 70 percent and 30 percent of the dataset was used as training and testing sets for classifying two onset of deafness. % = * 100

Preprocessing
As shown in Figure 5 the raw EEG signal has artifacst such eye blink, muscle movement, power line interference, and other high frequency and low frequency noises. These artifatcs were removed using Moving average Low pass filter of window size 13 with cut off frequency 30 Hz and FIR filter of aiser window parameter 0.37 and pass band frequency of 0.1 Hz, and given as figure 6.

Raw EEG
Eighteen channels were connected. In this plot of raw signal, the channels FP1-

Statistical Data Analysis of the extracted features
Using one-way ANOVA the statistical significance of all twelve features obtained using DWT from the EEG signal was studied. The p-values are marked in Table 3. The results brought by this study also rise some interesting issues that justify additional research. While comparing the three languages, Maximum, Skewness, Approximate Entropy and Hurst showed significant values whose p-values were less than 0.05 for DAA. It is also noted that for DAC, the features Variability and Approximate Entropy gave significant values.

Classification of Congenital and Acquired deafness
The classification accuracy(figure 7) is shown in Table 4. Classifier SVM gives the maximum accuracy for all features analysed as well as the best feature identified which is Entropy. Using Entropy feature,

Conclusion
The purpose of this study was to explore, through the use of a technological tool, the word level comprehension in the process of reading Malayalam, English and ISL words. Artifacts introduce spikes, which can be confused as rhythms in neurology. To ensure proper examination and diagnosis, these undesirable signals must be extracted or attenuated from the EEG. The PLI is introduced whenever there is an electrical interference from other devices. The acquired EEG was filtered to remove artifacts and 12 features were extracted. Approximate entropy gave the best value. Classification was done for that using SVM classifier gave the highest accuracy. For DAC, ISL words seem to be the best comprehended ones. However for DAA, both ISL and English are comprehended equally. Words used to give stimulus were chosen from the words found in their note books during graduation study. This was to make sure that they were exposed or taught those words, so that they can comprehend during EEG recording. This may be the reason for English to get equal accuracy as ISL in DAA.

Limitations and Scope for Future Studies
This study compares electrophysiological signals of English, Malayalam and Indian Sign Language. The limitations are less number participants and less recording time. There is a scope for future studies on the features and classifiers for comprehension with more participants.