Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> There have been many attempts to perform a frequency analysis of speech and to use the outputs of this analysis to provide cutaneous stimulation. The results of these tests have left in doubt the issue of whether or not cutaneous recognition of speech is actually possible. In addition to other difficulties, an optimum frequency analysis has never been achieved. Instead, filtering configurations have been chosen essentially arbitrarily. The systems considered may well have had insufficient channel capacity for speech recognition even in the event that the tactile stimulators were arranged in an optimum manner. In this paper, several sets of finite frequency band to discrete channels filters were considered. The frequency of each discrete channel was constrained to the fixed center frequency of the corresponding band so as to be directly translatable to the position of a tactile stimulator. Tests were conducted to measure the auditory recognition rate of speech, resynthesized as the sum of these discrete ch... <s> BIB001 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> Study objectives: We sought to improve speech in tracheostomized individuals receiving positivepressure ventilation. Such individuals often speak with short phrases, long pauses, and have problems with loudness and voice quality. Subjects: We studied 15 adults with spinal cord injuries or neuromuscular diseases receiving long-term ventilation. Interventions: The ventilator was adjusted using lengthened inspiratory time (TI), positive end-expiratory pressure (PEEP), and combinations thereof. Results: When TI was lengthened (by 8 to 35% of the ventilator cycle), speaking time increased by 19% and pause time decreased by 12%. When PEEP was added (5 to 10 cm H2O), speaking time was 25% longer and obligatory pauses were 21% shorter. When lengthened TI and PEEP were combined (with or without reduced tidal volume), their effects were additive, increasing speaking time by 55% and decreasing pause time by 36%. The combined intervention improved speech timing, loudness, voice quality, and articulation. Individual differences in subject response to the interventions were substantial in some cases. We also tested high PEEP (15 cm H2O) in three subjects and found speech to be essentially identical to that produced with a one-way valve. Conclusions: These simple interventions markedly improve ventilator-supported speech and are safe, at least when used on a short-term basis. High PEEP is a safer alternative than a one-way valve. (CHEST 2003; 124:1512–1521) <s> BIB002 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> Besides its clinical applications, various researchers have shown that EMG can be utilised in areas such as computer human interface and in developing intelligent prosthetic devices. The paper presents results from a preliminary study. The work describes the outcome in using an artificial neural network (ANN) to recognise and classify human speech based on EMG. The EMG signals were acquired from three articulatory facial muscles. Three subjects were selected and participated in the experiments. Preliminarily, five English vowels were used as recognition variables. The root mean square (RMS) values of the EMG signals were estimated and used as a set of features to feed the ANN. The findings indicate that such a system may have the capacity to recognise and classify speech signals with an accuracy of up to 88%. <s> BIB003 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> A technique for improving the recognition accuracy of EMG-based speech recognition by applying existing speech recognition technologies is proposed. The authors have proposed an EMG-based speech recognition system that requires only mouth movements, voice need not be generated. A multi-stream HMM (hidden Markov model) and feature extraction technique are applied to EMG-based speech recognition. 3 channel facial EMG signals are collected from ten subjects when uttering 10 Japanese isolated digits. One channel corresponds to one stream. By examining various features, we found that the delta component of the static parameter leads to higher accuracy. Compared to equal stream weighting, the individual optimization of stream weights increased recognition accuracy by 4.0% which corresponds to a 12.8% reduction in error rate. This result shows that multistream HMM is effective for the classification of EMG. <s> BIB004 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> A Coupled Hidden Markov Model (CHMM) is proposed in this paper to perform multimodal speech recognition using myoeletric signals (MES) from the muscles of vocal articulation. MES signals are immune to noise, and words that are acoustically similar manifest distinctly in MES. Hence, they would effectively complement the acoustic data in a multimodal speech recognition system. Research in Audio-Visual Speech Recognition has shown that CHMMs model the asynchrony between different data streams effectively. Hence, we propose CHMM for multimodal speech recognition using audio and MES as the two data streams. Our experiments indicate that the multimodal CHMM system significantly outperforms the audio only system at different SNRs. We have also provided a comparison between different features for MES and have found that wavelet features provide the best results. <s> BIB005 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> In this paper we introduce a speech recognition system based on myoelectric signals. The system handles audible and non-audible speech. Major challenges in surface electromyography based speech recognition ensue from repositioning electrodes between recording sessions, environmental temperature changes, and skin tissue properties of the speaker. In order to reduce the impact of these factors, we investigate a variety of signal normalization and model adaptation methods. An average word accuracy of 97.3% is achieved using seven EMG channels and the same electrode positions. The performance drops to 76.2% after repositioning the electrodes if no normalization or adaptation is performed. By applying our adaptation methods we manage to restore the recognition rates to 87.1%. Furthermore, we compare audibly to non-audibly spoken speech. The results suggest that large differences exist between the corresponding muscle movements. Still, our recognition system recognizes both speech manners accurately when trained on pooled data <s> BIB006 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> It is well known that a strong relationship exists between human voices and the movement of articulatory facial muscles. In this paper, we utilize this knowledge to implement an automatic speech recognition scheme which uses solely surface electromyogram (EMG) signals. The sequence of EMG signals for each word is modelled by a hidden Markov model (HMM) framework. The main objective of the work involves building a model for state observation density when multichannel observation sequences are given. The proposed model reflects the dependencies between each of the EMG signals, which are described by introducing a global control variable. We also develop an efficient model training method, based on a maximum likelihood criterion. In a preliminary study, 60 isolated words were used as recognition variables. EMG signals were acquired from three articulatory facial muscles. The findings indicate that such a system may have the capacity to recognize speech signals with an accuracy of up to 87.07%, which is superior to the independent probabilistic model. <s> BIB007 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> The aim of this work was to model facial electromyography (fEMG) to find optimal electrode positions for wearable human-computer interface system. The system is a head cap developed in our institute and with it we can measure fEMG and electro-oculogram (EOG). The signals can be used to control the computer interface: gaze directions move the cursor and muscle activations correspond to clicking. In this work a very accurate 3D model of the human head was developed and it was used in the modeling of fEMG. The optimal positions of four electrodes on the forehead measuring the activations of frontalis and corrugator muscles were defined. It resulted that the electrode pairs used in frontalis and corrugator measurements should be placed orthogonally by comparison with each other to get a signal that enables the separation of those two different activations. <s> BIB008 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> The electrolarynx (EL) is a common rehabilitative speech aid for individuals who have undergone total laryngectomy, but they typically lack pitch control and require the exclusive use of one hand. The viability of using neck and face surface electromyography (sEMG) to control the onset, offset, and pitch of an EMG-controlled EL (EMG-EL) was studied. Eight individuals who had undergone total laryngectomy produced serial and running speech using a typical handheld EL and the EMG-EL while attending to real-time visual sEMG biofeedback. Running speech tokens produced with the EMG-EL were examined for naturalness by 10 listeners relative to those produced with a typical EL using a visual analog scale. Serial speech performance was assessed as the percentage of words that were fully voiced and pauses that were successfully produced. Results of the visual analog scale assessment indicated that individuals were able to use the EMG-EL without training to produce running speech perceived as natural as that produced with a typical handheld EL. All participants were able to produce running and serial speech with the EMG-EL controlled by sEMG from multiple recording locations, with the superior ventral neck or submental surface locations providing at least one of the two best control locations. <s> BIB009 </s> Alternative communication systems for people with severe motor disabilities: a survey <s> Automatic speech recognition (ASR) <s> The possibility of speech processing in the absence of an intelligible acoustic signal has given rise to the idea of a 'silent speech' interface, to be used as an aid for the speech-handicapped, or as part of a communications system operating in silence-required or high-background-noise environments. The article first outlines the emergence of the silent speech interface from the fields of speech production, automatic speech processing, speech pathology research, and telecommunications privacy issues, and then follows with a presentation of demonstrator systems based on seven different types of technologies. A concluding section underlining some of the common challenges faced by silent speech interface researchers, and ideas for possible future directions, is also provided. <s> BIB010
Since the late 1960s, efforts have been made to achieve a system for speech recognition BIB001 . Several pieces of software are available in the market and modern operating systems for personal computers already offer built-in speech recognition. But there are a few drawbacks that offer some resistance for using usual ASR systems and intensify the research on silent speech interfaces: (i) the audible speech prohibits confidential conversation; (ii) it is not advised to use such systems during meetings or inside a library; (iii) the performance decays severely in adverse environments such as crowded places; and finally (iv) some clinical conditions hinder voice communication. There is a relation among the words pronounced and movements of articulatory facial muscles. Then, a feasible approach is to use the activation of those muscles to identify phonemes and therefore, words. This is not an easy task, as the act of speech employs several facial muscles, such as: Mentalis, Depressor anguli oris, Massetter, Digastric, Zygomaticus major, Levator anguli oris, Platysma, and Orbicularis oris. Studies usually regard people with voice impairment, such as conditions after a total laryngectomy BIB009 or to situations where the ambient noise impedes communication (e.g. by fire fighters and pilots). Therefore it must be carefully analysed if the level of disability may compromise the control over the muscles involved in the speech process. Nevertheless, people with severe motor impairment could use ASR by muscle activity to achieve a channel of communication. One example is people with tetraplegia using ventilator systems that are adjusted to accommodate cardiopulmonary requirements, but that are not optimal for speech. Speech produced with typical ventilator adjustments is often characterized by short phrases, long pauses between phrases, abnormal loudness, and poor voice quality BIB002 . In a study conducted by Denby et al. BIB010 over silent speech interfaces -systems enabling speech communication when an audible acoustic signal is unavailable -several solutions are compared regarding if the systems: (i) are invasive, (ii) work in noisy environments, (iii) require glottal activity, (iv) are ready for market, (v) work for laryngectomy and (vi) have low cost. Among the seven systems analyzed, the EMG based had the highest overall evaluation. In BIB007 , Hidden Markov Models were used to map muscle activation into phonemes. The features extracted from the EMG signal were mel-frequency cepstral coefficients (MFCC), as previous studies showed that discrete wavelet transform (DWT) coefficients were superior but slightly different. Only three channels were used with respect to the muscles levator anguli oris, the zygomaticus major, and the depressor anguli oris. The muscles used and the electrodes were defined heuristically. To evaluate the system a limited vocabulary of 60 words was used and accuracy of up to 85% was achieved. In BIB005 a multimodal ASR with the acoustic information and the EMG signal allowed a Coupled Hidden Markov Model (CHMM) to recognize speech. This solution is compared with two others: audio only and EMG only. Adding different levels to the signal, it showed that the audio-only approach is highly dependent on the SNR, while the EMG-only proposal was not affected. Five muscle channels were used: the Levator anguli oris, the Zygomaticus major, the Platysma, the Depressor anguli oris, and the anterior belly of the Digastric. No criteria were indicated for choosing neither muscles nor the electrodes position. The vocabulary used was extremely restricted, with only 10 words. In BIB003 only vowels were used as shape of lips and mouth cavity were stationary. Three channels were used, with information recorded from the Mentalis, Depressor anguli oris and Massetter muscles, since those are the most active muscles during vowels pronunciation. An artificial neural network (ANN) using the back-propagation algorithm was used to associate the RMS of the EMG signal with the vowels. Other studies provide the recognition of isolated words and small vocabulary BIB006 BIB004  . In  as the aim was to recognize speech of pilots that could be interpreted as commands, electrodes were embedded in a pilot oxygen mask. The error rate was very low, ranging from 0% to 10.4% during the task of recognizing the speech of the numbers 'zero' to 'nine'. The probability that electrodes are repositioned in the same place as the previous session is very low. In BIB006 a normalization method found that among sessions, the accuracy to drop about 10%, whereas without the method the accuracy dropped more than 21%. Eight channels of information were extracted from the following muscles Levator angulis oris, the Zygomaticus major, the Platysma, the Depressor anguli oris, the anterior belly of the Digastric and the tongue. Only the numbers 'zero' to 'nine' composed the vocabulary. Wand et al.  and Jou et al.  , implemented continuous speech recognition using an HMM algorithm. The vocabulary was phonetically balanced and formed by 108 words. A total of six channels were used. When compared to features from frequency and time-frequency domains, the Wavelet transform showed a slightly advantage. Studies show good results as almost 90% of accuracy is obtained. On the other hand, the vocabulary used is usually extremely restricted. But as it happens with other categories of assistive technology devices, people with severe motor impairments may find even a limited control extremely useful. If 60 words could be associated to different actions, common sense dictates that even this would be extremely helpful for daily activities. Other issue is that the data used to test each system were obtained under highly controlled situations, with the subjects being under supervision. During normal operation, the user will probably be less concentrated, the pronunciation sometimes will be less clear and the system may not respond very well. As the goal is to associate phonemes with activation of related muscles, it is interesting to define some criteria to decide which muscles should be used, as well the electrode positioning minimizing crosstalk. Although not applied to ASR, in BIB008 the assessment was conducted for better positioning of electrodes in the forehead, so both electrodes could gather information from different muscles with minimum interference.