Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> This paper represents an ongoing investigation of dexterous and natural control of upper extremity prostheses using the myoelectric signal (MES). The scheme described within uses pattern recognition to process four channels of MES, with the task of discriminating multiple classes of limb movement. The method does not require segmentation of the MES data, allowing a continuous stream of class decisions to be delivered to a prosthetic device. It is shown in this paper that, by exploiting the processing power inherent in current computing systems, substantial gains in classifier accuracy and response time are possible. Other important characteristics for prosthetic control systems are met as well. Due to the fact that the classifier learns the muscle activation patterns for each desired class for each individual, a natural control actuation results. The continuous decision stream allows complex sequences of manipulation involving multiple joints to be performed without interruption. Finally, minimal storage capacity is required, which is an important factor in embedded control systems. <s> BIB001 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> Context Improving the function of prosthetic arms remains a challenge, because access to the neural-control information for the arm is lost during amputation. A surgical technique called targeted muscle reinnervation (TMR) transfers residual arm nerves to alternative muscle sites. After reinnervation, these target muscles produce electromyogram (EMG) signals on the surface of the skin that can be measured and used to control prosthetic arms. Objective To assess the performance of patients with upper-limb amputation who had undergone TMR surgery, using a pattern-recognition algorithm to decode EMG signals and control prosthetic-arm motions. Design, Setting, and Participants Study conducted between January 2007 and January 2008 at the Rehabilitation Institute of Chicago among 5 patients with shoulder-disarticulation or transhumeral amputations who underwent TMR surgery between February 2002 and October 2006 and 5 control participants without amputation. Surface EMG signals were recorded from all participants and decoded using a pattern-recognition algorithm. The decoding program controlled the movement of a virtual prosthetic arm. All participants were instructed to perform various arm movements, and their abilities to control the virtual prosthetic arm were measured. In addition, TMR patients used the same control system to operate advanced arm prosthesis prototypes. Main Outcome Measure Performance metrics measured during virtual arm movements included motion selection time, motion completion time, and motion completion (“success”) rate. Results The TMR patients were able to repeatedly perform 10 different elbow, wrist, and hand motions with the virtual prosthetic arm. For these patients, the mean motion selection and motion completion times for elbow and wrist movements were 0.22 seconds (SD, 0.06) and 1.29 seconds (SD, 0.15), respectively. These times were 0.06 seconds and 0.21 seconds longer than the mean times for control participants. For TMR patients, the mean motion selection and motion completion times for hand-grasp patterns were 0.38 seconds (SD, 0.12) and 1.54 seconds (SD, 0.27), respectively. These patients successfully completed a mean of 96.3% (SD, 3.8) of elbow and wrist movements and 86.9% (SD, 13.9) of hand movements within 5 seconds, compared with 100% (SD, 0) and 96.7% (SD, 4.7) completed by controls. Three of the patients were able to demonstrate the use of this control system in advanced prostheses, including motorized shoulders, elbows, wrists, and hands. Conclusion These results suggest that reinnervated muscles can produce sufficient EMG information for real-time control of advanced artificial arms. <s> BIB002 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> A virtual reality system that improves the functional adjustment between an amputee and an active prosthesis is described. It includes the development of a virtual prosthesis and a myoelectric interface integration. The main purpose of this work is to provide a training system as a previous stage, to subjects who need to use an upper limb myoelectric prosthesis, which will allow that the users control easily a real prosthesis, optimizing the adaptation process through virtual training. <s> BIB003 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> Despite high classification accuracies (~95%) of myoelectric control systems based on pattern recognition, how well offline measures translate to real-time closed-loop control is unclear. Recently, a real-time virtual test analyzed how well subjects completed arm motions using a multiple-degree of freedom (DOF) classifier. Although this test provided real-time performance metrics, the required task was oversimplified: motion speeds were normalized and unintended movements were ignored. We included these considerations in a new, more challenging virtual test called the Target Achievement Control Test (TAC Test). Five subjects with transradial amputation attempted to move a virtual arm into a target posture using myoelectric pattern recognition, performing the test with vari- ous classifier (1- vs 3-DOF) and task complexities (one vs three required motions per posture). We found no significant difference in classification accuracy between the 1- and 3-DOF classifiers (97.2% +/- 2.0% and 94.1% +/- 3.1%, respectively; p = 0.14). Subjects completed 31% fewer trials in significantly more time using the 3-DOF classifier and took 3.6 +/- 0.8 times longer to reach a three-motion posture compared with a one- motion posture. These results highlight the need for closed-loop performance measures and demonstrate that the TAC Test is a use- ful and more challenging tool to test real-time pattern-recognition performance. <s> BIB004 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> Targeted muscle reinnervation (TMR) is a surgical technique that creates myoelectric prosthesis control sites for high-level amputees. The electromyographic (EMG) signal patterns provided by the reinnervated muscles are well-suited for pattern recognition control. Pattern recognition allows for control of a greater number of degrees of freedom (DOF) than the conventional, EMG amplitude-based approach. Previous pattern recognition studies have shown benefit in placing electrodes directly over the reinnervated muscles. Localizing the optimal TMR locations is inconvenient and time consuming. In this contribution, we demonstrate that a clinically practical grid arrangement of electrodes yields real-time control performance that is equivalent to, or better than, the site-specific electrode placement for simultaneous control of multiple DOFs using pattern recognition. Additional findings indicate that grid-like electrode arrangement yields significantly lower classification errors for classifiers with a large number of movement classes ( > 9). These findings suggest that a grid electrode arrangement can be effectively used to control a multi-DOF upper limb prosthesis while reducing the time and effort associated with fitting the prosthesis due to clinical localization of control sites on amputee patients. <s> BIB005 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> Electromyography shows the electrical activity of the muscles of the body. EMG signals can be recorded using surface electrodes. The SEMG signals thus obtained are used for varied applications most of which contribute towards prosthetic technology and rehabilitation engineering. SEMG signals are used as control signals for robotic arm functions as well. The aim of this study is to construct a prototype which uses these SEMG signals recorded form the brachioradialis muscle of the forearm to control the movement of powerpoint slides transmitted wirelessly in real time. The methodology involves using surface electrodes to acquire EMG signals which is sent to an ADC interfaced with a microcontroller. Using Zigbee protocol the signal is transmitted wirelessly to a PC where the powerpoint slides are contained. The powerpoint is displayed in visual basic window and embedded C language is used for microcontroller programming. The results showed 100mV voltage threshold for opening and closing of the fingers in the EMG signal. For the operation of the slides the fast upward action of the forearm caused a voltage higher than 90mV which leads to the backward movement of the slide and slow upward action of the forearm caused a voltage range between 85mV–89mV which leads to the forward movement of the slide. When this voltage is matched in the database the slides move. In conclusion a wireless control using EMG in real time application is achieved for operation. <s> BIB006 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> BackgroundMyoelectric pattern recognition systems can decode movement intention to drive upper-limb prostheses. Despite recent advances in academic research, the commercial adoption of such systems remains low. This limitation is mainly due to the lack of classification robustness and a simultaneous requirement for a large number of electromyogram (EMG) electrodes. We propose to address these two issues by using a multi-modal approach which combines surface electromyography (sEMG) with inertial measurements (IMs) and an appropriate training data collection paradigm. We demonstrate that this can significantly improve classification performance as compared to conventional techniques exclusively based on sEMG signals.MethodsWe collected and analyzed a large dataset comprising recordings with 20 able-bodied and two amputee participants executing 40 movements. Additionally, we conducted a novel real-time prosthetic hand control experiment with 11 able-bodied subjects and an amputee by using a state-of-the-art commercial prosthetic hand. A systematic performance comparison was carried out to investigate the potential benefit of incorporating IMs in prosthetic hand control.ResultsThe inclusion of IM data improved performance significantly, by increasing classification accuracy (CA) in the offline analysis and improving completion rates (CRs) in the real-time experiment. Our findings were consistent across able-bodied and amputee subjects. Integrating the sEMG electrodes and IM sensors within a single sensor package enabled us to achieve high-level performance by using on average 4-6 sensors.ConclusionsThe results from our experiments suggest that IMs can form an excellent complimentary source signal for upper-limb myoelectric prostheses. We trust that multi-modal control solutions have the potential of improving the usability of upper-extremity prostheses in real-life applications. <s> BIB007 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> Previous studies have showed that arm position variations would significantly degrade the classification performance of myoelectric pattern-recognition-based prosthetic control, and the cascade classifier (CC) and multiposition classifier (MPC) have been proposed to minimize such degradation in offline scenarios. However, it remains unknown whether these proposed approaches could also perform well in the clinical use of a multifunctional prosthesis control. In this study, the online effect of arm position variation on motion identification was evaluated by using a motion-test environment (MTE) developed to mimic the real-time control of myoelectric prostheses. The performance of different classifier configurations in reducing the impact of arm position variation was investigated using four real-time metrics based on dataset obtained from transradial amputees. The results of this study showed that, compared to the commonly used motion classification method, the CC and MPC configurations improved the real-time performance across seven classes of movements in five different arm positions (8.7% and 12.7% increments of motion completion rate, resp.). The results also indicated that high offline classification accuracy might not ensure good real-time performance under variable arm positions, which necessitated the investigation of the real-time control performance to gain proper insight on the clinical implementation of EMG-pattern-recognition-based controllers for limb amputees. <s> BIB008 </s> Real-Time EMG Based Pattern Recognition Control for Hand Prostheses: A Review on Existing Methods, Challenges and Future Implementation <s> Real-Time Using Virtual Reality <s> The myoelectric prosthetic hand is a powerful tool developed to help people with upper limb loss restore the functions of a biological hand. Recognizing multiple hand motions from only a few electromyography (EMG) sensors is one of the requirements for the development of prosthetic hands with high level of usability. This task is highly challenging because both classification rate and misclassification rate worsen with additional hand motions. This paper presents a signal processing technique that uses spectral features and an artificial neural network to classify 17 voluntary movements from EMG signals. The main highlight will be on the use of a small set of low-cost EMG sensor for classification of a reasonably large number of hand movements. The aim of this work is to extend the capabilities to recognize and produce multiple movements beyond what is currently feasible. This work will also show and discuss about how tailoring the number of hand motions for a specific task can help develop a more reliable prosthetic hand system. Online classification experiments have been conducted on seven male and five female participants to evaluate the validity of the proposed method. The proposed algorithm achieves an overall correct classification rate of up to 83%, thus, demonstrating the potential to classify 17 movements from 6 EMG sensors. Furthermore, classifying 9 motions using this method could achieve an accuracy of up to 92%. These results show that if the prosthetic hand is intended for a specific task, limiting the number of motions can significantly increase the performance and usability. <s> BIB009
To be able to control virtual prosthesis and to become familiar with a real-time prosthesis, voluntary muscle contraction control is very important; this can be done by using a visual feedback system. The improvement of learning depends on the user and visual feedback system, thus, the feedback system must allow the user to learn new tasks using their muscles BIB003 . Most of the virtual prosthesis followed the same stages to operate. Firstly, the EMG signal acquisition takes place using electrodes on the residual muscle of amputees. Then, the signal is amplified and filtered to acquire the myoelectric signal to be used. The interface between the virtual system and acquisition of myoelectric signal is created, which consists of isolation, pre-processing of the signal in the hardware, personal computer (PC) communication, communication between PC and MATLAB, processing in software and communication between MATLAB and virtual world BIB003 (part of MATLAB). The general idea followed by most of the virtual prosthesis is shown in Figure 4 . The continuing the examination of real-time control of prostheses using the myoelectric signal resulted in a robust scheme pattern recognition BIB001 . Twelve subject data from four channels were used for real-time control. Unlike the traditional methods involving transient control, which requires initiation from rest, a continuous stream of class decisions was delivered to the prosthetic device. Pattern recognition was performed on sliding time windows with 256 ms in duration and with the LDA classifier. The continuous decision (intended motion) permitted intricate classifications involving multiple joints without disruption. The continuous classifier performs very well with a significant gain in accuracy and response time over a wide range of analysis window lengths if accompanied by majority vote post-processing. Moreover, the control scheme required minimal storage capacity. Ann et al. BIB004 presented that the target achievement control (TAC) test in the virtual environment provides a good platform for PR control practice and testing. In a TAC test, virtual prosthesis moved from an inactive position to the target position. In 2015, Martina and Haripriya BIB006 constructed a prototype using sEMG signal to record the data from the brachioradialis muscle of forearm to control the movement of powerpoint slides transmitted in real-time. Furthermore, Agamemnon et al. BIB007 performed an experiment on 20 able-bodied and two amputees to find the outcome of two sensors (sEMG and inertial measurement (IM)). Twelve electrodes were used to acquire sEMG signal in a sampling frequency of 2 kHz. Feature sets such as MAV, WL, 4AR and logVar were extracted using a sliding window of 256 ms with 50 ms increments. Two sets of the experiment (offline and real-time) were performed. Real-time prosthetic hand control was based on offline observation. Touch Bionics 'robotic hand' have been used for real-time performance. It shows that the combination of both IM and sEMG improved the classification performance of a prosthetic hand. Additionally, the use of IM and sEMG reduce number of sensor require to achieve high level of accuracy. Yanjuan et al BIB008 investigated that both offline motion classification accuracy and real-time motion completion rate are important to assess the performance of EMG-PR control. Identifying multiple DOF (hand movements) using a few EMG sensors is one of the necessities for developing high levels of usability prosthetic hands. Trongmun et al. BIB009 present a signal processing technique that classifies 17 spontaneous classes of motion from EMG signals using spectral features and an ANN. Online classification experiments were performed on twelve subjects (seven male and five female) to assess the reliability of the proposed method. An overall correct classification rate of 83% was achieved, showing the ability to classify 17 movements from six EMG sensors. Moreover, the classification of nine movements could achieve accuracy of up to 92%. EMG pattern classification has been widely studied to decode user-determined for intuitive prosthesis control. The significant breakthrough was occurred with the introduction of surgical procedure to improve the control of hand prosthesis known as targeted muscle reinnervation (TMR) BIB005 . The real-time and offline performance of EMG-PR with TMR patients was presented using a generic electrode grid. Four amputee subjects (two trans-humeral, two shoulder articulation) that underwent TMR surgery participated in this study. In a real-time virtual analysis as well as offline classification, a generic grid-like electrode performed better than the control site (specific site for electrode placement). Although TMR has the potential to provide advanced control of wrist and grasp patterns for myoelectric control, the concept has not yet been a success in implementing it to multiple DOFs for the prosthesis. For assessing the real-time PR control of TMR based multifunction prostheses, Todd et al. BIB002 showed the performance outcome based on motion (selection time, completion time and completion rate). The experiment was performed on both virtual and real prosthesis. The performance was first ascertained by training and testing with a virtual multifunction prosthesis. Later on, the experiment was carried on three TMR patients with upper-limb prostheses. The mean classification accuracy of 88% with standard deviation (SD) of 7% for patients who had undergone TMR surgery and 97% with standard deviation (SD 2%) for control participants was achieved. Furthermore, the summary of some studies based on real-time using the virtual reality environment is presented in Table 3 . 