A Survey of Robot-Assisted Language Learning (RALL) <s> DISCUSSION 7.1 Challenges <s> Developmental changes in speech production introduce age-dependent spectral and temporal variability in the speech signal produced by children. Such variabilities pose challenges for robust automatic recognition of children's speech. Through an analysis of age-related acoustic characteristics of children's speech in the context of automatic speech recognition (ASR), effects such as frequency scaling of spectral envelope parameters are demonstrated. Recognition experiments using acoustic models trained from adult speech and tested against speech from children of various ages clearly show performance degradation with decreasing age. On average, the word error rates are two to five times worse for children speech than for adult speech. Various techniques for improving ASR performance on children's speech are reported. A speaker normalization algorithm that combines frequency warping and model transformation is shown to reduce acoustic variability and significantly improve ASR performance for children speakers (by 25-45% under various model training and testing conditions). The use of age-dependent acoustic models further reduces word error rate by 10%. The potential of using piece-wise linear and phoneme-dependent frequency warping algorithms for reducing the variability in the acoustic feature space of children is also investigated. <s> BIB001 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> DISCUSSION 7.1 Challenges <s> In this paper, we review: (1) the acoustic and linguistic properties of children's speech for both read and spontaneous speech, and (2) the developments in automatic speech recognition for children with application to spoken dialogue and multimodal dialogue system design. First, the effect of developmental changes on the absolute values and variability of acoustic correlates is presented for read speech for children ages 6 and up. Then, verbal child-machine spontaneous interaction is reviewed and results from recent studies are presented. Age trends of acoustic, linguistic and interaction parameters are discussed, such as sentence duration, filled pauses, politeness and frustration markers, and modality usage. Some differences between child-machine and human-human interaction are pointed out. The implications for acoustic modeling, linguistic modeling and spoken dialogue system design for children are presented. We conclude with a review of relevant applications of spoken dialogue technologies for children. <s> BIB002 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> DISCUSSION 7.1 Challenges <s> is the nature's gift to the human being which contributes towards the intelligence and discrimination from rest of the animal kingdom. Taking into consideration technological aspects, speech recognition is the buzzword today, as communication and hands free computing evolving day by day. Speech is a very important mode of the communication and interaction with the digital computer. Speech recognition along with the wide range of applicability in domain of computer science, medical science, psychology, sports, neurology has many challenges while developing. Developing real time speech recognizer may hurdle from adverse environment to anatomy of the human body. It also involves linguistic aspects too. This paper explores various challenges in developing a robust ASR system. General Terms Signal Processing and linguistics <s> BIB003 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> DISCUSSION 7.1 Challenges <s> ABSTRACTThe purpose of this research was to evaluate a prototype of an automatic speech recognition (ASR)-based language learning system that provides feedback on different aspects of speaking performance (pronunciation, morphology and syntax) to students of Dutch as a second language. We carried out usability reviews, expert reviews and user tests to gain insight into the potential of this prototype and the possible ways in which it could be further adapted or improved, with a view to developing specific language learning products. The evaluation revealed that domain experts and users (teachers and students) are generally positive about the system and intend to use it if they get the opportunity. In addition, recommendations have been made which range from specific changes and additions to the system to more general statements about the pedagogical and technological issues involved. These recommendations can be useful to improve this prototype and to develop other ASR-based systems, which can be deployed... <s> BIB004 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> DISCUSSION 7.1 Challenges <s> An increasing number of human-robot interaction (HRI) studies are now taking place in applied settings with children. These interactions often hinge on verbal interaction to effectively achieve their goals. Great advances have been made in adult speech recognition and it is often assumed that these advances will carry over to the HRI domain and to interactions with children. In this paper, we evaluate a number of automatic speech recognition (ASR) engines under a variety of conditions, inspired by real-world social HRI conditions. Using the data collected we demonstrate that there is still much work to be done in ASR for child speech, with interactions relying solely on this modality still out of reach. However, we also make recommendations for child-robot interaction design in order to maximise the capability that does currently exist. <s> BIB005 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> DISCUSSION 7.1 Challenges <s> Abstract Children speech recognition is challenging mainly due to the inherent high variability in children’s physical and articulatory characteristics and expressions. This variability manifests in both acoustic constructs and linguistic usage due to the rapidly changing developmental stage in children’s life. Part of the challenge is due to the lack of large amounts of available children speech data for efficient modeling. This work attempts to address the key challenges using transfer learning from adult’s models to children’s models in a Deep Neural Network (DNN) framework for children’s Automatic Speech Recognition (ASR) task evaluating on multiple children’s speech corpora with a large vocabulary. The paper presents a systematic and an extensive analysis of the proposed transfer learning technique considering the key factors affecting children’s speech recognition from prior literature. Evaluations are presented on (i) comparisons of earlier GMM-HMM and the newer DNN Models, (ii) effectiveness of standard adaptation techniques versus transfer learning, (iii) various adaptation configurations in tackling the variabilities present in children speech, in terms of (a) acoustic spectral variability, and (b) pronunciation variability and linguistic constraints. Our Analysis spans over (i) number of DNN model parameters (for adaptation), (ii) amount of adaptation data, (iii) ages of children, (iv) age dependent-independent adaptation. Finally, we provide Recommendations on (i) the favorable strategies over various aforementioned - analyzed parameters, and (ii) potential future research directions and relevant challenges/problems persisting in DNN based ASR for children’s speech. <s> BIB006
Compared to other types of robot-assisted learning, robots' speech recognition presents special challenges in language acquisition. Speech recognition can be faulty under "perfect" conditionsspeech produced by native speakers in quiet environments-due to differences in speakers, environmental noise, microphone quality, and language ambiguity (such as the presence of homophones or issues in detecting word boundaries in continuous speech) BIB003 . Speaker differences that may result in variability in automatic speech recognition (ASR) include the age, gender, speech rate, and regional dialects of speakers BIB003 . Research is underway to ameliorate these issues, but they are far from solved. Since participants in RALL studies are often children, it is of particular relevance to address the shortcomings of ASR in respect to this population. When attempting to detect the speech of children using ASR systems trained using adult speech, accuracy is reduced significantly, with word error rates 2-5 times greater than those for adults BIB001 . There are publicly available child-specific corpora that can be used to train ASR engines to improve recognition rates BIB002 . However, a paucity of relevant corpora for children still exists. Even with the use of corpora trained with children's speech, recognition rates are still not as high as they are for adults. Children exhibit greater developmental changes and variability in speech patterns BIB006 , yet they are treated as a homogeneous group. Collecting data specific to the target population's age and other demographics can improve error rate, but this method relies on intensive data collection that may not be practical for ASR systems meant for more general use BIB002 . Therefore, attempting to adapt adult-trained ASR systems to children's speech is also common BIB002 . Human understanding of speech is distinctly multimodal, with non-verbal cues serving to clarify or bolster meaning. Using such cues to improve the accuracy of such systems has been proposed BIB003 . Additionally, constraining user output to limit possible replies has been suggested and applied with success BIB004 . BIB005 evaluated automatic recognition of naturalistic child speech within HRI. Their work supports using multiple cues in addition to speech, such as a touchscreen, creating interaction scenarios to constrain learners' responses, and constraining what the ASR system can recognize. The authors additionally conclude that, for optimal speech recognition, the microphone is best placed within the robot, instead of within the environment, that the robot be able to track sound and orient itself toward it, and that the child be located in the best possible position in front of the robot BIB005 . It is important to realize that these issues with ASR are present for fluent language speakers. Language learners produce varied and unique errors that those who have fully acquired the language do not, and this amplifies speech recognition issues. These errors may be syntactic, semantic, or morphological. There may also be phonetic (pronunciation and accent) and pragmatic (contextual and real-world usage) differences that arise. There is still quite a ways to go to make ASR robust for use by language learners in naturalistic contexts, but this step is needed if robots are most useful in the conversational aspects of language learning. This makes the adaptive strategies mentioned earlier even more critical, as well as the employment of other strategies, whether they be based on technology or human intervention. Teaching methods, population-specific educational goals, and current technical means should inform when, if, and how to deal with such occurrences. When learning a language, it is also critical that the robot's speech be clear and correct. While this is favorable in any robot application, if learning occurs in a language where fluency has already been reached, reasonable interpretations can be made to the intent of the speech. This is more difficult when hearing a language in which one is a beginner and where the speech output serves as a model to mimic. Of course, there are several other challenges, such as designing for long-term interactions, reducing cost to make robots more commercially available, and getting teachers and users who may not have a technical background competent and comfortable in their operation. However, these are issues that are present in other robot-assisted learning areas, and indeed, in social robotics and HRI more generally.