A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> Advancement in robotic research enables robot can assist human in many way. However, few researches have been done on applying on education. This paper reports field trials of using robot in an English learning classroom. In our experiment, the robot plays as a partner of a teacher. Five models of collaboration between teachers and robot are proposed. Teachers design the course flow in advance based on the learning content and these five models. Formative evaluations of the experiments are presented to show the impact of adopting robot in the classroom. <s> BIB001 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> The scenarios and design process for a childcare robot, PaPeRo, are presented. The design process mainly consists of production and evaluations. The production is composed of three observations, namely, one in the user field, one with respect to the world view, and one about the assumed users and results for the basic role of the robot, its personality, and classification of users. Our investigation of available technologies deepened our understanding of the robot performance and characteristics. Preproduction, which consisted of a scenario design, prototyping, and a short play, made the scenario details more sophisticated. A six-month demonstration at the 2005 Aichi Expo started with seven different scenarios, which were improved using additional recognition words, more attractive robot personalities, and additional scenarios through on-site observations and discussion with staff members there. We conducted an extensive study on the robot personalities and recognizable words. Our study will play an important role in the scenario design process. <s> BIB002 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> We are developing a storytelling robot which learns by children's teaching. The robot is used on learning of the second language. It can reduce the negative emotion due to the anxiety of speaking foreign language, which thus motivates learning and enhances the speaking and listening abilities. Presently, our system is still at the initial stage of research. Therefore, we only interviewed and experimented on some of the children now. These results will be used to improve our system. <s> BIB003 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> voice recognition technology with a tangible learning companion/robot which our research team developed to enhance English learning conversation for beginners. In this study, we collected the users’ needs analysis of using tangible learning companion first then did a pilot test to analyze its usability. The results show that the learners improved their pronunciation to achieve right voice recognition through practicing with the learning companion and the tangible learning companion can promote the less proficient learners’confidence. Moreover, we also noticed that learning with the tangible learning companion could increase the pleasure of learning speaking English and tactile interactions between learners. <s> BIB004 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> Teaching is inherently a social interaction between teacher and student. Despite this knowledge, many educational tools, such as vocabulary training programs, still model the interaction in a tutoring scenario as unidirectional knowledge transfer rather than a social dialog. Therefore, ongoing research aims to develop virtual agents as more appropriate media in education. Virtual agents can induce the perception of a life-like social interaction partner that communicates through natural modalities such as speech, gestures and emotional expressions. This effect can be additionally enhanced with a physical robotic embodiment. This paper presents the development of social supportive behaviors for a robotic tutor to be used in a language learning application. The effect of these behaviors on the learning performance of students was evaluated. The results support that employing social supportive behavior increases learning efficiency of students. <s> BIB005 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> As robot technologies develop, many researchers have tried to use robots to support education. Studies have shown that robots can help students develop problem-solving abilities and learn computer programming, mathematics, and science. However, few studies discuss the use of robots to facilitate the teaching of second languages. We discuss whether language teaching needs robot support, whether robots present an appropriate medium for language instruction, and what challenges must be overcome. In order to explore the possibility of using robots as an instructional tool for teaching a second language, this paper reviews prior studies concerning educational robots and analyzes the characteristics of robots and instructional media. Furthermore, we report on the design and testing of five instruction scenarios for teaching second language. Based on our empirical experience, we provide suggestions for future research directions in the realm of robots for language education. Keyword <s> BIB006 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> There is an on-going study which aims to assist in teaching Sign Language (SL) to hearing impaired children by means of non-verbal communication and imitation based interaction games between a humanoid robot and the child. In this study, the robot will be able to express a word in SL among a set of chosen words using hand movements, body and face gestures. Having comprehended the word, the child will give relevant feedback to the robot. In the current study, we propose an interactive story telling game between a NAO H25 humanoid robot and preschool children based on Turkish Sign Language (TSL). Since most of the children do not know how to read and write, and they are not familiar with sign language, we prepared a short story including specially selected words which is performed by the robot verbally and with sign language as well. The children are expected to give feedback to the robot with matching colour flashcards when it implements a word in sign language. The robotic event covered 106 preschool children. The aim is to evaluate the children's sign language learning ability from a robot, and comparison of these results with the results of video based studies. <s> BIB007 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> Various studies on robot assisted learning have proven them to be effective tools for language education because robots are remarkably successful in eliminating the affective filter. This study demonstrates ROBOSEM with class content for sustaining long-term interaction with students in English classes of elementary schools as a teaching assistant, not a teacher. We close with some practical considerations for the continuous process of improving technology and its diffusion throughout society, such as in elementary schools, through interviews with developers and observations from field trials. <s> BIB008 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> Early school years are an important period to lay out the foundation for learning a second language. In addition to mastering the basic language skills and keeping the learning process fun, promoting a lifelong learning habit should also be emphasised. Motivating elementary school students to learn English and avoiding misconceptions associated with the learning process are very important issues. This study attempted to explore the integration of book, computer and robot to create a novel and joyful English learning environment. Books are the most common learning media. If we can design a special mechanism linking books with digital learning content and a robot, students will be able to obtain supplementary learning content including motions performed by the robot to enhance learning outcomes. To implement this idea, a new learning system was designed and developed. Learning activities included vocabulary, single sentence read-along, full article read-along, conversation, sing-along and dancing. Cloze tests were implemented and the feasibility of the learning system was evaluated. Data collected from field observation and interviews were analysed. The results suggest that this system did enrich students' learning experience and enhance their motivation and engagement. <s> BIB009 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> The work presented in this paper was carried out within an on-going project which aims to assist in Sign language tutoring to children with communication problems by means of interaction games with a humanoid robot. In this project, the robot is able to express words in the Turkish Sign language (TSL) and American Sign Language (ASL), and having comprehended the word, the child is motivated to give relevant feedback to the robot with colored flashcards, signs and sound. Within the multi-modal turn-taking games, child both learns the semantic and the realization of the words, besides have chance to test this information through imitation based interaction games. The performance of the children is also verified with flash card based activities as a part of the game, as a feedback for the teacher/parent. Child's performance is recognized and evaluated by the robot through an RGB-D camera based system. This paper summarizes a longitudinal exploratory study from a classroom work, where children were taught 6 gestures including signs from ASL and TSL with the assistance of a Nao H25 humanoid robot. The performances of the children and robot are presented and discussed in terms of children's subjective and objective evaluations. <s> BIB010 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> This paper investigates the role of interaction and communication kinesics in human–robot interaction. This study is part of a novel research project on sign language (SL) tutoring through interaction games with humanoid robots. The main goal is to motivate the children with communication problems to understand and imitate the signs implemented by the robot using basic upper torso gestures and sound. We present an empirical and exploratory study investigating the effect of basic nonverbal gestures consisting of hand movements, body and face gestures expressed by a humanoid robot, and having comprehended the word, the participants will give relevant feedback in SL. This way the participant is both a passive observer and an active imitator throughout the learning process in different phases of the game. A five-fingered R3 robot platform and a three-fingered Nao H-25 robot are employed within the games. Vision-, sound-, touch- and motion-based cues are used for multimodal communication between the robot, child and therapist/parent within the study. This paper presents the preliminary results of the proposed game tested with adult participants. The aim is to evaluate the SL learning ability of participants from a robot, and compare different robot platforms within this setup. <s> BIB011 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> This paper presents a design for a cutting-edge English program in which elementary school learners of English as a foreign language in Taiwan had lively interactions with a teaching assistant robot. Three dimensions involved in the design included (1) a pleasant and interactive classroom environment as the learning context, (2) a teaching assistant robot designed and built by the researchers as instructional technology (instructional tool/medium), and (3) the need for improved motivation and learning outcomes and positive learning experiences of the students as the core research problem. This Human–Robot Interaction (HRI) led to better pedagogical effects on learning and teaching by (1) employing a vivid, enjoyable teaching approach; (2) adopting practical, interesting learning materials; and (3) creating a natural, enjoyable learning context. Both quantitative and qualitative findings of this study indicate that the students' English learning experiences were enhanced, as were their motivation and learn... <s> BIB012 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> Though substantial research has been dedicated towards using technology to improve education, no current methods are as effective as one-on-one tutoring. A critical, though relatively understudied, aspect of effective tutoring is modulating the student's affective state throughout the tutoring session in order to maximize long-term learning gains. We developed an integrated experimental paradigm in which children play a second-language learning game on a tablet, in collaboration with a fully autonomous social robotic learning companion. As part of the system, we measured children's valence and engagement via an automatic facial expression analysis system. These signals were combined into a reward signal that fed into the robot's affective reinforcement learning algorithm. Over several sessions, the robot played the game and personalized its motivational strategies (using verbal and non-verbal actions) to each student. We evaluated this system with 34 children in preschool classrooms for a duration of two months. We saw that (1) children learned new words from the repeated tutoring sessions, (2) the affective policy personalized to students over the duration of the study, and (3) students who interacted with a robot that personalized its affective feedback strategy showed a significant increase in valence, as compared to students who interacted with a non-personalizing robot. This integrated system of tablet-based educational content, affective sensing, affective policy learning, and an autonomous social robot holds great promise for a more comprehensive approach to personalized tutoring. <s> BIB013 </s> A Survey of Robot-Assisted Language Learning (RALL) <s> Non-verbal Cues <s> Teaching with robots is a developing field, wherein one major challenge is creating lesson plans to be taught by a robot. We introduce a novel strategy for generating lesson material, in which we draw upon an existing corpus of electronic lesson material and develop a mapping from the original material to the robot lesson, thereby greatly reducing the time and effort required to create robot lessons. We present a system, KubiLingo, in which we implement content mapping for language lessons. With permission, we use Duolingo as the source of our content. In a study with 24 users, we demonstrate that user performance improves by a statistically similar amount with a robot lesson as with Duolingo lesson. We find that KubiLingo is more distracting and less likeable than Duolingo, indicating the need for improvements to the robot’s design. <s> BIB014
Non-verbal cues used in RALL include head nods and happy faces BIB005 and smiling and thumbs up gesturing BIB012 corresponding to correct answers or success; head shakes and sad faces BIB005 and LED color changes BIB010 BIB007 corresponding to incorrect answers; eye gaze for attention guiding, turntaking, and emotional display BIB013 BIB008 BIB005 BIB001 ; sound effects and music for emotional emphasis BIB006 BIB011 BIB003 BIB012 BIB001 or to increase energy BIB014 ; color and light changes to show emotion BIB011 BIB007 BIB005 or voice recognition BIB002 ; and body movement and motion to show animation BIB003 BIB005 BIB012 , express understanding BIB003 , or emphasize spoken material BIB009 BIB012 . These cues may be executed in response to stimuli or used to create idling motions to make the robot seem more lifelike BIB014 BIB005 . Overall, there is a tendency to use facial expressions and body movements to convey action, intention, and emotion. This is because "human perception of a robot's emotional expressions plays a crucial role in human robot interaction" [66:40] . It can also affect students' emotions. When measured, Gordon et al. found that children's affective states change in response to nonverbal cues presented by a social robot BIB013 . Specifically, their emotion was positively affected after the robot performed such behaviors such as leaning in, nodding, and using positive utterances, and negatively affected after expressions of sadness BIB013 . Students' emotions, both pre-existing and created through interaction with the robot, can influence learning and the learning environment BIB004 .