Social robots for education: A review <s> Technical challenges of building robot tutors <s> Teaching is inherently a social interaction between teacher and student. Despite this knowledge, many educational tools, such as vocabulary training programs, still model the interaction in a tutoring scenario as unidirectional knowledge transfer rather than a social dialog. Therefore, ongoing research aims to develop virtual agents as more appropriate media in education. Virtual agents can induce the perception of a life-like social interaction partner that communicates through natural modalities such as speech, gestures and emotional expressions. This effect can be additionally enhanced with a physical robotic embodiment. This paper presents the development of social supportive behaviors for a robotic tutor to be used in a language learning application. The effect of these behaviors on the learning performance of students was evaluated. The results support that employing social supportive behavior increases learning efficiency of students. <s> BIB001 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Based on a ‘learning by playing' concept, a basic arithmetic learning task was extended with an engaging game to achieve long-term educational interaction for children. Personalization was added to this learning task, to further support the child's motivation and success in learning. In an experiment, twenty children (aged 9-10) interacted three times, spread over days, with a robot using the combined imitation and arithmetic game to test this support. Two versions of the robot were implemented. In one implementation the complexity of the arithmetic progressed towards a predefined group target. In the other version the assignments increased in complexity until a personal end level was reached. A subsequent free-choice period showed that children's motivation to play (and learn) was high, particularly when the game progressed to a personal target. Furthermore results show that most children in the last condition reach higher levels compared to the predefined group level. <s> BIB002 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> In the development of companion robots capable of any-depth, long-term interaction, social scenarios enable exploration of the robot's capacity to engage a human interactant. These scenarios are typically constrained to structured task-based interactions, to enable the quantification of results for the comparison of differing experimental conditions. This paper introduces a hardware setup to facilitate and mediate human-robot social interaction, simplifying the robot control task while enabling an equalised degree of environmental manipulation for the human and robot, but without implicitly imposing an a priori interaction structure. <s> BIB003 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Embodied agents hold great promise as educational assistants, exercise coaches, and team members in collaborative work. These roles require agents to closely monitor the behavioral, emotional, and mental states of their users and provide appropriate, effective responses. Educational agents, for example, will have to monitor student attention and seek to improve it when student engagement decreases. In this paper, we draw on techniques from brain-computer interfaces (BCI) and knowledge from educational psychology to design adaptive agents that monitor student attention in real time using measurements from electroencephalography (EEG) and recapture diminishing attention levels using verbal and nonverbal cues. An experimental evaluation of our approach showed that an adaptive robotic agent employing behavioral techniques to regain attention during drops in engagement improved student recall abilities 43% over the baseline regardless of student gender and significantly improved female motivation and rapport. Our findings offer guidelines for developing effective adaptive agents, particularly for educational settings. <s> BIB004 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Gestures support and enrich speech across various forms of communication. Effective use of gestures by speakers improves not only the listeners’ comprehension of the spoken material but also their perceptions of the speaker. How might robots use gestures to improve human-robot interaction? What gestures are most effective in achieving such improvements? This paper seeks answers to these questions by presenting a model of human gestures and a system-level evaluation of how robots might selectively use different types of gestures to improve interaction outcomes, such as user task performance and perceptions of the robot, in a narrative performance scenario. The results show that robot deictic gestures consistently predict users’ information recall, that all types of gestures affect user perceptions of the robot’s performance as a narrator, and that males and females show significant differences in their responses to robot gestures. These results have strong implications for designing effective robot gestures that improve human-robot interaction. <s> BIB005 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> In social interaction, people draw on a large repertoire of social acts tailoring their use of these acts to meet the demands of the social situation and to achieve the goals of the interaction. This paper presents an approach to creating such a repertoire of social acts for robots and enabling designers to specify the social situation to which robots may adapt their behaviors. Drawing on principles of Activity Theory and social-scientific findings on human social behavior, this paper introduces an implementation of this approach---the Robot Behavior Toolkit---and two studies that use a limited, proof-of-concept repertoire of specifications for gaze cues to demonstrate the feasibility of this approach for controlling robot gaze behavior. The first study assessed the feasibility of the use of this repertoire, comparing it to alternative, baseline repertoires in two human-robot interaction tasks, and found that it enabled the robot to more effectively support the interaction goals. The second study investigated the feasibility of the robot adapting its use of the repertoire to a social situation by comparing different goal specifications in two human-robot interaction tasks. The results showed that these specifications enabled the robot to achieve some of its task and communicative goals, although participant gender strongly affected whether the robot elicited these interaction outcomes. <s> BIB006 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Objective Assess the effects of personalised robot behaviours on the enjoyment and motivation of children (8–12) with diabetes, and on their acquisition of health knowledge, in educational play. Methods Children (N = 5) played diabetes quizzes against a personal or neutral robot on three occasions: once at the clinic, twice at home. The personal robot asked them about their names, sports and favourite colours, referred to these data during the interaction, and engaged in small talk. Fun, motivation and diabetes knowledge was measured. Child–robot interaction was observed. Results Children said the robot and quiz were fun, but this appreciation declined over time. With the personal robot, the children looked more at the robot and spoke more. The children mimicked the robot. Finally, an increase in knowledge about diabetes was observed. Conclusion The study provides strong indication for how a personal robot can help children to improve health literacy in an enjoyable way. Children mimic the robot. When the robot is personal, they follow suit. Our results are positive and establish a good foundation for further development and testing in a larger study. Practice implications Using a robot in health care could contribute to self-management in children and help them to cope with their illness. <s> BIB007 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> In education research, there is a widely-cited result called “Bloom’s two sigma” that characterizes the differences in learning outcomes between students who receive one-on-one tutoring and those who receive traditional classroom instruction [1]. Tutored students scored in the 95th percentile, or two sigmas above the mean, on average, compared to students who received traditional classroom instruction. In human-robot interaction research, however, there is relatively little work exploring the potential benefits of personalizing a robot’s actions to an individual’s strengths and weaknesses. In this study, participants solved grid-based logic puzzles with the help of a personalized or non-personalized robot tutor. Participants’ puzzle solving times were compared between two non-personalized control conditions and two personalized conditions (n=80). Although the robot’s personalizations were less sophisticated than what a human tutor can do, we still witnessed a “one-sigma” improvement (68th percentile) in post-tests between treatment and control groups. We present these results as evidence that even relatively simple personalizations can yield significant benefits in educational or assistive human-robot interactions. Categories and Subject Descriptors I.2.9 [Artificial Intelligence]: Robotics; J.4 [Computer Applications]: Social And Behavioral Sciences—Psychology General Terms Experimentation <s> BIB008 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Changes in type of interaction (e.g., individual vs. group interactions) can potentially impact data-driven models developed for social robots. In this paper, we provide a first investigation in the effects of changing group size in datadriven models for HRI, by analyzing how a model trained on data collected from participants interacting individually performs in test data collected from group interactions, and \textit{vice-versa. Another model combining data from both individual and group interactions is also investigated. We perform these experimentsin the context of predicting disengagement behaviors in children interacting with two social robots. Our results show that a model trained with group data generalizes better to individual participants than the other way around. The mixed model seems a good compromise, but it does not achieve the performance levels of the models trained for a specific type of interaction. <s> BIB009 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI. <s> BIB010 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Social robots are finding increasing application in the domain of education, particularly for children, to support and augment learning opportunities. With an implicit assumption that social and adaptive behaviour is desirable, it is therefore of interest to determine precisely how these aspects of behaviour may be exploited in robots to support children in their learning. In this paper, we explore this issue by evaluating the effect of a social robot tutoring strategy with children learning about prime numbers. It is shown that the tutoring strategy itself leads to improvement, but that the presence of a robot employing this strategy amplifies this effect, resulting in significant learning. However, it was also found that children interacting with a robot using social and adaptive behaviours in addition to the teaching strategy did not learn a significant amount. These results indicate that while the presence of a physical robot leads to improved learning, caution is required when applying social behaviour to a robot in a tutoring context. Categories and Subject Descriptors H.1.2 [Models and Principles]: User/Machine Systems <s> BIB011 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Effective tutoring requires personalization of the interaction to each student. Continuous and efficient assessment of the student's skills are a prerequisite for such personalization. We developed a Bayesian active-learning algorithm that continuously and efficiently assesses a child's word-reading skills and implemented it in a social robot. We then developed an integrated experimental paradigm in which a child plays a novel story-creation tablet game with the robot. The robot is portrayed as a younger peer who wishes to learn to read, framing the assessment of the child's word-reading skills as well as empowering the child. We show that our algorithm results in an accurate representation of the child's word-reading skills for a large age range, 4-8 year old children, and large initial reading skill range. We also show that employing child-specific assessment-based tutoring results in an age- and initial reading skill-independent learning, compared to random tutoring. Finally, our integrated system enables us to show that implementing the same learning algorithm on the robot's reading skills results in knowledge that is comparable to what the child thinks the robot has learned. The child's perception of the robot's knowledge is age-dependent and may facilitate an indirect assessment of the development of theory-of-mind. <s> BIB012 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Measuring "how much the human is in the interaction" -- the level of engagement -- is instrumental in building effective interactive robots. Engagement, however, is a complex, multi-faceted cognitive mechanism that is only indirectly observable. This article formalizes "with-me-ness" as one of such indirect measures. With-me-ness, a concept borrowed from the field of Computer-Supported Collaborative Learning, measures in a well-defined way to what extent the human is "with" the robot over the course of an interactive task. As such, it is a meaningful precursor of engagement. We expose in this paper the full methodology, from real-time estimation of the human's focus of attention (relying on a novel, open-source, vision-based head pose estimator), to on-line computation of with-me-ness. We report as well on the experimental validation of this approach, using a naturalistic setup involving children during a complex robot-teaching task. <s> BIB013 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Though substantial research has been dedicated towards using technology to improve education, no current methods are as effective as one-on-one tutoring. A critical, though relatively understudied, aspect of effective tutoring is modulating the student's affective state throughout the tutoring session in order to maximize long-term learning gains. We developed an integrated experimental paradigm in which children play a second-language learning game on a tablet, in collaboration with a fully autonomous social robotic learning companion. As part of the system, we measured children's valence and engagement via an automatic facial expression analysis system. These signals were combined into a reward signal that fed into the robot's affective reinforcement learning algorithm. Over several sessions, the robot played the game and personalized its motivational strategies (using verbal and non-verbal actions) to each student. We evaluated this system with 34 children in preschool classrooms for a duration of two months. We saw that (1) children learned new words from the repeated tutoring sessions, (2) the affective policy personalized to students over the duration of the study, and (3) students who interacted with a robot that personalized its affective feedback strategy showed a significant increase in valence, as compared to students who interacted with a non-personalizing robot. This integrated system of tablet-based educational content, affective sensing, affective policy learning, and an autonomous social robot holds great promise for a more comprehensive approach to personalized tutoring. <s> BIB014 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> An increasing number of human-robot interaction (HRI) studies are now taking place in applied settings with children. These interactions often hinge on verbal interaction to effectively achieve their goals. Great advances have been made in adult speech recognition and it is often assumed that these advances will carry over to the HRI domain and to interactions with children. In this paper, we evaluate a number of automatic speech recognition (ASR) engines under a variety of conditions, inspired by real-world social HRI conditions. Using the data collected we demonstrate that there is still much work to be done in ASR for child speech, with interactions relying solely on this modality still out of reach. However, we also make recommendations for child-robot interaction design in order to maximise the capability that does currently exist. <s> BIB015 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> A common practice in education to accommodate the short attention spans of children during learning is to provide them with non-task breaks for cognitive rest. Holding great promise to promote learning, robots can provide these breaks at times personalized to individual children. In this work, we investigate personalized timing strategies for providing breaks to young learners during a robot tutoring interaction. We build an autonomous robot tutoring system that monitors student performance and provides break activities based on a personalized schedule according to performance. We conduct a field study to explore the effects of different strategies for providing breaks during tutoring. By comparing a fixed timing strategy with a reward strategy (break timing personalized to performance gains) and a refocus strategy (break timing personalized to performance drops), we show that the personalized strategies promote learning gains for children more effectively than the fixed strategy. Our results also reveal immediate benefits in enhancing efficiency and accuracy in completing educational problems after personalized breaks, showing the restorative effects of the breaks when administered at the right time. <s> BIB016 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> In this paper, we present an approach to adaptive language tutoring in child-robot interaction. The approach is based on a dynamic probabilistic model that represents the interrelations between the learner's skills, her observed behaviour in tutoring interaction, and the tutoring action taken by the system. Being implemented in a robot language tutor, the model enables the robot tutor to trace the learner's knowledge and to decide which skill to teach next and how to address it in a game-like tutoring interaction. Results of an evaluation study are discussed demonstrating how participants in the adaptive tutoring condition successfully learned foreign language words. CCS Concepts. Computing methodologies → Probabilistic reasoning; Cognitive robotics;. Applied computing → Interactive learning environments;. Human-centered computing → Empirical studies in HCI; <s> BIB017 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> Thinking aloud, while requiring extra mental effort, is a metacognitive technique that helps students navigate through complex problem-solving tasks. Social robots, bearing embodied immediacy that fosters engaging and compliant interactions, are a unique platform to deliver problem-solving support such as thinking aloud to young learners. In this work, we explore the effects of a robot platform and the think-aloud strategy on learning outcomes in the context of a one-on-one tutoring interaction. Results from a 2x2 between-subjects study (n=52) indicate that both the robot platform and use of the think-aloud strategy promoted learning gains for children. In particular, the robot platform effectively enhanced immediate learning gains, measured right after the tutoring session, while the think-aloud strategy improved persistent gains as measured approximately one week after the interaction. Moreover, our results show that a social robot strengthened students» engagement and compliance with the think-aloud support while they performed cognitively demanding tasks. Our work indicates that robots can support metacognitive strategy use to effectively enhance learning and contributes to the growing body of research demonstrating the value of social robots in novel educational settings. <s> BIB018 </s> Social robots for education: A review <s> Technical challenges of building robot tutors <s> This paper describes research aimed at supporting children's reading practices using a robot designed to interact with children as their reading companion. We use a learning by teaching scenario in which the robot has a similar or lower reading level compared to children, and needs help and extra practice to develop its reading skills. The interaction is structured with robot reading to the child and sometimes making mistakes as the robot is considered to be in the learning phase. Child corrects the robot by giving it instant feedbacks. To understand what kind of behavior can be more constructive to the interaction especially in helping the child, we evaluated the effect of a deictic gesture, namely pointing on the child's ability to find reading mistakes made by the robot. We designed three types of mistakes corresponding to different levels of reading mastery. We tested our system in a within-subject experiment with 16 children. We split children into a high and low reading proficiency even-though they were all beginners. For the high reading proficiency group, we observed that pointing gestures were beneficial for recognizing some types of mistakes that the robot made. For the earlier stage group of readers pointing were helping to find mistakes that were raised upon a mismatch between text and illustrations. However, surprisingly, for this same group of children, the deictic gestures were disturbing in recognizing mismatches between text and meaning. <s> BIB019
There are a number of challenges in using technology to support education. Using a social robot adds to this set of challenges because of the robot's presence in the social and physical environment and because of the expectations the robot creates in the user. The social element of the interaction is especially difficult to automate: Although robot tutors can operate autonomously in restricted contexts, fully autonomous social tutoring behavior in unconstrained environments remains elusive. Perceiving the social world is a first step toward being able to act appropriately. Robot tutors should be able to not only correctly interpret the user's responses to the educational content offered but also interpret the rapid and nuanced social cues that indicate task engagement, confusion, and attention. Although automatic speech recognition and social signal processing have improved in recent years, sufficient progress has not been made for all populations. Speech recognition for younger users, for example, is still insufficiently robust for most interactions BIB015 . Instead, alternative input technologies, such as a touch-screen tablets or wearable sensors, are used to read responses from the learner and can be used as a proxy to detect engagement and to track the performance of the student BIB003 BIB016 BIB004 . Robots can also use explicit models of disengagement in a given context BIB009 and strategies, such as activity switching, to sustain engagement over the interaction BIB010 . Computational vision has made great strides in recent years but is still limited when dealing with the range of environments and social expressions typically found in educational and domestic settings. Although advanced sensing technologies for reading gesture, posture, and gaze BIB013 have found their way into tutoring robots, most social robot tutors continue to be limited by the degree to which they can accurately interpret the learner's social behavior. Armed with whatever social signals can be read from the student, the robot must choose an action that advances the long-term goals of the educational program. However, this can often be a difficult choice, even for experienced human instructors. Should the instructor press on and attempt another problem, advance to a more challenging problem, review how to solve the current problem, offer a hint, or even offer a brief break from instruction? There are often conflicting educational theories in human-based instruction, and whether or not these same theories hold when considering robot instructors is an open question. These choices are also present in ITSs, but the explicit agentic nature of robots often introduces additional options and, at times, complications. Choosing an appropriate emotional support strategy based on the affective state of the child (36), assisting with a meta-cognitive learning strategy BIB018 , deciding when to take a break BIB016 , and encouraging appropriate help-seeking behavior (4) have all been shown to increase student learning gains. Combining these actions with appropriate gestures BIB005 , appropriate and congruent gaze behavior BIB006 , expressive behaviors and attention-guiding behaviors BIB001 , and timely nonverbal behaviors (3) also positively affects student recall and learning. However, merely increasing the amount of social behavior for a robot does not lead to increased learning gains: Certain studies have found that social behavior may be distracting BIB011 BIB019 . Instead, the social behavior of the robot must be carefully designed in conjunction with the interaction context and task at hand to enhance the educational interaction. Last, substantial research has focused on personalizing interactions to the specific user. Within the ITS community, computational techniques such as dynamic Bayesian networks, fuzzy decision trees, and hidden Markov models are used to model student knowledge and learning. Similar to on-screen tutoring systems, robot tutors use these same techniques to help tailor the complexity of problems to the capabilities of the student, providing more complex problems only when easier problems have been mastered BIB012 BIB008 BIB017 . In addition to the selection of personalized content, robotic tutoring systems often provide additional personalization to support individual learning styles and interaction preferences. Even straightforward forms of personalization, such as using a child's name or referencing personal details within an educational setting, can enhance user perception of the interaction and are important factors in maintaining engagement within learning interactions BIB002 BIB007 . Other affective personalization strategies have been explored to maintain engagement during a learning interaction by using reinforcement learning to select the robot's affective responses to the behavior of children BIB014 . A field study showed that students who interacted with a robot that simultaneously demonstrated three types of personalization (nonverbal behavior, verbal behavior, and adaptive content progression) showed increased learning gains and sustained engagement when compared with students interacting with a nonpersonalized robot (48) Although progress has been made in constituent technologies of robot tutors-from perception to action selection and production of behaviors that promote learning-the integration of these technologies and balancing their use to elicit prosocial behavior and consistent learning still remain open challenges.