Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Open Questions <s> Abstract Sound films of social interaction are analysed and some detailed descriptions of the interrelations of the movements of speakers ans listeners are given. These descriptions provide further examples of ‘interactional synchrony’, first described by Condon and Ogston (1966). It is found that the flow of movement in the listener may be rhythmically coordinated with the speech and movements of the speaker. It is also shown how the way in which individuals may be in synchrony with one another can vary, and that these variations are related to their respective roles in the interaction. Some of the implications of the phenomenon of interactional synchrony are discussed. <s> BIB001 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Open Questions <s> In this review we examine empirical and theoretical work in three eras—infancy, toddlerhood, and early childhood—and for each era describe the structure of dyadic synchrony in interactions involving children and their caregivers, as well as offer speculation about its developmental function for the child. We review divergent literatures dealing with synchrony-related constructs which, together, suggest that although the structure and function of synchrony change throughout the course of early development, the ability to achieve synchrony may represent a crucial developmental achievement for significant dyadic relationships, one that facilitates social, emotional, and cognitive growth for the child. <s> BIB002 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Open Questions <s> Precursors of inferential capacities concerning self- and other- understanding may be found in the basic experience of social contingency and emotional sharing. The emergence of a sense of self- and other-agency receives special attention here, as a foundation for self-understanding. We propose that synchrony, an amodal parameter of contingent self-other relationships, should be especially involved in the development of a sense of agency. To explore this framework, we have manipulated synchrony in various ways, either by delaying mother’s response to infant’s behaviour, disorganizing mother’s internal synchrony between face and voice, freezing the partner in a still attitude, or on the contrary maximizing synchrony through imitation. We report results obtained with healthy and clinical populations that are supposed to be at the beginning of basic experiences concerning the ownership of their actions: infants of 2 months and 6 months, low-functioning children with autism and MA matched young children with Down Syndrome. Our results support the idea of a two-step process linking understanding of self to understanding of other and leading on to form the concept of human beings as universally contingent entities. <s> BIB003 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Open Questions <s> This paper describes recent work on the automatic extraction of visual and audio parameters relating to the detection of synchrony in discourse, and to the modelling of active listening for advanced speech technology. It reports findings based on image processing that reliably identify the strong entrainment between members of a group conversation, and describes techniques for the extraction and analysis of such information. <s> BIB004 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Open Questions <s> Movement synchrony has been theoretically linked to the emergence of a social unit. To empirically investigate whether similar movement rhythms are an antecedent of perceived entitativity, movement rhythms were experimentally manipulated in four studies. Using this novel approach, stick figures waving in synchrony were found to be rated higher on entitativity than stick figures waving in different rhythms (Study 1), and this effect was extended to interactional synchrony, where different movements are performed in the same rhythm (Study 2). Objective differences in movement rhythms are linearly related to ratings of perceived entitativity, and this relationship is partially mediated by the subjectively perceived similarity of movement rhythms (Study 3). These results also held for entitativity judgments for videotaped individuals waving rhythmically (Study 4). These results support the hypothesis that movement rhythms are an important source of information which observers use to infer the extent to which individuals are a social unit. <s> BIB005 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Open Questions <s> Understanding the ability to coordinate with a partner constitutes a great challenge in social signal processing and social robotics. In this paper, we designed a child-adult imitation task to investigate how automatically computable cues on turn-taking and movements can give insight into high-level perception of coordination. First we collected a human questionnaire to evaluate the perceived coordination of the dyads. Then, we extracted automatically computable cues and information on dialog acts from the video clips. The automatic cues characterized speech and gestural turn-takings and coordinated movements of the dyad. We finally confronted human scores with automatic cues to search which cues could be informative on the perception of coordination during the task. We found that the adult adjusted his behavior according to the child need and that a disruption of the gestural turn-taking rhythm was badly perceived by the judges. We also found, that judges rated negatively the dyads that talked more as speech intervenes when the child had difficulties to imitate. Finally, coherence measures between the partners' movement features seemed more adequate than correlation to characterize their coordination. <s> BIB006 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Open Questions <s> In this paper we introduce a multi-modal database for the analysis of human interaction, in particular mimicry, and elaborate on the theoretical hypotheses of the relationship between the occurrence of mimicry and human affect. The recorded experiments are designed to explore this relationship. The corpus is recorded with 18 synchronised audio and video sensors, and is annotated for many different phenomena, including dialogue acts, turn-taking, affect, head gestures, hand gestures, body movement and facial expression. Recordings were made of two experiments: a discussion on a political topic, and a role-playing game. 40 participants were recruited, all of whom selfreported their felt experiences. The corpus will be made available to the scientific community. <s> BIB007
Several questions regarding the dimension and perception of synchrony remain to be explored. These questions are fundamental to the development of an automatic model to assess synchrony. A first question relates to the time scale of synchrony: second, minute, all interaction. Is it appropriate to break behavior into small units? Is it possible to operationalize synchrony and/or measure occurrences of synchrony? A second question concerns the dimension of synchrony: Is synchrony an all-or-none condition (synchronous versus nonsynchronous)? Is synchrony a continuous or a discrete notion? Or can dyadic interaction can approach or move away from synchrony BIB002 ? Six-month-olds can detect when a modality is synchronous and another modality is not BIB003 . Thus, with regard to the unimodal versus intermodal question, synchrony appears to be experienced as an all-ornone condition. Out of the question of modality and when addressing naturalistic interaction, much of the current information suggests that synchrony is a continuous notion. Various sources indicate that synchrony varies over the course of interaction, being stronger at the beginning and the ending of an exchange BIB001 or at moments of particular engagement BIB004 . Feldman operationalizes synchrony as the degree to which the partners change their affective behavior in reference to one another and obtains a number ranging between 0 and 1  . When addressing the matter of movement synchrony and its relation to perceived entitativity, Lakens observed that objective differences in movement rhythms were linearly related to ratings of perceived entitativity BIB005 . Finally, a recent study showed that the perception of coordination was more unanimous when coordination was very high or very low. However, judges were not reliable when judging dyads with "medium" coordination BIB006 . The question of the corpus is also crucial. Until Sun et al.'s BIB007 recent contribution of their mimicry database, no publicly available annotated corpus was dedicated to the detection of synchrony. We can hope that this effort will benefit the field, aiding engineers in their work to develop new algorithms, skipping the data collection and annotation phases.