Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> Cross-correlation and most other longitudinal analyses assume that the association between 2 variables is stationary. Thus, a sample of occasions of measurement is expected to be representative of the association between variables regardless of the time of onset or number of occasions in the sample. The authors propose a method to analyze the association between 2 variables when the assumption of stationarity may not be warranted. The method results in estimates of both the strength of peak association and the time lag when the peak association occurred for a range of starting values of elapsed time from the beginning of an experiment. <s> BIB001 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> Mirror symmetry between individuals is a behavioral phenomenon that is commonly observed in conversation: individuals tend to mimic each other’s postures and gestures as a part of a shared dialog. The present work studies the process of symmetry building and symmetry breaking in the movements of pairs of individuals while imitating each others’ movements in dance. Spatial and temporal symmetries are found in the overall velocities from the results of full body motion tracking. This work was supported in part by NIA grant R29 AG14983-01 and a grant from the University of Notre Dame. We wish to thank Elizabeth Marsh who assisted in gathering these data. Corresponding author: Steven M. Boker, Department of Psychology, The University of Notre Dame, Notre Dame Indiana 46556, USA; sboker@nd.edu; http://www.nd.edu/ sboker. SYMMETRY IN SYNCHRONIZED MOVEMENT 2 <s> BIB002 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> Synchrony is a pervasive concept relevant to diverse domains in physics, biology, and the social sciences. This article reviews some of the evidence both from natural and human systems. Our main focus is on the role of synchrony in psychotherapy research. Its association with empathy, rapport and the therapeutic relationship has been pointed out repeatedly, yet close evaluation of empirical studies suggests that the evidence remains inconclusive. Particularly in naturalistic studies, research employing synchrony as a specific measure is still lacking. We suggest a new approach to studying synchrony in psychotherapies under field conditions: Image-differencing is a video-based algorithm quantifying the amount of movement in pre-selected regions. Our methodology can be compared to the approach from the seminal work of Condon & Ogston (1966). The statistical analysis we employ detects synchrony on a global level, irrespective of the specific body parts moving. Synchrony thus defined is considered a general measure of coordination of interacting individuals. Exemplary data show the feasibility of this approach and its potential contribution to psychotherapy research. <s> BIB003 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> Conversational interactions are cooperatively constructed activities in which participants negotiate their entrances, turns and alignments with other speakers, oftentimes with an underlying long-term objective of obtaining some agreement. Obtaining a final and morally binding accord in a conversational interaction is of importance in a great variety of contexts, particularly in psychotherapeutic interactions, in contractual negotiations or in educational contexts. Various prosodic and gestural elements in a conversational interaction can be interpreted as signals of a speaker's agreement and they are probably of importance in the emergence of an accord in a conversational exchange. In this paper, we survey the social and psychological context of agreement seeking, as well as the existing literature on the visual and prosodic measurement of agreement in conversational settings. <s> BIB004 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> This study examined the influence of sex, social dominance, and context on motion-tracked head movements during dyadic conversations. Windowed cross-correlation analyses found high peak correlation between conversants' head movements over short (≈2-s) intervals and a high degree of nonstationarity. Nonstationarity in head movements was found to be positively related to the number of men in a conversation. Surrogate data analysis offsetting the conversants' time series by a large lag was unable to reject the null hypothesis that the observed high peak correlations were unrelated to short-term coordination between conversants. One way that high peak correlations could be observed when 2 time series are offset by a large time lag is for each time series to exhibit self-similarity over a range of scales. Multifractal analysis found small-scale fluctuations to be persistent, τ(q) 0.5. These results are consistent with a view that symmetry is formed between conversants over short intervals and that this symmetry is broken at longer, irregular intervals. <s> BIB005 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> Nonverbal synchrony is an important and natural element in human-human interaction. It can also play various roles in human-computer interaction. In particular this is the case in the interaction between humans and the virtual humans that inhabit our cyber worlds. Virtual humans need to adapt their behavior to the behavior of their human interaction partners in order to maintain a natural and continuous interaction synchronization. This paper surveys approaches to modeling synchronization and applications where this modeling is important. Apart from presenting this framework, we also present a quantitative method for measuring the level of nonverbal synchrony in an interaction and observations on future research that allows embedding such methods in models of interaction behavior of virtual humans. <s> BIB006 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> The decisive role of interpersonal coordination in social interactions is well established. As human beings, we are "experts" in decoding and producing social signals, learning since our birth. Though, equipping a machine with the same abilities represents a great challenge to design socially-accepted interfaces. Assessing automatically this coordination requires selecting the most relevant features. In this paper, various audio and visual features are investigated to characterize interactional synchrony in a cooperative task. A hierarchical representation of the similarity between extracted features is proposed. The properties of time-domain (correlation coefficients) and frequency-domain (coherence) synchrony measures are confronted. The presented results support the evidence that synchrony is a multimodal phenomenon. Moreover, it can be assessed both on local (short-length windows of interaction) or global scales (entire interaction). <s> BIB007 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> Mimicry occurs in conversations both when people agree with each other and when they do not. However, it has been reported that there is more mimicry when people agree than when they disagree: when people want to express shared opinions and attitudes, they do so by displaying behavior that is similar to their interlocutors' behavior. In a conversation, mimicry occurs in order to gain acceptance from an interaction partner by conforming to that person's attitudes, opinions, and behavior. In this paper we describe how visual behavioral information expressed between two interlocutors can be used to detect and identify visual mimicry. We extract and encode visual features that are expected to represent mimicry in a useful way. In order to show that mimicry has indeed occurred, we calculate correlations between visual features extracted from the interactants and compare these against each other and against a baseline. We show that it is possible to visualize the occurrence of visual mimicry during the progress of a conversation which allows us to research in which situations and to what extent mimicry occurs. <s> BIB008 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Correlation <s> During face-to-face interpersonal interaction, people have a tendency to mimic each other. People not only mimic postures, mannerisms, moods or emotions, but they also mimic several speech-related behaviors. In this paper we describe how visual and vocal behavioral information expressed between two interlocutors can be used to detect and identify visual and vocal mimicry. We investigate expressions of mimicry and aim to learn more about in which situation and to what extent mimicry occurs. The observable effects of mimicry can be explored by representing and recognizing mimicry using visual and vocal features. In order to automatically analyze how to extract and integrate this behavioral information into a multimodal mimicry detection framework for improving affective computing, this paper addresses the main challenge: mimicry representation in terms of optimal behavioral feature extraction and automatic integration in both audio and video modalities. <s> BIB009
Correlation is certainly the most commonly used method to evaluate interactional movement synchrony. After extracting the movement time series of the interactional partners, a time-lagged cross correlation is applied between the two time series using short windows of interaction. Several studies also use a peak picking algorithm to estimate the time lag of the predictive association between two time series (i.e., the peak cross correlation that is closest to a lag of zero) BIB005 , BIB001 ,  . A critical question is the choice of the length of the windows of interaction. In the studies reviewed, the length of the window varies from 1 s to 10 min with time lags of 0 to 5 s. Boker et al. raise the question of the time series stationarity BIB001 . They compare the cross correlation between 1) the movements of two dancers synchronized with each other and the rhythm of the music and 2) the head movements of two people conversing. In case 1, there is a stable pattern of synchronization during the entire interaction; the data follow the assumptions for a stationary process and the cross correlation calculated on the whole interaction is high. In case 2, there might be a "high degree of association on short scales, but due to nonstationarity, overall there might be only low values of correlation." Thus, a weak correlation between time series could indicate either little coordination in the conversation or nonstationary short-term coordination. An additional issue is linked to the representation or metaparameters extracted from the cross-correlation coefficients. A color-coded correlation map is the most common way to represent cross-correlation coefficients  , BIB005 , BIB001 , BIB002 , BIB003 , BIB004 , BIB006 , BIB008 , BIB009 . Time is represented on one axis, and the different time lags are represented on the other. The correlation strength is represented by different color shades. Correlation maps have the advantage of showing a global snapshot of an interaction. Sequences of high synchrony are easy to identify, and the difference between two dyads can be grasped immediately. When the time lag between partners is measured using a peak-picking algorithm, researchers usually plot the evolution of the time lag over time BIB005 , BIB001 . Such plots reveal leading-following relationships between the partners and any dominance traits. These representations are particularly useful when only one signal is being studied. When partners are characterized by several features, the relationship must be represented across the different features. Delaherche and Chetouani proposed using dendrograms to characterize the hierarchy in feature similarity BIB007 : A clustering algorithm was applied to understand how local synchrony was established across all features and to represent their similarity hierarchy. Dendrograms are tree diagrams illustrating the hierarchical relationship between data, often used to represent taxonomy or a hierarchical clustering in biology. U-shaped lines connect features according to their similarity. The height of each U-shaped line represents the distance between the two connected features. The construction of a dendrogram relies on a similarity symmetric matrix containing the distance between every possible pair of features. Dendrograms have two main advantages over the previously described techniques: They characterize the coordination across more than Apart from the representation of synchrony, there is also a need to aggregate the measures from the cross-correlation matrices in synthetic parameters to quantitatively compare different dyads or study the relation between the existence of synchrony and an outcome variable (e.g., smoothness of interaction). Two metaparameters are traditionally assessed to characterize synchrony between partners. First is the degree of synchrony, the percentage of synchronous sequences. When a peak-picking algorithm is applied, the mean and variance of the correlation coefficient's peak value indicate the strength and the variability, respectively, of the coordination during the interaction. Second, the orientation of synchrony indicates who is driving the interaction. The orientation is measured using the time lag between synchronous windows of the video BIB005 , BIB001 . A positive lag between partner 1's features and partner 2's features indicates partner 1 is leading the interaction, while a negative lag indicates partner 2 is leading the interaction. A zero lag between each partner's features indicates mutual synchrony.