Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> As early as the first day of life, the human neonate moves in precise and sustained segments of movement that are synchronous with the articulated structure of adult speech. These observations suggest a view of development of the infant as a participant at the outset in multiple forms of interactional organization, rather than as an isolate. <s> BIB001 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> Coordination in social interaction means that persons adjust their actions to those of their partners. Common methods for measuring coordination include judgments and behavioral covariation. Sixteen 1-min segments of interaction were chosen (8 high and 8 low in behavioral coordination). In Study 1, 51 people judged the 16 segments, rating each for coordination. Study 2 (N = 17) used different items. Study 3 (N = 22) replicated Study 2 without sound and with a mosaic pattern imposed on the faces. Results indicated judges were reliable, able to distinguish high from low coordination interactions on the basis of 1-min slices for male but not female dyads. Segments judged to be coordinated had partners smiling in synchrony but with complementary patterns of gazing and gesturing. Both measures correlated with conversational satisfaction, but only behavioral coordination predicted attraction. <s> BIB002 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> The effect of mother–infant skin-to-skin contact (kangaroo care, or KC) on self-regulatory processes of premature infants was studied. Seventy-three infants who received KC were compared with 73 infants matched for birth weight, gestational age, medical risk, and family demographics. State organization was measured in 10-s epochs over 4 hr before KC and again at term. No differences between KC infants and controls were found before KC. At term, KC infants showed more mature state distribution and more organized sleep–wake cyclicity. At 3 months, KC infants had higher thresholds to negative emotionality and more efficient arousal modulation while attending to increasingly complex stimuli. At 6 months, longer duration of and shorter latencies to mother–infant shared attention and infant sustained exploration in a toy session were found for KC infants. The results underscore the importance of maternal body contact for infants’ physiological, emotional, and cognitive regulatory capacities. Premature birth deprives the infant not only of the protective and nurturing environment of the mother’s womb but also of the benefits of maternal proximity and contact. Prematurity thus creates a unique phase in the life of the human infant, a period in which portions of the newborn’s prenatal development take place in an extrauterine environment under conditions of maternal separation. Although the effects of prematurity on the physiology and behavior of the developing child have been frequently studied, far less research has examined the associations between intervention applied in the neonatal period and developmental consequences. Most studies on the long-term effects of intervention for premature infants have shown either general gains in infants’ cognitive abilities or a global increase in maternal sensitivity (Als et al., 1994; Bernard & Bee, 1983; Resnick, Eyler, Nelson, Eitzman, & Bucciarelli, 1987; Scarr-Salapatek & Williams, 1973; Solkoff, Yaffe, Weintraub, & Blase, 1969). Such global findings, however, provide limited understanding of the mechanisms that link intervention to specific outcomes. This study examined the effects of mother–infant skin-to-skin contact in the neonatal period on the development of selfregulatory skills in premature infants. Specifically, we examined the effects of such contact on the infant’s capacities to regulate sleep and wake states, organize behavior, regulate negative emotions, modulate arousal, coordinate attention to mother and an object, and sustain effortful exploration of the environment. Skinto-skin contact was expected to modify each of the adverse conditions associated with the prematurity state, that is, maternal separation, stimulus overload caused by routine nursery care, sensory deprivation resulting from limited tactile and maternal contact during the incubation period, and pain exposure (Als et al., 1994; Feldman & Eidelman, 1998; White-Traut & Nelson, 1988). We hypothesized that skin-to-skin contact would promote physiological and behavior organization as expressed in more mature state organization in the neonatal period and in improved emotion regulation, joint engagement, and exploratory skills in the first <s> BIB003 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> Eighty-six premature infants were tested to examine the effects of maternal breast milk on infant development. Infants were classified by breast-milk consumption during the hospitalization period (M = 57.4 days) into three groups: those receiving minimal ( 75%) amounts of breast milk. Infants in the three groups were matched for birth weight, gestational age (GA), medical risk, and family demographics. At 37 weeks GA, mother–infant interaction was videotaped, maternal depression self-reported, and neurobehavioral maturation assessed by the Neonatal Behavior Assessment Seale (Brazelton, 1973). At 6 months corrected age, infants were tested with the Bayley II (Bayley, 1993). Infants receiving substantial amounts of breast milk showed better neurobehavioral profiles—in particular, motor maturity and range of state. These infants also were more alert during social interactions, and their mothers provided more affectionate touch. Higher maternal depression scores were associated with lower quantities of breast milk, longer latencies to the first breast-milk feeding, reduced maternal affectionate touch, and lower infant cognitive skills. Maternal affectionate touch moderated the relations between breast milk and cognitive development, with infants receiving a substantial amount of breast milk and frequent touch scoring the highest. In addition to its nutritional value, breast milk may be related to improved maternal mood and interactive behaviors, thereby indirectly contributing to development in premature infants. © 2003 Wiley Periodicals, Inc. Dev Psychobiol 43: 109–119, 2003. <s> BIB004 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> ABSTRACT Objective To examine mother and child's touch patterns in infant feeding disorders within a transactional framework. Method Infants (aged 9–34 months) referred to a community-based clinic were diagnosed with feeding disorders ( n = 20) or other primary disorder ( n = 27) and were case matched with nonreferred controls ( n = 47). Mother–child play and feeding were observed and the home environment was assessed. Microcoding detected touch patterns, response to partner's touch, and proximity at play. Relational behaviors were coded during feeding. Results Compared with infants with other primary disorder and case-matched controls, less maternal affectionate, proprioceptive, and unintentional touch was observed in those with feeding disorders. Children with feeding disorders displayed less affectionate touch, more negative touch, and more rejection of the mother's touch. More practical and rejecting maternal responses to the child's touch were observed, and children were positioned more often out of reach of the mothers' arms. Children with feeding disorders exhibited more withdrawal during feeding and the home environment was less optimal. Feeding efficacy was predicted by mother–child touch, reduced maternal depression and intrusiveness, easy infant temperament, and less child withdrawal, controlling for group membership. Conclusions Proximity and touch are especially disturbed in feeding disorders, suggesting fundamental relationship difficulties. Mothers provide less touch that supports growth, and children demonstrate signs of touch aversion. Touch patterns may serve as risk indicators of potential growth failure. <s> BIB005 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> To examine the development of triplets, 23 sets of triplets were matched with 23 sets of twins and 23 singletons (N 5138). Maternal sensitivity was observed at newborn, 3, 6, and 12 months, and infants’ cognitive and symbolic skills at 1 year. Triplets received lower maternal sensitivity across infancy and exhibited poorer cognitive competencies compared with singletons and twins. The most medically compromised triplet showed the lowest regulation, received lower maternal sensitivity, and demonstrated the weakest outcomes compared with siblings. Structural modeling charted three levels of influence on cognitive outcomes: direct, indirect, and contextual. The triplet ecology provides a context for assessing the relations among infant inborn dispositions, the rearing environment, and the role of exclusive parenting in development. Although the absolute numbers of triplets and higher order multiples born yearly may be relatively small, triplets are the most rapidly growing birth population. Rates of triplet births in the United States have increased by tenfold since the 1980s, and the same growth rates are reported in other industrial countries, such as England, France, Belgium, Hol <s> BIB006 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> We interpret early age-related developments in intentions and socially responsive behaviour with data from home videos of infants who later develop autism or Rett syndrome. Detailed evidence is given from a micro-analytic study of videos of monozygotic twin girls at 11 months, one of whom became autistic in the second year. Changes in this twin’s attention, motor tonus, initiative and emotion reduce her prospective control of movements and her anticipations in awareness compared to her sister. These changes were reflected in the child’s asynchronous social behaviour, which frustrated the father’s attempts to support her attempts to walk, share toys, or play a game, confusing his anticipations, and this further reduced mutual attention and joint activity. Observations of the development of girls with Rett syndrome in the first year reveal changes in motor coordination, attention and communicative initiative, indicative of a failure of intrinsic core brain regulations of neural development and conscious activity. Notwithstanding that the two conditions show clear differences in both brain growth and early development of skills and sociability, the first signs of autism and Rett syndrome have important similarities. We conclude with recommendations for an approach to early diagnosis and treatment, applicable for the whole range of developmental brain disorders, including Rett syndrome and autism, that attempts to identify residual capacities for sympathetic motivation and collaborative learning—an approach that deliberately tries to support weakened rhythmic impulses for motor, perceptual and communicative functions in the growing infant brain. q 2005 Elsevier B.V. All rights reserved. <s> BIB007 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> Synchrony, a construct used across multiple fields to denote the temporal relationship between events, is applied to the study of parent-infant interactions and suggested as a model for intersubjectivity. Three types of timed relationships between the parent and child's affective behavior are assessed: concurrent, sequential, and organized in an ongoing patterned format, and the development of each is charted across the first year. Viewed as a formative experience for the maturation of the social brain, synchrony impacts the development of self-regulation, symbol use, and empathy across childhood and adolescence. Different patterns of synchrony with mother, father, and the family and across cultures describe relationship-specific modes of coordination. The capacity to engage in temporally-matched interactions is based on physiological mechanisms, in particular oscillator systems, such as the biological clock and cardiac pacemaker, and attachment-related hormones, such as oxytocin. Specific patterns of synchrony are described in a range of child-, parent- and context-related risk conditions, pointing to its ecological relevance and usefulness for the study of developmental psychopathology. A perspective that underscores the organization of discrete relational behaviors into emergent patterns and considers time a central parameter of emotion and communication systems may be useful to the study of interpersonal intimacy and its potential for personal transformation across the lifespan. <s> BIB008 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> We present a new coding mechanism, spatiotemporal coding, that allows coders to annotate points and regions in the video frame by drawing directly on the screen. Coders can not only attach labels to time intervals in the video but can specify a possibly moving region on the video screen. This opens up the spatial dimension for multi-track video coding and is an essential asset in almost every area of video coding, e.g. gesture coding, facial expression coding, encoding semantics for information retrieval etc. We discuss conceptual variants, design decisions and the relation to the MPEG-7 standard and tools. <s> BIB009 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> To examine the relations between maternal representations, infant socio-emotional difficulties, and mother–child relational behavior, 49 clinic-referred infants and their mothers were compared to 30 non-referred controls. Clinic-referred infants’ psychiatric status was determined with the DC 0-3-R classification of Zeanah and Benoit (Child Adolesc Psychiatry Clin N Am 4:539–554, 1995) and controls were screened for socio-emotional difficulties. Mothers were interviewed with the parent development interview (Aber et al. in The parent development interview. Unpublished manuscript, 1985) and dyads were observed in free play and problem-solving interactions. Group differences emerged for maternal representations and relational behaviors. Representations of clinic-referred mothers were characterized by lower joy, coherence, and richness, and higher anger experienced in the mother–infant relationship compared to controls. During free play, clinic-referred mothers showed lower sensitivity and higher intrusiveness and provided less adequate instrumental and emotional assistance and support during problem solving. Referred children showed lower social engagement during free play. Associations were found among maternal representations, maternal interactive behavior, child social engagement, and the child’s ability to self-regulate during a challenging task. These findings provide empirical support for theoretical and clinical perspectives suggesting a reciprocal link between maternal negative representations and mother and child’s maladaptive behaviors in the context of early socio-emotional difficulties and mental health referrals. <s> BIB010 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> Although feeding problems are common during infancy and are typically accompanied by relational difficulties, little research observed the mother–infant feeding relationship across the first year as an antecedent to the development of feeding difficulties. We followed 76 low-risk premature infants and their mothers from the transition to oral feeding in the neonatal period to the end of the first year. Prior to hospital discharge, microlevel patterns of maternal touch and gaze were coded during feeding and nonfeeding interactions, global patterns of maternal adaptation were assessed, and infants' neurobehavioral status was tested. Psychomotor development was evaluated at 4 months. At 1 year, feeding difficulties were determined on the basis of maternal interview and direct observations of feeding interactions. Mothers of infants who exhibited feeding difficulties at 1 year showed less affectionate touch and gaze during nonfeeding interactions and more gaze aversion and lower adaptability during feeding in... <s> BIB011 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> Abstract The current study reviewed all prior studies conducted on family home movies of infants who would be later diagnosed with autism spectrum disorder (ASD). Out of 41 original reports found since 1975, we retained 18 studies (317 films, maximum), sorted according to their methodological design using a quality grid. In the first 2 years of life, signs that differentiated children with ASD from children with developmental delays were as follows: less of a response to their name, less looking at others, lower eye contact quality and quantity, less positive facial expression and intersubjective behaviors (e.g., showing shared attention). Studies focusing on regression confirmed the clinical validity of the phenomena. We conclude that findings from home movies studies along with prospective studies have created the bases for identification of infants and toddlers at risk of developing ASD before the 18–24-month period, despite early diagnosis of autism remains a complex challenge. <s> BIB012 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> This study focuses on incorporating knowledge from multiple annotators into a machine-learning framework for detecting psychological traits using multimodal data. We present a model that is designed to exploit the judgements of multiple annotators on a social trait labeling task. Our two-stage model first estimates a ground truth by modeling the annotators using both the annotations and annotators’ self-reported confidences. In the second stage, we train a classifier using the estimated ground truth as labels. Our experiments on a dominance estimation task in a group interaction scenario on the DOME corpus, in addition to synthetically generated data, give satisfactory results, outperforming the commonly used majority voting as well as other approaches in the literature. <s> BIB013 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> NONCOMPUTATIONAL METHODS OF SYNCHRONY ASSESSMENT <s> Objective: The authors quantified nonverbal synchrony—the coordination of patient's and therapist's movement—in a random sample of same-sex psychotherapy dyads. The authors contrasted nonverbal synchrony in these dyads with a control condition and assessed its association with session-level and overall psychotherapy outcome. Method: Using an automated objective video analysis algorithm (Motion Energy Analysis; MEA), the authors calculated nonverbal synchrony in (n = 104) videotaped psychotherapy sessions from 70 Caucasian patients (37 women, 33 men, mean age = 36.5 years, SD = 10.2) treated at an outpatient psychotherapy clinic. The sample was randomly drawn from an archive (N = 301) of routinely videotaped psychotherapies. Patients and their therapists assessed session impact with self-report postsession questionnaires. A battery of pre- and postsymptomatology questionnaires measured therapy effectiveness. Results: The authors found that nonverbal synchrony is higher in genuine interactions contrasted with pseudointeractions (a control condition generated by a specifically designed shuffling procedure). Furthermore, nonverbal synchrony is associated with session-level process as well as therapy outcome: It is increased in sessions rated by patients as manifesting high relationship quality and in patients experiencing high self-efficacy. Higher nonverbal synchrony characterized psychotherapies with higher symptom reduction. Conclusions: The results suggest that nonverbal synchrony embodies the patients' self-reported quality of the relationship and further variables of therapy process. This hitherto overlooked facet of therapeutic relationships might prove useful as an indicator of therapy progress and outcome. <s> BIB014
In the earliest days of synchrony research, instances of synchrony were directly perceived in the data by trained observers. Several methods have been proposed to evaluate interactional synchrony, ranging from behavior microanalysis to global perception of synchrony. Behavioral coding methods propose evaluating the behavior of each interactional partner on a local scale. These methods require the use of computer-based coding (e.g., Observer or Anvil BIB009 ) and trained raters. Various category and time scales can be used for coding. In  , Cappella synthesized the three crucial questions to be addressed when conducting an interaction study: "what to observe (coding), how to represent observations (data representations) and when and how frequently to make the observations (time)." For instance, Condon and Sander as well as Cappella BIB001 , BIB002 proposed analyzing microunits of behavior. They annotated the speech segments and the direction of movement of different body parts (head, eyes, mouth, elbows, trunk, shoulder, wrists, and fingers). At a higher scale, some grids directly analyze interactive behaviors (smiles, gazes, illustrative gestures, adaptors, head gestures (nods, shakes)) or functional states (alertness, orientation between the partners, communicative expression, emotion, body contact, and postural tension) BIB007 ,  . Generally, a measure of synchrony is deduced from the covariation of the annotated behaviors. The codes can be either continuous (speed of a gesture) or categorical (type of gesture). This type of grid has been widely used for coding home movies. Coding home movies is particularly complex because of the naturalistic setting and the varying quality of the films. This approach has been largely used in the field of autism to improve our understanding of the early developmental course of children who will be eventually diagnosed with ASD BIB012 . This knowledge is needed to better understand the complex pathogenic phenomena of autism and to improve the early screening and management of autism. Behavioral coding methods are time consuming and tedious with regard to the training of observers, the number of behaviors coded, and the duration of the video files to be coded, particularly for longitudinal studies. Cappella BIB002 and Bernieri et al.  proposed an alternative to behavior microanalysis: the judgment method. In their studies, they investigated the use of human raters to evaluate video clips of infants interacting with their mothers. Raters judge for simultaneous movement, tempo similarity, and coordination and smoothness on a longer time scale using a Likert scale. Cappella showed that untrained judges were consistent with one another and reliably judged the synchrony between partners BIB002 . Nevertheless, considering the complexity of the underlying phenomenon, the reliability of the coders' judgment can be problematic. This question can be partially circumvented by the use of a scale of several items to test a given construct and several judges. Another method is the Coding Interactive Behavior (CIB)  , a well-validated system for coding mother-infant interactions requiring trained observers  , BIB008 . The CIB is a global rating system of parent-child interaction that contains both microlevel codes and global rating scales. Codes are averaged into six composites (maternal sensitivity, mother intrusiveness, limits, mother-infant positive affect, infant involvement, and negative dyadic status) that are theoretically derived and address several aspects of the early mother-infant relationship, showing acceptable to high internal consistency  . The coding of feeding interactions has been shown to differentiate infants diagnosed with psychiatric disorders in infancy BIB005 , BIB010 or prematurity BIB011 . The CIB was also used in studies assessing the effect on early childhood development and interactive behavior of various maternal/parental styles, such as breastfeeding BIB004 , skin-to-skin contact (kangaroo care) BIB003 , and parent or child gender BIB006 . Noncomputational methods suffer serious drawbacks. Within the tedious task of coding, segmenting, and annotating behaviors can be confusing: When does a behavior start, when does it end, how should it be labeled? Often, the annotator makes a tradeoff because no label accurately describes what he observes. The judges' reliability in assessing such a subjective and complex construct is also questionable, and no general framework for synchrony assessment has been accepted to date. A method was recently proposed to convert the judgments of multiple annotators in a study on dominance into a machine-learning framework BIB013 . Finally, conversational partners are often studied individually when coding. Thus, it is particularly difficult to recreate the dynamic and interpersonal aspects of social interaction manually and after coding. Nonetheless, annotation and judgment methods are essential in proposing automatic systems for synchrony assessment and testing their performance. Currently, no automatic systems modeling synchrony using real interaction data are free from annotation. Annotation is mainly used in two different manners. First, annotation is used to train automatic systems to model and learn communication dynamics (see Section 6) . These studies often rely on behavioral coded databases. Second, another set of studies intends to measure the degree of synchrony between dyadic partners with unsupervised methods (see Section 5) . In these studies, the measure of synchrony is not validated per se but is judged by its ability to predict an outcome variable that has been manually annotated, often using judgment methods. The outcome variable can be friendship  , conflicting situations  , success in psychotherapy BIB014 , etc.