Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Modeling Social Interaction as Sequences of Behaviors <s> A new variant ‘PMF’ of factor analysis is described. It is assumed that X is a matrix of observed data and σ is the known matrix of standard deviations of elements of X. Both X and σ are of dimensions n × m. The method solves the bilinear matrix problem X = GF + E where G is the unknown left hand factor matrix (scores) of dimensions n × p, F is the unknown right hand factor matrix (loadings) of dimensions p × m, and E is the matrix of residuals. The problem is solved in the weighted least squares sense: G and F are determined so that the Frobenius norm of E divided (element-by-element) by σ is minimized. Furthermore, the solution is constrained so that all the elements of G and F are required to be non-negative. It is shown that the solutions by PMF are usually different from any solutions produced by the customary factor analysis (FA, i.e. principal component analysis (PCA) followed by rotations). Usually PMF produces a better fit to the data than FA. Also, the result of PF is guaranteed to be non-negative, while the result of FA often cannot be rotated so that all negative entries would be eliminated. Different possible application areas of the new method are briefly discussed. In environmental data, the error estimates of data can be widely varying and non-negativity is often an essential feature of the underlying models. Thus it is concluded that PMF is better suited than FA or PCA in many environmental applications. Examples of successful applications of PMF are shown in companion papers. <s> BIB001 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Modeling Social Interaction as Sequences of Behaviors <s> This study examines the existence of behavioral correlates of synchronization on different levels of analysis and methods. We were unable to demonstrate a relation between synchronization defined in terms of movement echo or position mirroring and subjective experience of pleasure and interest in opposite-sex encounters. Significant results were found for a phenomenon we describe as hierarchically patterned synchronization. These patterns were identified with the help of a newly developed search algorithm. If a female is interested in a male, highly complex patterns of behavior with a constant time structure emerge. The patterns are pair-specific and independent from behavioral content. This rhythmic structure of interactions is discussed in functional terms of human courtship. <s> BIB002 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Modeling Social Interaction as Sequences of Behaviors <s> This paper introduces the problem of combining multiple partitionings of a set of objects into a single consolidated clustering without accessing the features or algorithms that determined these partitionings. We first identify several application scenarios for the resultant 'knowledge reuse' framework that we call cluster ensembles. The cluster ensemble problem is then formalized as a combinatorial optimization problem in terms of shared mutual information. In addition to a direct maximization approach, we propose three effective and efficient techniques for obtaining high-quality combiners (consensus functions). The first combiner induces a similarity measure from the partitionings and then reclusters the objects. The second combiner is based on hypergraph partitioning. The third one collapses groups of clusters into meta-clusters which then compete for each object to determine the combined clustering. Due to the low computational costs of our techniques, it is quite feasible to use a supra-consensus function that evaluates all three approaches against the objective function and picks the best solution for a given situation. We evaluate the effectiveness of cluster ensembles in three qualitatively different application scenarios: (i) where the original clusters were formed based on non-identical sets of features, (ii) where the original clustering algorithms worked on non-identical sets of objects, and (iii) where a common data-set is used and the main purpose of combining multiple clusterings is to improve the quality and robustness of the solution. Promising results are obtained in all three situations for synthetic as well as real data-sets. <s> BIB003 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Modeling Social Interaction as Sequences of Behaviors <s> This article deals with the definition and detection of particular kinds of temporal patterns in behavior, which are sometimes obvious or well known, but other times difficult to detect, either directly or with standard statistical methods. Characteristics of well-known behavior patterns were abstracted and combined in order to define a scale-independent, hierarchical time pattern type, called aT-pattern. A corresponding detection algorithm was developed and implemented in a computer program, called Theme. The proposed pattern typology and detection algorithm are based on the definition and detection of a particular relationship between pairs of events in a time series, called acritical interval relation. The proposed bottom-up, level-by-level (or breadth-first) search algorithm is based on a binary tree of such relations. The algorithm first detects simpler patterns. Then, more complex and complete patterns evolve through the connection of simpler ones, pattern completeness competition, and pattern selection. Interindividual T-patterns in a quarter-hour interaction between two children are presented, showing that complex hidden T-patterns may be found by Theme in such behavioral streams. Finally, implications for studies of complexity, self-organization, and dynamic patterns are discussed. <s> BIB004 </s> Interpersonal Synchrony: A Survey of Evaluation Methods across Disciplines <s> Modeling Social Interaction as Sequences of Behaviors <s> There are considerable differences among infants in the quality of interaction with their parents. These differences depend especially on the infants development which affects the parent behaviors. In our study we are interested on 3 groups of infants: typical development infants (TD), autistic infants (AD) and mental retardation infants (MR). In order to identify the groups of signs/behaviors that differentiate the development of the three groups of children we investigated a clustering method NMF (non-negative matrix factorization), an algorithm based on decomposition by parts that can reduce the dimension of interaction signs to a few number of interaction behaviors groups. Coupled with a statistical data representation tf-idf, usually used for document clustering and adapted to our work, NMF provides an efficient method for identification of distinct interaction groups. Forty-two infants and their parents were observed in this study. Parent-infant interactions were videotaped by one of the parents at home during the first two years age of child. <s> BIB005
Machine learning methods offer an interesting framework for the exploration of interactive behaviors. A key challenge is proposing models with the content and temporal structure of dyadic interactions. Various sequential learning models, such as Hidden Markov Models (HMMs) or Conditional Random Fields (CRFs), are usually used to characterize the temporal structure of social interactions. Messinger et al. employ related techniques for the understanding of communicative development, which is characterized by mutual influences during interaction: Infants and parents influence and respond to one another during communication  . The authors focus on some specific social signals (e.g., smiling) and propose statistical approaches for the characterization of this signal on the response of the partner, who can be the parent or the infant. Probability distributions of transitions between states of behaviors previously annotated are estimated by maximum likelihood approach: pði a ; m a ; i aÀ1 ; m aÀ1 Þ. These probabilities are used to characterize the dynamic of the early dyadic interaction using high-level labeled information, such as smiles. The comparison between the interactive situations among infant development is accomplished using a similarity metric (Battacharyya coefficient). Modeling temporal structure is not always sufficient. Magnusson underlines the complexity of the detection of such interactive patterns using methods that simply study the sequence of events BIB004 . Interactive patterns are 1) hierarchical, a pattern is often formed of different levels of subpatterns, and 2) variable, the number and types of behavior can vary greatly from one instance to another of the same pattern. He proposed an algorithm to characterize the complex hidden and repetitive temporal structure of verbal and nonverbal behavior (T-pattern). A T-pattern is defined as a repeated occurrence of a sequence of events that are separated by a "relatively invariant" time interval, the critical interval. The invariance of the time interval is evaluated by comparing the probability of observing this time interval between two successive events with the probability of observing the same time interval in a randomly distributed sequence of events. The strength of the algorithm is that it allows the insertion or omission of events according to the instance of the pattern. Moreover, the algorithm provides a hierarchical tree diagram to model the existence of patterns and subpatterns. Running the algorithm on randomized time series allows an analysis of whether some of the patterns are expected to occur by chance. Magnusson also suggests the danger of simply focusing on two successive events, which could lead to a misinterpretation of the causality of events, while the n prior behaviors of the T-pattern may also have an influence. T-pattern methodology and the Theme software have been applied to the analysis of social interaction with autistic children  , movement synchrony in interpersonal attraction  , BIB002 , and symmetric behaviors in social interaction  . Dynamic verbal and nonverbal communications usually involve multimodal signals more or less simultaneously produced. Investigations of communication dynamics should address this phenomenon; integrated models offer relevant solutions. In [120] , BIB005 , an integrative approach is proposed to explicitly consider the interaction synchrony of behaviors. The model is applied to the characterization of parent-infant interactions for differential diagnosis: autism (AD), intellectual disability (ID), and typical development (TD). As in  , the authors estimate transitions between behaviors of the infant and the parent by analyzing behaviors co-occurring in a 3 s window. Assuming a Markovian process, they used a maximum likelihood estimation to estimate the probability of each interactive pattern, resulting in bigram models characterizing the temporal structure. To study these interactive patterns from an integrative perspective, the authors proposed employing a more global model using nonnegative matrix factorization (NMF) BIB001 , coupled with statistical representation, namely, term frequencyinverse document frequency (tf-idf), to transform the scene annotations into a representation suitable for the learning algorithm and the clustering task (NMF). A global nonnegative matrix grouping all interactive behaviors is constructed and decomposed into a few interactive behaviors groups. NMF is an unsupervised feature extraction method involving the decomposition of the nonnegative matrix into two nonnegative matrices. The nonnegativity constraints are relevant for the analysis of human behaviors as they allow only additive, not subtractive, combinations. Because of the mathematical properties of NMF, the analysis accommodates an integrative perspective by providing clusters of interactive behaviors. In addition, to understand the development similarity of TD infants compared with AD and ID infants, the authors used Normalized Mutual Information (NMI), as proposed by Strehl and Ghosh BIB003 ; the NMI between two different clustering solutions measures their agreement. Interestingly, the NMI profiles fitted the clinical hypothesis closely, showing a pervasive development in AD and a delayed development in ID [120] .