This paper presents a method to reconstruct dense semantic trajectory stream of human interactions in 3D from synchronized multiple videos. The interactions inherently introduce self-occlusion and illumination appearance shape changes, resulting in highly fragmented trajectory reconstruction with noisy and coarse semantic labels. Our conjecture is that among many views, there exists a set of views that can confidently recognize the visual semantic label of a 3D trajectory. We introduce a new representation called 3D semantic map---a probability distribution over the semantic labels per trajectory. We construct the 3D semantic map by reasoning about visibility and 2D recognition confidence based on view-pooling, i.e., finding the view that best represents the semantics of the trajectory. Using the 3D semantic map, we precisely infer all trajectory labels jointly by considering the affinity between long range trajectories via estimating their local rigid transformations. This inference quantitatively outperforms the baseline approaches in terms of predictive validity, representation robustness, and affinity effectiveness. We demonstrate that our algorithm can robustly compute the semantic labels of a large scale trajectory set involving real-world human interactions with object, scenes, and people.
Humans can effortlessly read the intent of others through subtle behavioral cues in a fraction of second @cite_8 , and high resolution videos are now able to capture such cues via our interactions with surrounding environments. The pixels in the videos can be tracked to form long term trajectories to encode the interactions both in 2D and 3D.
@cite_8
2053907973
A meta-analysis was conducted on the accuracy of predictions of various objective outcomes in the areas of social and clinical psychology from short observations of expressive behavior (under 5 min). The overall effect size ( ) for the accuracy of predictions for 38 different results was .39. Studies using longer periods of behavioral observation did not yield greater predictive accuracy; predictions based on observations under Vi min in length did not differ significantly from predictions based on 4- and 5-min observations. The type of behavioral channel (such as the face, speech, the body, tone of voice) on which the ratings were based was not related to the accuracy of predictions. Accuracy did not vary significantly between behaviors manipulated in a laboratory and more naturally occurring behavior. Last, effect sizes did not differ significantly for predictions in the areas of clinical psychology, social psychology, and the accuracy of detecting deception. The way in which people move, talk, and gesture—their facial expressions, posture, and speech—all contribute to the formation of impressions about them. Many of the judgments we make about others in our everyday lives are based on cues from these expressive behaviors. Gordon Allport (1937) believed that expressive behaviors were important indicators of personality and that impressions from brief interactions were often verified upon further acquaintance. Allport and Vernon (1933) demonstrated that people's expressive styles were quite consistent across a variety of situations. They then began to investigate the accuracy of perceivers' impressions that were based on observations of these expressive styles. For reasons that have been discussed elsewhere, this issue—like other issues concerning the accuracy of interpersonal and social perception— was neglected for a long time (Funder, 1987; Kenny & Albright, 1987). Recently, however, there has been a resurgence of interest in the accuracy of social and interpersonal perception and judgment (Funder, 1987; Kenny & Albright, 1987; Kruglanski, 1989; Swann, 1984) and in the study of expressive behavior (Lippa, 1983; Riggio & Friedman, 1986). Recent work confirms earlier findings (Passini & Norman, 1966) that the ratings of strangers converge surprisingly well with self-ratings of personality by targets (Albright, Kenny, & Malloy, 1988; Funder & Cohan, 1988; Watson, 1989). This correspondence seems to confirm Gordon Allport's observation that there is something in the nature of individuals that