This paper presents a method to predict the future movements (location and gaze direction) of basketball players as a whole from their first person videos. The predicted behaviors reflect an individual physical space that affords to take the next actions while conforming to social behaviors by engaging to joint attention. Our key innovation is to use the 3D reconstruction of multiple first person cameras to automatically annotate each other's the visual semantics of social configurations. We leverage two learning signals uniquely embedded in first person videos. Individually, a first person video records the visual semantics of a spatial and social layout around a person that allows associating with past similar situations. Collectively, first person videos follow joint attention that can link the individuals to a group. We learn the egocentric visual semantics of group movements using a Siamese neural network to retrieve future trajectories. We consolidate the retrieved trajectories from all players by maximizing a measure of social compatibility---the gaze alignment towards joint attention predicted by their social formation, where the dynamics of joint attention is learned by a long-term recurrent convolutional network. This allows us to characterize which social configuration is more plausible and predict future group trajectories.
A group as a whole naturally creates a distinctive geometry of social formation that accommodates its social activity, e.g., a street busker's performance surrounded by a crowd with a half circular formation. Therefore, the formation can be a key indicator to classify the type of social configurations that influence individual behaviors with respect to the group. For instance, Kendon's F-formation theory @cite_2 characterizes the spatial arrangements of a social group, that can be used to identify social interactions in an image @cite_7 , and its validity is empirically proven using a large social interaction dataset @cite_11 . In dynamic social scenes, the formation enables re-identifying a group of people in a crowd from non-overlapping camera views @cite_12 , and the progression of formation change can be learned via inverse reinforcement learning @cite_45 and discriminative analysis (LSTM) @cite_29 .
@cite_7 @cite_29 @cite_45 @cite_2 @cite_12 @cite_11
2010236945 1912797782 1977186326 2168780230
We present a novel approach for detecting social interactions in a crowded scene by employing solely visual cues. The detection of social interactions in unconstrained scenarios is a valuable and important task, especially for surveillance purposes. Our proposal is inspired by the social signaling literature, and in particular it considers the sociological notion of F-formation. An F-formation is a set of possible configurations in space that people may assume while participating in a social interaction. Our system takes as input the positions of the people in a scene and their (head) orientations; then, employing a voting strategy based on the Hough transform, it recognizes F-formations and the individuals associated with them. Experiments on simulations and real data promote our idea. This paper presents a method to predict social saliency, the likelihood of joint attention, given an input image or video by leveraging the social interaction data captured by first person cameras. Inspired by electric dipole moments, we introduce a social formation feature that encodes the geometric relationship between joint attention and its social formation. We learn this feature from the first person social interaction data where we can precisely measure the locations of joint attention and its associated members in 3D. An ensemble classifier is trained to learn the geometric relationship. Using the trained classifier, we predict social saliency in real-world scenes with multiple social groups including scenes from team sports captured in a third person view. Our representation does not require directional measurements such as gaze directions. A geometric analysis of social interactions in terms of the F-formation theory is also presented. Sociologists since Simmel have been interested in social circles as essential features of friendship networks. Although network analysis has been increasingly used to uncover patterns among social relationships, theoretical explanations of these patterns have been inadequate. This paper presents a theory of the social organization of friendship ties. The approach is based upon Homans's concepts of activities, interactions, and sentiments and upon the concept of extra-network foci organizing social activities and interaction. The theory is contrasted with Heider's balance theory. Implications for transitivity, network bridges, and density of personal networks are discussed and presented as propositions. The focus theory is shown to help explain patterns of friendships in the 1965-66 Detroit Area Study. This paper is intended as a step toward the development of integrated theory to explain interrelationships between networks and other aspects of social structure. Implications for data analysis are discussed. Sociologists have long recognized the importance of patterns in networks of relations that connect individuals with each other. Simmel (1955) described modern society as consisting of loosely connected social circles of relationships. Granovetter (1973) has indicated the general significance of these social circles for communication, community organization, and social conflict. Various studies have supported this picture of the essential patterns in social networks, including Moreno's sociometry (1953), Milgram's "small world" experiments (1967), and Kadushin's observations (1966). Unfortunately, the study of social networks has often been carried out without concern for the origins in the larger social context. Most network analysis ends with description and labeling of patterns; and when explanations of patterns are offered, they frequently rely upon inherent tendencies within networks to become consistent, balanced, or transitive. As a consequence of such atheoretical and or self-contained network theoretical approaches, data are collected and data analysis techniques are devised for Schools of fish and flocks of birds are examples of self-organized animal groups that arise through social interactions among individuals. We numerically study two individual-based models, which recent empirical studies have suggested to explain self-organized group animal behavior: (i) a zone-based model where the group communication topology is determined by finite interacting zones of repulsion, attraction, and orientation among individuals; and (ii) a model where the communication topology is described by Delaunay triangulation, which is defined by each individual's Voronoi neighbors. The models include a tunable parameter that controls an individual's relative weighting of attraction and alignment. We perform computational experiments to investigate how effectively simulated groups transfer information in the form of velocity when an individual is perturbed. A cross-correlation function is used to measure the sensitivity of groups to sudden perturbations in the heading of individual members. The results show how relative weighting of attraction and alignment, location of the perturbed individual, population size, and the communication topology affect group structure and response to perturbation. We find that in the Delaunay-based model an individual who is perturbed is capable of triggering a cascade of responses, ultimately leading to the group changing direction. This phenomenon has been seen in self-organized animal groups in both experiments and nature.