In this work, we present a framework based on multi-stream convolutional neural networks (CNNs) for group activity recognition. Streams of CNNs are separately trained on different modalities and their predictions are fused at the end. Each stream has two branches to predict the group activity based on person and scene level representations. A new modality based on the human pose estimation is presented to add extra information to the model. We evaluate our method on the Volleyball and Collective Activity datasets. Experimental results show that the proposed framework is able to achieve state-of-the-art results when multiple or single frames are given as input to the model with 90.50 and 86.61 accuracy on Volleyball dataset, respectively, and 87.01 accuracy of multiple frames group activity on Collective Activity dataset.
There have been efforts to use probabilistic graphical models to tackle the group activity recognition problem. @cite_26 propose a graphical model with person-person and group-person factors and employ a two-stage inference mechanism to find the optimal graph structure and the best possible labels for the individual actions and collective activity. In @cite_15 , the authors explore the idea of tracking persons and predicting their activity as a group in a joint probabilistic framework. @cite_17 use a latent graph model for multi-target tracking, activity group localization and group activity recognition.
@cite_15 @cite_26 @cite_17
2320007652 2023048829 100367037 2003708924
A latent graphical model integrating multi-target tracking, group discovery, and activity recognition is proposed.Performance of activity recognition improves when multi-target tracking and group clustering are incorporated.Group activities are better recognized based on the structured relations within the group and group-group compatibilities.Increasing the connectivity of different groups improves the overall performance.Incorporating activity information leads to robust group localization in the video. Beyond recognizing actions of individuals, activity group localization in videos aims to localize groups of persons in spatiotemporal spaces and recognize what activity the group performs. In this paper, we propose a latent graph model to simultaneously address the problem of multi-target tracking, group discovery and activity recognition. Our key insight is to exploit the contextual relations among people. We present them as a latent relational graph, which hierarchically encodes the association potentials between tracklets, intra-group interactions, correlations, and inter-group compatibilities. Our model is capable of propagating multiple evidences among different layers of the latent graph. Particularly, associated tracklets assist accurate group discovery, activity recognition can benefit from knowing the whole structured groups, and the group and activity information in turn provides strong cues for establishing coherent associations between tracklets. Experiments on five datasets demonstrate that our model achieves both significant improvements in activity group localization and competitive performance on activity recognition. This paper addresses the challenge of recognizing behavior of groups of individuals in unconstraint surveillance environments. As opposed to approaches that rely on agglomerative or decisive hierarchical clustering techniques, we propose to recognize group interactions without making hard decisions about the underlying group structure. Instead we use a probabilistic grouping strategy evaluated from the pairwise spatial-temporal tracking information. A path-based grouping scheme determines a soft segmentation of groups and produces a weighted connection graph where its edges express the probability of individuals belonging to a group. Without further segmenting this graph, we show how a large number of low- and high-level behavior recognition tasks can be performed. Our work builds on a mature multi-camera multi-target person tracking system that operates in real-time. We derive probabilistic models to analyze individual track motion as well as group interactions. We show that the soft grouping can combine with motion analysis elegantly to robustly detect and predict group-level activities. Experimental results demonstrate the efficacy of our approach. We present a coherent, discriminative framework for simultaneously tracking multiple people and estimating their collective activities. Instead of treating the two problems separately, our model is grounded in the intuition that a strong correlation exists between a person's motion, their activity, and the motion and activities of other nearby people. Instead of directly linking the solutions to these two problems, we introduce a hierarchy of activity types that creates a natural progression that leads from a specific person's motion to the activity of the group as a whole. Our model is capable of jointly tracking multiple people, recognizing individual activities (atomic activities), the interactions between pairs of people (interaction activities), and finally the behavior of groups of people (collective activities). We also propose an algorithm for solving this otherwise intractable joint inference problem by combining belief propagation with a version of the branch and bound algorithm equipped with integer programming. Experimental results on challenging video datasets demonstrate our theoretical claims and indicate that our model achieves the best collective activity classification results to date. This paper describes a stochastic methodology for the recognition of various types of high-level group activities. Our system maintains a probabilistic representation of a group activity, describing how individual activities of its group members must be organized temporally, spatially, and logically. In order to recognize each of the represented group activities, our system searches for a set of group members that has the maximum posterior probability of satisfying its representation. A hierarchical recognition algorithm utilizing a Markov chain Monte Carlo (MCMC)-based probability distribution sampling has been designed, detecting group activities and finding the acting groups simultaneously. The system has been tested to recognize complex activities such as a group of thieves stealing an object from another group' and a group assaulting a person'. Videos downloaded from YouTube as well as videos that we have taken are tested. Experimental results show that our system recognizes a wide range of group activities more reliably and accurately, as compared to previous approaches.