Activity analysis in which multiple people interact across a large space is challenging due to the interplay of individual actions and collective group dynamics. We propose an end-to-end approach for learning person trajectory representations for group activity analysis. The learned representations encode rich spatio-temporal dependencies and capture useful motion patterns for recognizing individual events, as well as characteristic group dynamics that can be used to identify groups from their trajectories alone. We develop our deep learning approach in the context of team sports, which provide well-defined sets of events (e.g. pass, shot) and groups of people (teams). Analysis of events and team formations using NHL hockey and NBA basketball datasets demonstrate the generality of our approach.
The existing literature on analyzing human activities is extensive. Thorough surveys of earlier work include Gavrila @cite_22 and @cite_18 . Below, we review closely related work in activity recognition, including individual actions, group multi-person activities, and trajectory analysis.
@cite_18 @cite_22
2098339052 2121899951
Human activity recognition is an important area of computer vision research. Its applications include surveillance systems, patient monitoring systems, and a variety of systems that involve interactions between persons and electronic devices such as human-computer interfaces. Most of these applications require an automated recognition of high-level activities, composed of multiple simple (or atomic) actions of persons. This article provides a detailed overview of various state-of-the-art research papers on human activity recognition. We discuss both the methodologies developed for simple human actions and those for high-level activities. An approach-based taxonomy is chosen that compares the advantages and limitations of each approach. Recognition methodologies for an analysis of the simple actions of a single person are first presented in the article. Space-time volume approaches and sequential approaches that represent and recognize activities directly from input images are discussed. Next, hierarchical recognition methodologies for high-level activities are presented and compared. Statistical approaches, syntactic approaches, and description-based approaches for hierarchical recognition are discussed in the article. In addition, we further discuss the papers on the recognition of human-object interactions and group activities. Public datasets designed for the evaluation of the recognition methodologies are illustrated in our article as well, comparing the methodologies' performances. This review will provide the impetus for future research in more productive areas. Most approaches to detection and classification of human activity deal with observing individual persons. However, people often tend to organize into groups to achieve certain goals, and human activity is sometimes more readily defined and observed in the context of whole group, where the activity is coordinated among its members. An excellent example of this are team sports, which can provide valuable test ground for development of methods for analysis of coordinated group activity. We used basketball play in this work and developed a probabilistic model of a team play, which is based on the detection of key events in the team behavior. The model is based on expert coach knowledge and has been used to assess the team performance in three different types of basketball offense, based on trajectories of all players, obtained by whole-body tracker. Results show that our high-level behaviour model may be used both for activity recognition and performance evaluation in certain basketball activities.