The ability to extract the sequence of game events for a given player's play-through has traditionally required access to the game's engine or source code. This serves as a barrier to researchers, developers, and hobbyists who might otherwise benefit from these game logs. In this paper we present two approaches to derive game logs from game video via convolutional neural networks and transfer learning. We evaluate the approaches in a Super Mario Bros. clone, Mega Man and Skyrim. Our results demonstrate our approach outperforms random forest and other transfer baselines.
There exists a large set of applications of activity recognition for real world games and sports @cite_18 . However, this is largely enabled by the existence of large datasets of real world human activity @cite_21 . Na "ively, one might think that differences in game aesthetics would indicate a need for individual datasets of this size for each game to apply these methods. However, we demonstrate that one can create high-quality activity recognition models for realistic games with only a small corpus.
@cite_18 @cite_21
2086598255 24089286
Recognition of player actions in broadcast sports video is a challenging task due to low resolution of the players in video frames. In this paper, we present a novel method to recognize the basic player actions in broadcast tennis video. Different from the existing appearance-based approaches, our method is based on motion analysis and considers the relationship between the movements of different body parts and the regions in the image plane. A novel motion descriptor is proposed and supervised learning is employed to train the action classifier. We also propose a novel framework by combining the player action recognition with other multimodal features for semantic and tactic analysis of the broadcast tennis video. Incorporating action recognition into the framework not only improves the semantic indexing and retrieval performance of the video content, but also conducts highlights ranking and tactics analysis in tennis matches, which is the first solution to our knowledge for tennis game. The experimental results demonstrate that our player action recognition method outperforms existing appearance-based approaches and the multimodal framework is effective for broadcast tennis video analysis. We introduce UCF101 which is currently the largest dataset of human actions. It consists of 101 action classes, over 13k clips and 27 hours of video data. The database consists of realistic user uploaded videos containing camera motion and cluttered background. Additionally, we provide baseline action recognition results on this new dataset using standard bag of words approach with overall performance of 44.5 . To the best of our knowledge, UCF101 is currently the most challenging dataset of actions due to its large number of classes, large number of clips and also unconstrained nature of such clips.