In this paper we consider the problem of human pose estimation in real-world videos of swimmers. Swimming channels allow filming swimmers simultaneously above and below the water surface with a single stationary camera. These recordings can be used to quantitatively assess the athletes' performance. The quantitative evaluation, so far, requires manual annotations of body parts in each video frame. We therefore apply the concept of CNNs in order to automatically infer the required pose information. Starting with an off-the-shelf architecture, we develop extensions to leverage activity information – in our case the swimming style of an athlete – and the continuous nature of the video recordings. Our main contributions are threefold: (a) We apply and evaluate a fine-tuned Convolutional Pose Machine architecture as a baseline in our very challenging aquatic environment and discuss its error modes, (b) we propose an extension to input swimming style information into the fully convolutional architecture and (c) modify the architecture for continuous pose estimation in videos. With these additions we achieve reliable pose estimates with up to +16 more correct body joint detections compared to the baseline architecture.
Computer vision has been adopted for various applications in the sports domain. Prominent tasks include sports type @cite_17 and activity recognition @cite_4 @cite_19 , tracking athletes and other objects of interest in videos @cite_8 @cite_24 and human pose estimation @cite_20 @cite_31 . @cite_23 offer an overview of a wide range of application.
@cite_4 @cite_8 @cite_24 @cite_19 @cite_23 @cite_31 @cite_20 @cite_17
2279003914 1939033588 2071555787 2260521078
Human activity recognition is a fundamental problem in computer vision with many applications such as video retrieval, automatic visual surveillance and human computer interaction. Sports represent one of the most viewed content on digital tv and the web. Automatically collected statistics of team sports game play represent actionable information for many end users such as coaches and broadcast speakers. Many computer vision methods applied to sport activity classification are often based on multi-camera setups, player tracking and exploit information on the ground-plane. In this work we overcome this limitations and propose an approach that exploits the spatio-temporal structure of a video grouping local spatio-temporal features unsupervisedly. Our robust representation allows to measure video similarity making correspondences among arbitrary patterns. We tested our method on two dataset of Volleyball and Soccer actions outperforming previous results by a large margin. Finally we show how our representation allows to highlight discriminative regions for each action. The first book of its kind devoted to this topic, this comprehensive text reference presents state-of-the-art research and reviews current challenges in the application of computer vision to problems in sports. Opening with a detailed introduction to the use of computer vision across the entire life-cycle of a sports event, the text then progresses to examine cutting-edge techniques for tracking the ball, obtaining the whereabouts and pose of the players, and identifying the sport being played from video footage. The work concludes by investigating a selection of systems for the automatic analysis and classification of sports play. The insights provided by this pioneering collection will be of great interest to researchers and practitioners involved in computer vision, sports analysis and media production. Systems based on bag-of-words models from image features collected at maxima of sparse interest point operators have been used successfully for both computer visual object and action recognition tasks. While the sparse, interest-point based approach to recognition is not inconsistent with visual processing in biological systems that operate in ‘saccade and fixate’ regimes, the methodology and emphasis in the human and the computer vision communities remains sharply distinct. Here, we make three contributions aiming to bridge this gap. First, we complement existing state-of-the art large scale dynamic computer vision annotated datasets like Hollywood-2 [1] and UCF Sports [2] with human eye movements collected under the ecological constraints of visual action and scene context recognition tasks. To our knowledge these are the first large human eye tracking datasets to be collected and made publicly available for video, vision.imar.ro eyetracking (497,107 frames, each viewed by 19 subjects), unique in terms of their (a) large scale and computer vision relevance, (b) dynamic, video stimuli, (c) task control, as well as free-viewing . Second, we introduce novel dynamic consistency and alignment measures , which underline the remarkable stability of patterns of visual search among subjects. Third, we leverage the significant amount of collected data in order to pursue studies and build automatic, end-to-end trainable computer vision systems based on human eye movements. Our studies not only shed light on the differences between computer vision spatio-temporal interest point image sampling strategies and the human fixations, as well as their impact for visual recognition performance, but also demonstrate that human fixations can be accurately predicted, and when used in an end-to-end automatic system, leveraging some of the advanced computer vision practice, can lead to state of the art results. While recent advances in computer vision have provided reliable methods to recognize actions in both images and videos, the problem of assessing how well people perform actions has been largely unexplored in computer vision. Since methods for assessing action quality have many real-world applications in healthcare, sports, and video retrieval, we believe the computer vision community should begin to tackle this challenging problem. To spur progress, we introduce a learning-based framework that takes steps towards assessing how well people perform actions in videos. Our approach works by training a regression model from spatiotemporal pose features to scores obtained from expert judges. Moreover, our approach can provide interpretable feedback on how people can improve their action. We evaluate our method on a new Olympic sports dataset, and our experiments suggest our framework is able to rank the athletes more accurately than a non-expert human. While promising, our method is still a long way to rivaling the performance of expert judges, indicating that there is significant opportunity in computer vision research to improve on this difficult yet important task.