Assigning team labels to players in a sport game is not a trivial task when no prior is known about the visual appearance of each team. Our work builds on a Convolutional Neural Network (CNN) to learn a descriptor, namely a pixel-wise embedding vector, that is similar for pixels depicting players from the same team, and dissimilar when pixels correspond to distinct teams. The advantage of this idea is that no per-game learning is needed, allowing efficient team discrimination as soon as the game starts. In principle, the approach follows the associative embedding framework introduced in arXiv:1611.05424 to differentiate instances of objects. Our work is however different in that it derives the embeddings from a lightweight segmentation network and, more fundamentally, because it considers the assignment of the same embedding to unconnected pixels, as required by pixels of distinct players from the same team. Excellent results, both in terms of team labelling accuracy and generalization to new games arenas, have been achieved on panoramic views of a large variety of basketball games involving players interactions and occlusions. This makes our method a good candidate to integrate team separation in many CNN-based sport analytics pipelines.
In recent works dealing with sport video analysis, FCNs have been considered for specific segmentation tasks, including player jersey number extraction @cite_25 , soccer field lines and players segmentation @cite_22 . @cite_5 , a two-steps architecture, inspired by @cite_28 and @cite_26 , is even proposed to extract players bounding-boxes with team labels. The network however needs to be trained on a game-per-game basis, which is impractical for large scale deployment. None of these works is thus able to differentiate player teams without requiring a dedicated training for each game, as proposed in , where a real-time amenable FCN provides the player segmentation mask, as well as a pixel-wise team-discriminant feature vector.
@cite_26 @cite_22 @cite_28 @cite_5 @cite_25
2963840672 2892012605 2474389331 2794012889 2608840289
Abstract: State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification. However, dense prediction and image classification are structurally different. In this work, we develop a new convolutional network module that is specifically designed for dense prediction. The presented module uses dilated convolutions to systematically aggregate multi-scale contextual information without losing resolution. The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage. We show that the presented context module increases the accuracy of state-of-the-art semantic segmentation systems. In addition, we examine the adaptation of image classification networks to dense prediction and show that simplifying the adapted network can increase accuracy. Automatic interpretation of sports games is a major challenge, especially when these sports feature complex players organizations and game phases. This paper describes a bottom-up approach based on the extraction of semantic features from the video stream of the main camera in the particular case of soccer using scene-specific techniques. In our approach, all the features, ranging from the pixel level to the game event level, have a semantic meaning. First, we design our own scene-specific deep learning semantic segmentation network and hue histogram analysis to extract pixel-level semantics for the field, players, and lines. These pixel-level semantics are then processed to compute interpretative semantic features which represent characteristics of the game in the video stream that are exploited to interpret soccer. For example, they correspond to how players are distributed in the image or the part of the field that is filmed. Finally, we show how these interpretative semantic features can be used to set up and train a semantic-based decision tree classifier for major game events with a restricted amount of training data. The main advantages of our semantic approach are that it only requires the video feed of the main camera to extract the semantic features, with no need for camera calibration, field homography, player tracking, or ball position estimation. While the automatic interpretation of sports games remains challenging, our approach allows us to achieve promising results for the semantic feature extraction and for the classification between major soccer game events such as attack, goal or goal opportunity, defense, and middle game. In this paper, we investigate two new strategies to detect objects accurately and efficiently using deep convolutional neural network: 1) scale-dependent pooling and 2) layerwise cascaded rejection classifiers. The scale-dependent pooling (SDP) improves detection accuracy by exploiting appropriate convolutional features depending on the scale of candidate object proposals. The cascaded rejection classifiers (CRC) effectively utilize convolutional features and eliminate negative object proposals in a cascaded manner, which greatly speeds up the detection while maintaining high accuracy. In combination of the two, our method achieves significantly better accuracy compared to other state-of-the-arts in three challenging datasets, PASCAL object detection challenge, KITTI object detection benchmark and newly collected Inner-city dataset, while being more efficient. Abstract Vision-based player detection and classification are important in sports applications. Accuracy, efficiency, and low memory consumption are desirable for real-time tasks such as intelligent broadcasts and event classification. In this paper, we present a convolutional neural network (CNN) that satisfies all these requirements. The network contains a three-branch proposal network and a four-cascade classification network. Our method first trains these cascaded networks from labeled image patches. Then, we efficiently apply the network to a whole image by using a dilation strategy in testing. We conducted experiments on soccer, basketball, ice hockey and pedestrian datasets. Experimental results demonstrate that our method can accurately detect players under challenging conditions. Compared with CNNs that are adapted from general object detection networks such as Faster-RCNN, our approach achieves state-of-the-art accuracy on three types of games (basketball, soccer and ice hockey) with 1000 × fewer parameters. The generality of our method is also demonstrated on a standard pedestrian detection dataset in which our method achieves competitive performance compared with state-of-the-art methods. Abstract Identifying players in soccer videos is a challenging task, especially in overview shots. Face recognition is not feasible due to low resolution, and jersey number recognition suffers from low resolution, motion blur and unsuitable player pose. Therefore, a method to improve visual identification using spatial constellations is proposed here. This method models a spatial constellation as a histogram over relative positions among all players of the team. Using constellation features might increase identification performance but is not expected to work well as a single mean of identification. Thus, this constellation-based recognition is combined with jersey number recognition using convolutional neural networks. Recognizing the numbers on a player’s shirt is the most straight-forward approach, as there is a direct mapping between numbers and players. Using spatial constellation as a feature for identification is based on the assumption that players do not move randomly over the pitch. Players rather follow a tactical role such as central defender, winger, forward, etc. However in soccer, players do not strictly adhere to these roles, variations occur more or less frequently. By learning constellation models for each player, we avoid a categorical assignment of a player to one single tactical role and therefore incorporate each player’s typical behaviour in terms of switching positions. The presented player identification process is expressed as an assignment problem. Here, an optimal assignment of manually labeled trajectories to known player identities is calculated. Using an assignment problem allows for a flexible fusion of constellation features and jersey numbers by combining the respective cost matrices. Evaluation is performed on 14 different shots of six different Bundesliga matches. By combining both modalities, the accuracy is improved from 0.69 to 0.82 when compared with jersey number recognition only.