Bullet-screen is a technique that enables the website users to send real-time comment bullet' cross the screen. Compared with the traditional review of a video, bullet-screen provides new features of feeling expression to video watching and more iterations between video viewers. However, since all the comments from the viewers are shown on the screen publicly and simultaneously, some low-quality bullets will reduce the watching enjoyment of the users. Although the bullet-screen video websites have provided filter functions based on regular expression, bad bullets can still easily pass the filter through making a small modification. In this paper, we present SmartBullets, a user-centered bullet-screen filter based on deep learning techniques. A convolutional neural network is trained as the classifier to determine whether a bullet need to be removed according to its quality. Moreover, to increase the scalability of the filter, we employ a cloud-assisted framework by developing a backend cloud server and a front-end browser extension. The evaluation of 40 volunteers shows that SmartBullets can effectively remove the low-quality bullets and improve the overall watching experience of viewers.
Inspired by the rapidly increased social media impact of bullet-screen videos, there are more and more studies related to danmaku proposed. @cite_8 analyzed the comment distribution of bullets over natural time and discovers the burst patterns of danmaku system. @cite_23 designed a new application that extracts time-sync tags for video shots by automatically exploiting bullet comments of the video. After that, Lv al proposed T-DSSM, a temporal deep structured semantic model which can represent bullet-screen comment into semiotic vectors @cite_26 . T-DSSM is further used to label highlight shots in videos. Chen al took advantage of the real-time property of bullets and proposed a personalized keyframe recommendation system @cite_1 . In 2016, He al made use of danmaku to predict the popularity of a videos @cite_11 . On the other hand, Chen al employed deep learning model that trained by a bullet-screen comment dataset to predict the attractiveness of fine-grained videos @cite_6 . Other research related to danmaku in recent years can be found in @cite_15 @cite_14 @cite_12 ping2017video .
@cite_26 @cite_14 @cite_8 @cite_1 @cite_6 @cite_23 @cite_15 @cite_12 @cite_11
2562476021 2749929993 2751445731 2506342487
Recent years have witnessed the boom of online sharing media contents, which raise significant challenges in effective management and retrieval. Though a large amount of efforts have been made, precise retrieval on video shots with certain topics has been largely ignored. At the same time, due to the popularity of novel time-sync comments, or so-called "bullet-screen comments", video semantics could be now combined with timestamps to support further research on temporal video labeling. In this paper, we propose a novel video understanding framework to assign temporal labels on highlighted video shots. To be specific, due to the informal expression of bullet-screen comments, we first propose a temporal deep structured semantic model (T-DSSM) to represent comments into semantic vectors by taking advantage of their temporal correlation. Then, video highlights are recognized and labeled via semantic vectors in a supervised way. Extensive experiments on a real-world dataset prove that our framework could effectively label video highlights with a significant margin compared with baselines, which clearly validates the potential of our framework on video understanding, as well as bullet-screen comments interpretation. DanMu, an emerging type of user-generated comment, has become increasingly popular in recent years. Many online video platforms such as Tudou.com have provided the DanMu function. Unlike traditional online reviews such as reviews at Youtube.com that are outside the videos, DanMu is a scrolling marquee comment, which is overlaid directly on top of the video and synchronized to a specific playback time. Such comments are displayed as streams of moving subtitles overlaid on the video screen. Viewers could easily write DanMus while watching videos, and the written DanMus will be immediately overlaid onto the video and displayed to writers themselves and other viewers as well. Such DanMu systems have greatly enabled users to communicate with each other in a much more direct way, creating a real-time sharing experience. Although there are several unique features of DanMu and has had a great impact on online video systems, to the best of our knowledge, there is no work that has provided a comprehensive study on DanMu. In this article, as a pilot study, we analyze the unique characteristics of DanMu from various perspectives. Specifically, we first illustrate some unique distributions of DanMus by comparing with traditional reviews (TReviews) that we collected from a real DanMu-enabled online video system. Second, we discover two interesting patterns in DanMu data: a herding effect and multiple-burst phenomena that are significantly different from those in TRviews and reveal important insights about the growth of DanMus on a video. Towards exploring antecedents of both th herding effect and multiple-burst phenomena, we propose to further detect leading DanMus within bursts, because those leading DanMus make the most contribution to both patterns. A framework is proposed to detect leading DanMus that effectively combines multiple factors contributing to leading DanMus. Based on the identified characteristics of DanMu, finally we propose to predict the distribution of future DanMus (i.e., the growth of DanMus), which is important for many DanMu-enabled online video systems, for example, the predicted DanMu distribution could be an indicator of video popularity. This prediction task includes two aspects: One is to predict which videos future DanMus will be posted for, and the other one is to predict which segments of a video future DanMus will be posted on. We develop two sophisticated models to solve both problems. Finally, intensive experiments are conducted with a real-world dataset to validate all methods developed in this article. 3-D convolutional neural networks (3-D-convNets) have been very recently proposed for action recognition in videos, and promising results are achieved. However, existing 3-D-convNets has two “artificial” requirements that may reduce the quality of video analysis: 1) It requires a fixed-sized (e.g., 112 @math 112) input video; and 2) most of the 3-D-convNets require a fixed-length input (i.e., video shots with fixed number of frames). To tackle these issues, we propose an end-to-end pipeline named Two-stream 3-D-convNet Fusion , which can recognize human actions in videos of arbitrary size and length using multiple features. Specifically, we decompose a video into spatial and temporal shots. By taking a sequence of shots as input, each stream is implemented using a spatial temporal pyramid pooling (STPP) convNet with a long short-term memory (LSTM) or CNN-E model, softmax scores of which are combined by a late fusion. We devise the STPP convNet to extract equal-dimensional descriptions for each variable-size shot, and we adopt the LSTM CNN-E model to learn a global description for the input video using these time-varying descriptions. With these advantages, our method should improve all 3-D CNN-based video analysis methods. We empirically evaluate our method for action recognition in videos and the experimental results show that our method outperforms the state-of-the-art methods (both 2-D and 3-D based) on three standard benchmark datasets (UCF101, HMDB51 and ACT datasets). Recent years have witnessed the prosperity of a new type of real-time user-generated comment, or so-called DanMu, in many recent online video platforms. These DanMu-enabled video platforms present scrolling marquee comments overlaid directly on top of the videos by synchronizing these comments to specific playback times. In this paper, we study the prediction of video popularity in these platforms, which may benefit a lot of applications ranging from online advertising for website holders to popular video recommendation for audiences. Different from traditional online video platforms where only traditional reviews are available, these DanMus make viewers easily see other viewers' opinions and communicate with each other in a much more direct way. Consequently, viewers are easily influenced by others' behaviors over time, which is considered as the herding effect in social science. However, how to address the unique characteristics i.e., the herding effect of DanMu-enabled online videos for more accurate popularity prediction is still under-explored. To that end, in this paper, we first explore and measure the herding effect of DanMu-enabled video popularity from multiple aspects, including the popular videos, the popular DanMus and the newly updated videos. Also, we recognize that the uploaders' influence and video quality affect the video popularity as well. Along this line, we propose a model that incorporates the herding effect, uploaders' influence and video quality for predicting the video popularity. An effective estimation method is also proposed. Finally, experimental results on real-world data show that our proposed prediction model improves the prediction accuracy by 47.19i?ź compared to the baselines.