Bullet-screen is a technique that enables the website users to send real-time comment bullet' cross the screen. Compared with the traditional review of a video, bullet-screen provides new features of feeling expression to video watching and more iterations between video viewers. However, since all the comments from the viewers are shown on the screen publicly and simultaneously, some low-quality bullets will reduce the watching enjoyment of the users. Although the bullet-screen video websites have provided filter functions based on regular expression, bad bullets can still easily pass the filter through making a small modification. In this paper, we present SmartBullets, a user-centered bullet-screen filter based on deep learning techniques. A convolutional neural network is trained as the classifier to determine whether a bullet need to be removed according to its quality. Moreover, to increase the scalability of the filter, we employ a cloud-assisted framework by developing a backend cloud server and a front-end browser extension. The evaluation of 40 volunteers shows that SmartBullets can effectively remove the low-quality bullets and improve the overall watching experience of viewers.
Inspired by the rapidly increased social media impact of bullet-screen videos, there are more and more studies related to danmaku proposed. @cite_8 analyzed the comment distribution of bullets over natural time and discovers the burst patterns of danmaku system. @cite_23 designed a new application that extracts time-sync tags for video shots by automatically exploiting bullet comments of the video. After that, Lv al proposed T-DSSM, a temporal deep structured semantic model which can represent bullet-screen comment into semiotic vectors @cite_26 . T-DSSM is further used to label highlight shots in videos. Chen al took advantage of the real-time property of bullets and proposed a personalized keyframe recommendation system @cite_1 . In 2016, He al made use of danmaku to predict the popularity of a videos @cite_11 . On the other hand, Chen al employed deep learning model that trained by a bullet-screen comment dataset to predict the attractiveness of fine-grained videos @cite_6 . Other research related to danmaku in recent years can be found in @cite_15 @cite_14 @cite_12 ping2017video .
@cite_26 @cite_14 @cite_8 @cite_1 @cite_6 @cite_23 @cite_15 @cite_12 @cite_11
2562476021 2566760506 2749929993 2740409734 2963834699 2735561960 2295356354 2506342487
Recent years have witnessed the boom of online sharing media contents, which raise significant challenges in effective management and retrieval. Though a large amount of efforts have been made, precise retrieval on video shots with certain topics has been largely ignored. At the same time, due to the popularity of novel time-sync comments, or so-called "bullet-screen comments", video semantics could be now combined with timestamps to support further research on temporal video labeling. In this paper, we propose a novel video understanding framework to assign temporal labels on highlighted video shots. To be specific, due to the informal expression of bullet-screen comments, we first propose a temporal deep structured semantic model (T-DSSM) to represent comments into semantic vectors by taking advantage of their temporal correlation. Then, video highlights are recognized and labeled via semantic vectors in a supervised way. Extensive experiments on a real-world dataset prove that our framework could effectively label video highlights with a significant margin compared with baselines, which clearly validates the potential of our framework on video understanding, as well as bullet-screen comments interpretation. In this paper, we propose a method that estimates new tags from user comments on videos on the Nico Nico Douga website. On Nico Nico Douga, users can post tags and comments on videos. However, users cannot post more than 12 tags on a video, therefore, there are some important tags that could be posted but are sometimes missed. We present a technique to acquire some of these missing tags by choosing new tags that score well in a scoring method developed by us. DanMu, an emerging type of user-generated comment, has become increasingly popular in recent years. Many online video platforms such as Tudou.com have provided the DanMu function. Unlike traditional online reviews such as reviews at Youtube.com that are outside the videos, DanMu is a scrolling marquee comment, which is overlaid directly on top of the video and synchronized to a specific playback time. Such comments are displayed as streams of moving subtitles overlaid on the video screen. Viewers could easily write DanMus while watching videos, and the written DanMus will be immediately overlaid onto the video and displayed to writers themselves and other viewers as well. Such DanMu systems have greatly enabled users to communicate with each other in a much more direct way, creating a real-time sharing experience. Although there are several unique features of DanMu and has had a great impact on online video systems, to the best of our knowledge, there is no work that has provided a comprehensive study on DanMu. In this article, as a pilot study, we analyze the unique characteristics of DanMu from various perspectives. Specifically, we first illustrate some unique distributions of DanMus by comparing with traditional reviews (TReviews) that we collected from a real DanMu-enabled online video system. Second, we discover two interesting patterns in DanMu data: a herding effect and multiple-burst phenomena that are significantly different from those in TRviews and reveal important insights about the growth of DanMus on a video. Towards exploring antecedents of both th herding effect and multiple-burst phenomena, we propose to further detect leading DanMus within bursts, because those leading DanMus make the most contribution to both patterns. A framework is proposed to detect leading DanMus that effectively combines multiple factors contributing to leading DanMus. Based on the identified characteristics of DanMu, finally we propose to predict the distribution of future DanMus (i.e., the growth of DanMus), which is important for many DanMu-enabled online video systems, for example, the predicted DanMu distribution could be an indicator of video popularity. This prediction task includes two aspects: One is to predict which videos future DanMus will be posted for, and the other one is to predict which segments of a video future DanMus will be posted on. We develop two sophisticated models to solve both problems. Finally, intensive experiments are conducted with a real-world dataset to validate all methods developed in this article. Key frames are playing a very important role for many video applications, such as on-line movie preview and video information retrieval. Although a number of key frame selection methods have been proposed in the past, existing technologies mainly focus on how to precisely summarize the video content, but seldom take the user preferences into consideration. However, in real scenarios, people may cast diverse interests on the contents even for the same video, and thus they may be attracted by quite different key frames, which makes the selection of key frames an inherently personalized process. In this paper, we propose and investigate the problem of personalized key frame recommendation to bridge the above gap. To do so, we make use of video images and user time-synchronized comments to design a novel key frame recommender that can simultaneously model visual and textual features in a unified framework. By user personalization based on her his previously reviewed frames and posted comments, we are able to encode different user interests in a unified multi-modal space, and can thus select key frames in a personalized manner, which, to the best of our knowledge, is the first time in the research field of video content analysis. Experimental results show that our method performs better than its competitors on various measures. In recent years, more and more people are like to watch videos online because of its convenience and social features. Due to the limit of entertainment time, there is a new requirement that people prefer to watch some hot video segments rather than an entire video. However, it is a quite time-consuming work to extract the highlight segments in videos manually because the number of videos uploaded to the internet is huge. In this paper, we propose a model of event detection on videos using Time-Sync comments provided by online users. In the model, three features of Time-Sync comments are extracted firstly. Then, user behavior relevance in time series are analyzed to find the video shots that people are interested in most. Metric and its optimization to score video shots for event detection are introduced lastly. Experiments on several movies shows that the events detected by our method coincide with the highlights in the movies. Experiments on movies show that the events detected by our method coincide with the highlights in the movies. Online videos have become indispensable to people's daily lives. Everyday, millions of people watch online videos at video websites such as Nico Nico Douga and they comment on these websites. In Nico Nico Douga, comments are useful for online advertisements and video retrieval methods. Therefore, it is important to annotate comments in detail for them. In this paper, we annotate comments based on referring contents automatically. Recent years have witnessed the prosperity of a new type of real-time user-generated comment, or so-called DanMu, in many recent online video platforms. These DanMu-enabled video platforms present scrolling marquee comments overlaid directly on top of the videos by synchronizing these comments to specific playback times. In this paper, we study the prediction of video popularity in these platforms, which may benefit a lot of applications ranging from online advertising for website holders to popular video recommendation for audiences. Different from traditional online video platforms where only traditional reviews are available, these DanMus make viewers easily see other viewers' opinions and communicate with each other in a much more direct way. Consequently, viewers are easily influenced by others' behaviors over time, which is considered as the herding effect in social science. However, how to address the unique characteristics i.e., the herding effect of DanMu-enabled online videos for more accurate popularity prediction is still under-explored. To that end, in this paper, we first explore and measure the herding effect of DanMu-enabled video popularity from multiple aspects, including the popular videos, the popular DanMus and the newly updated videos. Also, we recognize that the uploaders' influence and video quality affect the video popularity as well. Along this line, we propose a model that incorporates the herding effect, uploaders' influence and video quality for predicting the video popularity. An effective estimation method is also proposed. Finally, experimental results on real-world data show that our proposed prediction model improves the prediction accuracy by 47.19i?ź compared to the baselines.