Thanks to the advances in the technology of low-cost digital cameras and the popularity of the self-recording culture, the amount of visual data on the Internet is going to the opposite side of the available time and patience of the users. Thus, most of the uploaded videos are doomed to be forgotten and unwatched in a computer folder or website. In this work, we address the problem of creating smooth fast-forward videos without losing the relevant content. We present a new adaptive frame selection formulated as a weighted minimum reconstruction problem, which combined with a smoothing frame transition method accelerates first-person videos emphasizing the relevant segments and avoids visual discontinuities. The experiments show that our method is able to fast-forward videos to retain as much relevant information and smoothness as the state-of-the-art techniques in less time. We also present a new 80-hour multimodal (RGB-D, IMU, and GPS) dataset of first-person videos with annotations for recorder profile, frame scene, activities, interaction, and attention1.
Although not focused on the creation of hyperlapses, Yao al @cite_6 present a highlight-driven summarization approach that generates skimming and timelapse videos as summaries from first-person videos. They assign scores to the video segments by using late fusion of spatial and temporal deep convolution neural networks (DCNNs). The segments with higher scores are selected as video highlights. For the video timelapse, they calculate proper speedup rates such that the summary is compressed in the non-highlight segments and expanded in highlight segments. It is noteworthy that timelapse videos do not handle the suavity constraint that is a mandatory requirement for hyperlapse videos. Differently from the aforementioned work, our approach optimizes semantic, length and smoothness to create semantic hyperlapses. Most importantly, it keeps the path taken by the recorder avoiding to lose the flow of the story and thus, conveying the full message from the original video in a shorter and smoother version.
@cite_6
2467794422
The emergence of wearable devices such as portable cameras and smart glasses makes it possible to record life logging first-person videos. Browsing such long unstructured videos is time-consuming and tedious. This paper studies the discovery of moments of user's major or special interest (i.e., highlights) in a video, for generating the summarization of first-person videos. Specifically, we propose a novel pairwise deep ranking model that employs deep learning techniques to learn the relationship between high-light and non-highlight video segments. A two-stream network structure by representing video segments from complementary information on appearance of video frames and temporal dynamics across frames is developed for video highlight detection. Given a long personal video, equipped with the highlight detection model, a highlight score is assigned to each segment. The obtained highlight segments are applied for summarization in two ways: video time-lapse and video skimming. The former plays the highlight (non-highlight) segments at low (high) speed rates, while the latter assembles the sequence of segments with the highest scores. On 100 hours of first-person videos for 15 unique sports categories, our highlight detection achieves the improvement over the state-of-the-art RankSVM method by 10.5 in terms of accuracy. Moreover, our approaches produce video summary with better quality by a user study from 35 human subjects.