Adaptive bitrate (ABR) streaming is the de facto solution for achieving smooth viewing experiences under unstable network conditions. However, most of the existing rate adaptation approaches for ABR are content-agnostic, without considering the semantic information of the video content. Nevertheless, semantic information largely determines the informativeness and interestingness of the video content, and consequently affects the QoE for video streaming. One common case is that the user may expect higher quality for the parts of video content that are more interesting or informative so as to reduce overall subjective quality loss. This creates two main challenges for such a problem: First, how to determine which parts of the video content are more interesting? Second, how to allocate bitrate budgets for different parts of the video content with different significances? To address these challenges, we propose a Content-of-Interest (CoI) based rate adaptation scheme for ABR. We first design a deep learning approach for recognizing the interestingness of the video content, and then design a Deep Q-Network (DQN) approach for rate adaptation by incorporating video interestingness information. The experimental results show that our method can recognize video interestingness precisely, and the bitrate allocation for ABR can be aligned with the interestingness of video content while not compromising the performances on objective QoE metrics.
Cavallaro @cite_18 showed that the use of semantic video analysis prior to encoding for adaptive content delivery reduces bandwidth requirements. Hu @cite_6 proposed a semantics-aware adaptation scheme for ABR streaming by semantic analysis for soccer video. Fan @cite_5 utilized various features collected from streaming services to determine if a video segment attracts viewers for optimizing live game streaming. Dong @cite_13 designed a personalized emotion-aware video streaming system based on the user's emotional status. In this line of works, they considered different subjective factors for optimizing video streaming services to improve QoE.
@cite_5 @cite_18 @cite_13 @cite_6
2541487934 2160881085 2327187098 2753909025
Live game streaming is tremendously popular, and recent reports indicate that such platforms impose high traffic volume, leading to degraded user experience. In this paper, we propose a Segment-of-Interest (SoI) driven platform, so as to optimize live game streaming. Our platform uses various features collected from streamers and viewers to determine if the current segments of gameplays attract viewers. Upon determining the importance of individual segments, the limited bandwidth is allocated to the interested viewers in a Rate-Distortion (R-D) optimized manner, where the levels of segment importance are used as weights of game streaming quality. The underlaying intuition is: viewer experience is degraded only when the game streaming degradation is noticed by viewers. Simulation results show the benefits of our proposed solution: (i) it improves viewing quality by up to 5 dB, (ii) it saves bandwidth by up to 50 Gbps, and (iii) it efficiently performs resource allocation and scales to many viewers. Our presented testbed is opensource and can be leveraged by researchers and engineers to further improve live game streaming platforms. We present an encoding framework which exploits semantics for video content delivery. The video content is organized based on the idea of main content message. In the work reported in this paper, the main content message is extracted from the video data through semantic video analysis, an application-dependent process that separates relevant information from non relevant information. We use here semantic analysis and the corresponding content annotation under a new perspective: the results of the analysis are exploited for object-based encoders, such as MPEG-4, as well as for frame-based encoders, such as MPEG-1. Moreover, the use of MPEG-7 content descriptors in conjunction with the video is used for improving content visualization for narrow channels and devices with limited capabilities. Finally, we analyze and evaluate the impact of semantic video analysis in video encoding and show that the use of semantic video analysis prior to encoding sensibly reduces the bandwidth requirements compared to traditional encoders not only for an object-based encoder but also for a frame-based encoder. As a useful tool for improving the user’s quality of experience (QoE), delay announcement has received substantial attention recently. However, how to make a simple and efficient delay announcement in the cloud mobile media environment is still an open and challenging problem. Unfortunately, traditional convex and stochastic optimization-based methods cannot address this issue due to the subjective user response with respect to the announced delay. To resolve this problem, this paper analytically studies the characteristics of delay announcement by analyzing the components of the user response and designs a QoE-driven delay announcement scheme by establishing an objective user response function. On the methodology end, the user response associated with the announced delay is approximated in the framework of fluid model, where the interaction between the system performance and delay announcement is well described by a series of mathematical functions. On the technology end, this paper develops a novel state-dependent announcement scheme that is more reliable than the other competing ones and can improve the user’s QoE dramatically. Extensive simulation results validate the efficiency of the proposed delay announcement scheme. In recent years, quality of experience (QoE) has been investigated and proved to have both influential factors on user's visual quality and perceptual quality, while the perceptual quality means user's requirement on personalized content should be acquired in optimized quality. That's to say, those segments holding user interested content such as highlights need to be allocated more network resource in a resource-limited streaming scenario. However, all the existing HTTP-based adaptive methods only focus the content-agnostic bitrate adaptation according to limited network resources or energy resource, since they ignored user perceived semantics on some important segments, which suffered less quality on the important segments than on those ordinary ones, so as to hurt the overall QoE. In this paper, we have proposed a new semantic-aware adaptation scheme for MPEG-DASH services, which decides how to preserve bandwidth and buffering time depending on content descriptors for the perceived important content to users. Further, a semantic-aware probe and adaptation (SMA-PANDA) algorithm has been implemented in a DASH client to compare with conventional bitrate adaptions. Preliminary results show that SMA-PANDA achieves better QoE and flexibility on streaming user's interested content on MPEG-DASH platform, and it also aggressively helps user interested content compete more resource to deliver high quality presentation.