In multiview video systems, multiple cameras generally acquire the same scene from different perspectives, such that users have the possibility to select their preferred viewpoint. This results in large amounts of highly redundant data, which needs to be properly handled during encoding and transmission over resource-constrained channels. In this work, we study coding and transmission strategies in multicamera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the 3D scene reconstruction, and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies. More in details, we show the gain offered by our dynamic scheduling policy compared to static camera allocation strategies and to schemes with constant coding strategies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems.
Finally, in our scheduling optimization we aim at minimizing the experienced distortion as well as the temporal variations of the experienced distortion for interactive users. Recently, it has been shown the importance of studying the temporal quality variations in adaptive streaming strategies @cite_9 @cite_12 . These works target single view adaptive streaming over HTTP. We follow similar intuitions and extend the mixed objective function composed on both the perceived distortion and the temporal distortion variation to multiview video navigation applications.
@cite_9 @cite_12
2142876645 2013294225 2111229404 2057415916 1967330816 2150396122 2098033563 2906364736 2357930737 2769327901 2753909025 2748103247 2064793449 2074883210 2121827745 2131723117 2766773582 2337116907 2120848596 2122883255
An important issue of supporting multi-user video streaming over wireless networks is how to optimize the systematic scheduling by intelligently utilizing the available network resources while, at the same time, to meet each video's Quality of Service (QoS) requirement. In this work, we study the problem of video streaming over multi-channel multi-radio multihop wireless networks, and develop fully distributed scheduling schemes with the goals of minimizing the video distortion and achieving certain fairness. We first construct a general distortion model according to the network?s transmission mechanism, as well as the rate distortion characteristics of the video. Then, we formulate the scheduling as a convex optimization problem, and propose a distributed solution by jointly considering channel assignment, rate allocation, and routing. Specifically, each stream strikes a balance between the selfish motivation of minimizing video distortion and the global performance of minimizing network congestions. Furthermore, we extend the proposed scheduling scheme by addressing the fairness problem. Unlike prior works that target at users' bandwidth or demand fairness, we propose a media-aware distortion-fairness strategy which is aware of the characteristics of video frames and ensures max-min distortion-fairness sharing among multiple video streams. We provide extensive simulation results which demonstrate the effectiveness of our proposed schemes. This paper presents a rate-distortion (RD) optimized interactive streaming method for multiview video pre-compressed by H.264 Joint Multiview Video Model (JMVM). In the proposed method, multiple encodings are first used to facilitate the flexible server-client interaction. Second, a RD-optimized scheduling strategy is provided to guarantee the optimal view-dependent delivery of multiview video. In the RD-optimized scheduling strategy, a distortion model is proposed to estimate the expected end-to-end distortion by accounting for both coding and packet-loss-induced distortions, as well as rendering-induced distortion. With the end-to-end distortion model, the server can select the optimal encoding combination for transmission. Experimental results demonstrate that the proposed method can achieve a significant end-to-end RD performance improvement over the selective streaming methods with simulcast coding or scalable multiview coding. In addition, it has better error-resilience performance to combat with packet-losses over the Internet protocol (IP) networks. We present a novel client-driven multiview video streaming system that allows a user to watch 3D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his her head position. The user's head position is tracked and predicted into the future to select the views that best match the user's current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be prefetched in order to account for delays due to network transport and stream switching. The system allocates more bandwidth to the selected views in order to render the current viewing angle. Highly compressed, lower quality versions of some other views are also prefetched for concealment if the current user viewpoint differs from the predicted viewpoint. An objective measure based on the abruptness of the head movements and delays in the system is introduced to determine the number of additional lower quality views to be prefetched. The proposed system makes use of multiview coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compression efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions. We consider the design of a scheduling policy for video streaming in a wireless network formed by several users and helpers (e.g., base stations). In such networks, any user is typically in the range of multiple helpers. Hence, an efficient policy should allow the users to dynamically select the helper nodes to download from and determine adaptively the quality level of the requested video segment. In order to obtain a tractable formulation, we follow a “divide and conquer” approach. First, we formulate a network utility maximization (NUM) problem where the network utility function is a concave and component-wise nondecreasing function of the time-averaged users' requested video quality index, and maximization is subject to the stability of all queues in the system. Second, we solve the NUM problem by using a Lyapunov drift plus penalty approach, obtaining a dynamic adaptive scheme that decomposes into two building blocks: 1) adaptive video quality and helper selection (run at the user nodes); and 2) dynamic allocation of the helper-to-user transmission rates (run at the help nodes). Our solution provably achieves NUM optimality in a strong per-sample path sense (i.e., without assumptions of stationarity and ergodicity). Third, we observe that, since all queues in the system are stable, all requested video chunks shall be eventually delivered. Fourth, in order to translate the requested video quality into the effective video quality at the user playback, it is necessary that the chunks are delivered within their playback deadline. This requires that the largest delay among all queues at the helpers serving any given user is less than the pre-buffering time of that user at its streaming session startup phase. In order to achieve this condition with high probability, we propose an effective and decentralized (albeit heuristic) scheme to adaptively calculate the pre-buffering and re-buffering time at each user. In this way, the system is forced to work in the “smooth streaming regime,” i.e., in the regime of very small playback buffer underrun rate. Through simulations, we evaluate the performance of the proposed algorithm under realistic assumptions of a network with densely deployed helper and user nodes, including user mobility, variable bit-rate video coding, and users joining or leaving the system at arbitrary times. Internet video makes up a significant part of the Internet traffic and its fraction is constantly growing. In order to guarantee best user experience throughout different network access technologies with dynamically varying network conditions, it is fundamental to adopt technologies enabling a proper delivery of the media content. One of such technologies is adaptive streaming. It allows to dynamically adapt the bit-rate of the stream to varying network conditions. There are various approaches to adaptive streaming. In our work, we focus on the receiver-driven approach where the media file is subdivided into segments, each of the segments is provided at multiple bit-rates, and the task of the client is to select the appropriate bit-rate for each of the segments. With this approach, the challenges are (i) to properly estimate the dynamics of the available network throughput, (ii) to control the filling level of the client buffer in order to avoid underflows resulting in playback interruptions, (iii) to maximize the quality of the stream, while avoiding unnecessary quality fluctuations, and, finally, (iv) to minimize the delay between the user's request and the start of the playback. During our work, we designed and implemented a receiver-driven adaptation algorithm for adaptive streaming that does not rely on cross-layer information or server assistance. We integrated the algorithm with a prototype implementation of a streaming client based on the MPEG DASH (Dynamic Adaptive Streaming over HTTP) standard. We evaluated the implemented prototype in real-world scenarios and found that it performes remarkably well even under challenging network conditions. Further, it exhibits stable and fair operation if a common link is shared among multiple clients. Managing the Quality-of-Experience (QoE) of video streaming for wireless clients is becoming increasingly important due to the rapid growth of video traffic on wireless networks. The inherent variability of the wireless channel as well as the Variable Bit Rate (VBR) of the compressed video streams make QoE management a challenging problem. Prior work has studied this problem in the context of transmitting a single video stream. In this paper, we investigate multiplexing schemes to transmit multiple video streams from a base station to mobile clients that use number of playout stalls as a performance metric. In this context, we present an epoch-by-epoch framework to fairly allocate wireless transmission slots to streaming videos. In each epoch our scheme essentially reduces the vulnerability to stalling by allocating slots to videos in a way that maximizes the minimum ‘playout lead’ across all videos. Next, we show that the problem of allocating slots fairly is NP-complete even for a constant number of videos. We then present a fast lead-aware greedy algorithm for the problem. Our choice of greedy algorithm is motivated by the fact that this algorithm is optimal when the channel quality of a user remains unchanged within an epoch (but different users may experience different channel quality). Moreover, our experimental results based on public MPEG-4 video traces and wireless channel traces that we collected from a WiMAX test-bed show that the lead-aware greedy approach performs a fair distribution of stalls across the clients when compared to other algorithms, while still maintaining similar or lower average number of stalls per client. We derive an optimization framework for joint view and rate scalable coding of multi-view video content represented in the texture plus depth format. The optimization enables the sender to select the subset of coded views and their encoding rates such that the aggregate distortion over a continuum of synthesized views is minimized. We construct the view and rate embedded bitstream such that it delivers optimal performance simultaneously over a discrete set of transmission rates. In conjunction, we develop a user interaction model that characterizes the view selection actions of the client as a Markov chain over a discrete state-space. We exploit the model within the context of our optimization to compute user-action-driven coding strategies that aim at enhancing the client's performance in terms of latency and video quality. Our optimization outperforms the state-of-the-art H.264 SVC codec as well as a multi-view wavelet-based coder equipped with a uniform rate allocation strategy, across all scenarios studied in our experiments. Equally important, we can achieve an arbitrarily fine granularity of encoding bit rates, while providing a novel functionality of view embedded encoding, unlike the other encoding methods that we examined. Finally, we observe that the interactivity-aware coding delivers superior performance over conventional allocation techniques that do not anticipate the client's view selection actions in their operation. In this survey, we present state-of-the-art bitrate adaptation algorithms for HTTP adaptive streaming (HAS). As a key distinction from other streaming approaches, the bitrate adaptation algorithms in HAS are chiefly executed at each client, i.e. , in a distributed manner. The objective of these algorithms is to ensure a high quality of experience (QoE) for viewers in the presence of bandwidth fluctuations due to factors like signal strength, network congestion, network reconvergence events, etc. While such fluctuations are common in public Internet, they can also occur in home networksor even managed networks where there is often admission control and QoS tools. Bitrate adaptation algorithms may take factors like bandwidth estimations, playback buffer fullness, device features, viewer preferences, and content features into account, albeit with different weights. Since the viewer’s QoE needs to be determined in real-time during playback, objective metrics are generally used including number of buffer stalls, duration of startup delay, frequency and amount of quality oscillations, and video instability. By design, the standards for HAS do not mandate any particular adaptation algorithm, leaving it to system builders to innovate and implement their own method. This survey provides an overview of the different methods proposed over the last several years. Although HTTP-based video streaming can easily penetrate firewalls and profit from Web caches, the underlying TCP may introduce large delays in case of a sudden capacity loss. To avoid an interruption of the video stream in such cases we propose the Backward-Shifted Coding (BSC). Based on Scalable Video Coding (SVC), BSC adds a time-shifted layer of redundancy to the video stream such that future frames are downloaded at any instant. This pre-fetched content maintains a fluent video stream even under highly variant network conditions and leads to high Quality of Experience (QoE). We characterize this QoE gain by analyzing initial buffering time, re-buffering time and content resolution using the Ballot theorem. The probability generating functions of the playback interruption and of the initial buffering latency are provided in closed form. We further compute the quasi-stationary distribution of the video quality, in order to compute the average quality, as well as temporal variability in video quality. Employing these analytic results to optimize QoE shows interesting trade-offs and video streaming at outstanding fluency. HTTP Adaptive Streaming is a successful and largely adopted content delivery technology. Yet poor bandwidth prediction, notably in mobile networks, may cause bit-rate oscillations, increased segment delivery delays, video freezes, and may thus negatively impact the end user quality of experience. To address this issue, we propose to exploit the stream prioritization and termination features of the HTTP 2 protocol to achieve video frame filtering and scheduling, so as to maximize the amount of video data received on time by the client. We evaluate with optimal scheduling and filtering algorithms the maximum gain we may expect from such delivery schemes where video frames are carried in dedicated HTTP 2 streams. Evaluation shows that our HTTP 2-based video frame scheduling scheme brings benefits for video quality. In recent years, quality of experience (QoE) has been investigated and proved to have both influential factors on user's visual quality and perceptual quality, while the perceptual quality means user's requirement on personalized content should be acquired in optimized quality. That's to say, those segments holding user interested content such as highlights need to be allocated more network resource in a resource-limited streaming scenario. However, all the existing HTTP-based adaptive methods only focus the content-agnostic bitrate adaptation according to limited network resources or energy resource, since they ignored user perceived semantics on some important segments, which suffered less quality on the important segments than on those ordinary ones, so as to hurt the overall QoE. In this paper, we have proposed a new semantic-aware adaptation scheme for MPEG-DASH services, which decides how to preserve bandwidth and buffering time depending on content descriptors for the perceived important content to users. Further, a semantic-aware probe and adaptation (SMA-PANDA) algorithm has been implemented in a DASH client to compare with conventional bitrate adaptions. Preliminary results show that SMA-PANDA achieves better QoE and flexibility on streaming user's interested content on MPEG-DASH platform, and it also aggressively helps user interested content compete more resource to deliver high quality presentation. A continuous evaluation of the end user’s quality-of-experience (QoE) is essential for efficient video streaming. This is crucial for networks with constrained resources that offer time-varying channel quality to its users. In hypertext transfer protocol-based video streaming, the QoE is measured by quantifying the perceptual impact of distortions caused by rate adaptation or interruptions in playback due to rebuffering events. The resulting impact on the QoE due to these distortions has been studied individually in the literature. However, the QoE is determined by an interplay of these distortions, and therefore necessitates a combined study of them. To the best of our knowledge, there is no publicly available database that studies these distortions jointly on a continuous time basis. In this paper, our contributions are twofold. First, we present a database consisting of videos at full high definition and ultrahigh definition resolutions. We consider various levels of rate adaptation and rebuffering distortions together in these videos as experienced in a typical realistic setting. A subjective evaluation of these videos is conducted on a continuous time scale. Second, we present a QoE evaluation framework comprising a learning-based model during playback and an exponential model during rebuffering. Furthermore, we perform an objective evaluation of popular video quality assessment and continuous time QoE metrics over the constructed database. The objective evaluation study demonstrates that the performance of the proposed QoE model is superior to that of the objective metrics. The database is publicly available for download at http: www.iith.ac.in lfovia downloads.html . In conventional HTTP-based adaptive streaming (HAS), a video source is encoded at multiple levels of constant bitrate representations, and a client makes its representation selections according to the measured network bandwidth. While greatly simplifying adaptation to the varying network conditions, this strategy is not the best for optimizing the video quality experienced by end users. Quality fluctuation can be reduced if the natural variability of video content is taken into consideration. In this work, we study the design of a client rate adaptation algorithm to yield consistent video quality. We assume that clients have visibility into incoming video within a finite horizon. We also take advantage of the client-side video buffer, by using it as a breathing room for not only network bandwidth variability, but also video bitrate variability. The challenge, however, lies in how to balance these two variabilities to yield consistent video quality without risking a buffer underrun. We propose an optimization solution that uses an online algorithm to adapt the video bitrate step-by-step, while applying dynamic programming at each step. We incorporate our solution into PANDA -- a practical rate adaptation algorithm designed for HAS deployment at scale. It is generally recognized that severe video distortions that are transient in space and or time have a large effect on overall perceived video quality. In order to understand this phenomena, we study the distribution of spatio-temporally local quality scores obtained from several video quality assessment (VQA) algorithms on videos suffering from compression and lossy transmission over communication channels. We propose a content adaptive spatial and temporal pooling strategy based on the observed distribution. Our method adaptively emphasizes “worst” scores along both the spatial and temporal dimensions of a video sequence and also considers the perceptual effect of large-area cohesive motion flow such as egomotion. We demonstrate the efficacy of the method by testing it using three different VQA algorithms on the LIVE Video Quality database and the EPFL-PoliMI video quality database. We propose a new scheme for streaming media systems that combines adaptive media playout (AMP) with rate-distortion (R-D)optimized packet transmission. AMP, the client-controlled, adaptive modification of the media playout rate, allows us to flexibly adjust the playout deadlines of individual packets, and can therefore reduce reconstruction distortion. This added flexibility incurs a subjective cost, however. We introduce functions that assess the subjective cost of a schedule of playout rate modifications, and we show how to optimize the schedule with respect to these costs and distortion. Because the optimal playout rate schedule and the R-D optimal transmission schedule are interdependent, we solve the two problems jointly. In simulations that model a receiver-driven scenario, results for a short media clip show a more than 2 dB improvement in mean PSNR for R-D optimized transmission scheduling combined with a moderate amount of AMP, over R-D optimized transmission scheduling alone. While much of multiview video coding focuses on the rate-distortion performance of compressing all frames of all views for storage or non-interactive video delivery over networks, we address the problem of designing a frame structure to enable interactive multiview streaming, where clients can interactively switch views during video playback. Thus, as a client is playing back successive frames (in time) for a given view, it can send a request to the server to switch to a different view while continuing uninterrupted temporal playback. Noting that standard tools for random access (i.e., I-frame insertion) can be bandwidth-inefficient for this application, we propose a redundant representation of I-, P-, and “merge” frames, where each original picture can be encoded into multiple versions, appropriately trading off expected transmission rate with storage, to facilitate view switching. We first present ad hoc frame structures with good performance when the view-switching probabilities are either very large or very small. We then present optimization algorithms that generate more general frame structures with better overall performance for the general case. We show in our experiments that we can generate redundant frame structures offering a range of tradeoff points between transmission and storage, e.g., outperforming simple I-frame insertion structures by up to 45 in terms of bandwidth efficiency at twice the storage cost. Over-the-top mobile adaptive video streaming is invariably influenced by volatile network conditions, which can cause playback interruptions (stalling or rebuffering events) and bitrate fluctuations, thereby impairing users’ quality of experience (QoE). Video quality assessment models that can accurately predict users’ QoE under such volatile network conditions are rapidly gaining attention, since these methods could enable more efficient design of quality control protocols for media-driven services such as YouTube, Amazon, Netflix, and many others. However, the development of improved QoE prediction models requires data sets of videos afflicted with diverse stalling events that have been labeled with ground-truth subjective opinion scores. Toward this end, we have created a new mobile video quality database that we call LIVE Mobile Stall Video Database-II. Our database contains a total of 174 videos afflicted with distortions caused by 26 different stalling patterns. We describe the way we simulated the diverse stalling events to create a corpus of distorted videos, and we detail the human study we conducted to obtain continuous-time subjective scores from 54 subjects. We also present the outcomes of our comprehensive analysis of the impact of several factors that influence subjective QoE, and report the performance of existing QoE-prediction models on our data set. We are making the database (videos, subjective data, and video metadata) publicly available in order to help the advance state-of-the-art research on user-centric mobile network planning and management. The database may be accessed at http: live.ece.utexas.edu research LIVEStallStudy liveMobile.html . In recent years, empowered by rich media generation devices and convenient Internet access, Crowdsourced Live Game Video Streaming (CLGVS) has become one of the most popular Internet services. Twitch.tv, the most well-known CLGVS platform in the world, allows gamers to broadcast their gaming videos over the Internet. With the prevalence of mobile devices, viewers can watch gamers playing video games anywhere, anytime, on any devices (e.g., smartphones, tablets, or personal computers). However, the heterogeneity of user devices makes conventional solutions hard to ensure user-perceived quality. In this paper, we address the problem of cost-effective adaptive live game video streaming from the perspective of CLGVS service providers. Our purpose is to minimize the operational cost for CLGVS service providers by making live transcoding decisions, bit-rate adaptation decisions, and datacenter assignment decisions dynamically. Meanwhile, our algorithm also ensures good-enough service quality for viewers. Due to the diversity of game genres, we also consider game genres when designing our algorithm. To achieve the above purpose, we formulate the problem into a constrained stochastic optimization problem. By leveraging the Lyapunov optimization framework, we derive the online strategy with provable performance bound. To evaluate the effectiveness of our proposed algorithm, we further conduct a series of trace-driven simulations. The experimental results demonstrate the effectiveness of our algorithm in terms of operational cost and service quality. Our proposed algorithm can reduce operational cost by up to 50 while achieving good-enough viewer QoE compared with other alternatives. Multi-user video streaming over wireless channels is a challenging problem, where the demand for better video quality and small transmission delays needs to be reconciled with the limited and often time-varying communication resources. This paper presents a framework for joint network optimization, source adaptation, and deadline-driven scheduling for multi-user video streaming over wireless networks. We develop a joint adaptation, resource allocation and scheduling (JARS) algorithm, which allocates the communication resource based on the video users' quality of service, adapts video sources based on smart summarization, and schedules the transmissions to meet the frame delivery deadlines. The proposed algorithm leads to near full utilization of the network resources and satisfies the delivery deadlines for all video frames. Substantial performance improvements are achieved compared with heuristic schemes that do not take the interactions between multiple users into consideration. In this paper, we concentrate on opportunistic scheduling for multicast information. We pose the problem as a multicast throughput optimization problem. As a solution we present how one can jointly utilize fixed-rate and rateless erasure coding along with simple rate adaptation techniques in order to achieve the optimal multicast throughput per user. We first investigate the performance of the proposed system under i.i.d. channel conditions. Our analysis shows a linear gain for the multicast capacity over i.i.d. Rayleigh fading channels with respect to the number of users. Since the established results require coding over large number of blocks and hence induce large decoding delays, we extend our analysis to the cases where we code over shorter block lengths and thus quantify the delay-capacity tradeoffs under a simple setting. We further look into non-i.i.d. channel conditions and show achievable gains by modifying a scheduling heuristic whose fairness is well- established for opportunistic scheduling of unicast flows. Our overall evaluations demonstrate that under both i.i.d. and non-i.i.d. channel conditions, opportunistic multicasting with erasure coding can significantly improve the performance over the traditional techniques used in today's communication systems.