In multiview video systems, multiple cameras generally acquire the same scene from different perspectives, such that users have the possibility to select their preferred viewpoint. This results in large amounts of highly redundant data, which needs to be properly handled during encoding and transmission over resource-constrained channels. In this work, we study coding and transmission strategies in multicamera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the 3D scene reconstruction, and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies. More in details, we show the gain offered by our dynamic scheduling policy compared to static camera allocation strategies and to schemes with constant coding strategies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems.
Finally, in our scheduling optimization we aim at minimizing the experienced distortion as well as the temporal variations of the experienced distortion for interactive users. Recently, it has been shown the importance of studying the temporal quality variations in adaptive streaming strategies @cite_9 @cite_12 . These works target single view adaptive streaming over HTTP. We follow similar intuitions and extend the mixed objective function composed on both the perceived distortion and the temporal distortion variation to multiview video navigation applications.
@cite_9 @cite_12
2142876645 2013294225 2111229404 2057415916
An important issue of supporting multi-user video streaming over wireless networks is how to optimize the systematic scheduling by intelligently utilizing the available network resources while, at the same time, to meet each video's Quality of Service (QoS) requirement. In this work, we study the problem of video streaming over multi-channel multi-radio multihop wireless networks, and develop fully distributed scheduling schemes with the goals of minimizing the video distortion and achieving certain fairness. We first construct a general distortion model according to the network?s transmission mechanism, as well as the rate distortion characteristics of the video. Then, we formulate the scheduling as a convex optimization problem, and propose a distributed solution by jointly considering channel assignment, rate allocation, and routing. Specifically, each stream strikes a balance between the selfish motivation of minimizing video distortion and the global performance of minimizing network congestions. Furthermore, we extend the proposed scheduling scheme by addressing the fairness problem. Unlike prior works that target at users' bandwidth or demand fairness, we propose a media-aware distortion-fairness strategy which is aware of the characteristics of video frames and ensures max-min distortion-fairness sharing among multiple video streams. We provide extensive simulation results which demonstrate the effectiveness of our proposed schemes. This paper presents a rate-distortion (RD) optimized interactive streaming method for multiview video pre-compressed by H.264 Joint Multiview Video Model (JMVM). In the proposed method, multiple encodings are first used to facilitate the flexible server-client interaction. Second, a RD-optimized scheduling strategy is provided to guarantee the optimal view-dependent delivery of multiview video. In the RD-optimized scheduling strategy, a distortion model is proposed to estimate the expected end-to-end distortion by accounting for both coding and packet-loss-induced distortions, as well as rendering-induced distortion. With the end-to-end distortion model, the server can select the optimal encoding combination for transmission. Experimental results demonstrate that the proposed method can achieve a significant end-to-end RD performance improvement over the selective streaming methods with simulcast coding or scalable multiview coding. In addition, it has better error-resilience performance to combat with packet-losses over the Internet protocol (IP) networks. We present a novel client-driven multiview video streaming system that allows a user to watch 3D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his her head position. The user's head position is tracked and predicted into the future to select the views that best match the user's current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be prefetched in order to account for delays due to network transport and stream switching. The system allocates more bandwidth to the selected views in order to render the current viewing angle. Highly compressed, lower quality versions of some other views are also prefetched for concealment if the current user viewpoint differs from the predicted viewpoint. An objective measure based on the abruptness of the head movements and delays in the system is introduced to determine the number of additional lower quality views to be prefetched. The proposed system makes use of multiview coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compression efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions. We consider the design of a scheduling policy for video streaming in a wireless network formed by several users and helpers (e.g., base stations). In such networks, any user is typically in the range of multiple helpers. Hence, an efficient policy should allow the users to dynamically select the helper nodes to download from and determine adaptively the quality level of the requested video segment. In order to obtain a tractable formulation, we follow a “divide and conquer” approach. First, we formulate a network utility maximization (NUM) problem where the network utility function is a concave and component-wise nondecreasing function of the time-averaged users' requested video quality index, and maximization is subject to the stability of all queues in the system. Second, we solve the NUM problem by using a Lyapunov drift plus penalty approach, obtaining a dynamic adaptive scheme that decomposes into two building blocks: 1) adaptive video quality and helper selection (run at the user nodes); and 2) dynamic allocation of the helper-to-user transmission rates (run at the help nodes). Our solution provably achieves NUM optimality in a strong per-sample path sense (i.e., without assumptions of stationarity and ergodicity). Third, we observe that, since all queues in the system are stable, all requested video chunks shall be eventually delivered. Fourth, in order to translate the requested video quality into the effective video quality at the user playback, it is necessary that the chunks are delivered within their playback deadline. This requires that the largest delay among all queues at the helpers serving any given user is less than the pre-buffering time of that user at its streaming session startup phase. In order to achieve this condition with high probability, we propose an effective and decentralized (albeit heuristic) scheme to adaptively calculate the pre-buffering and re-buffering time at each user. In this way, the system is forced to work in the “smooth streaming regime,” i.e., in the regime of very small playback buffer underrun rate. Through simulations, we evaluate the performance of the proposed algorithm under realistic assumptions of a network with densely deployed helper and user nodes, including user mobility, variable bit-rate video coding, and users joining or leaving the system at arbitrary times.