In multiview video systems, multiple cameras generally acquire the same scene from different perspectives, such that users have the possibility to select their preferred viewpoint. This results in large amounts of highly redundant data, which needs to be properly handled during encoding and transmission over resource-constrained channels. In this work, we study coding and transmission strategies in multicamera systems, where correlated sources send data through a bottleneck channel to a central server, which eventually transmits views to different interactive users. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. The optimization relies both on a novel rate-distortion model, which captures the importance of each view in the 3D scene reconstruction, and on an objective function that optimizes resources based on a client navigation model. The latter takes into account the distortion experienced by interactive clients as well as the distortion variations that might be observed by clients during multiview navigation. We solve the scheduling problem with a novel trellis-based solution, which permits to formally decompose the multivariate optimization problem thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies. More in details, we show the gain offered by our dynamic scheduling policy compared to static camera allocation strategies and to schemes with constant coding strategies. Finally, we show that the best scheduling policy consistently adapts to the most likely user navigation path and that it minimizes distortion variations that can be very disturbing for users in traditional navigation systems.
Some prior studies address the problem of providing to users interactivity in selecting views, while saving on transmitted bandwidth and view-switching delay @cite_19 @cite_24 @cite_3 @cite_31 @cite_0 @cite_26 . The work in @cite_6 is mainly focused on coding views with a minimum level of redundancy in order to simplify the view switching, and the works in @cite_32 @cite_21 optimize the selection of views to be encoded and transmitted based on the user interest. The authors in @cite_0 @cite_8 investigate the transmission of multiview video coded streams on P2P networks and IP multicast, respectively. These works mainly focus on the coding optimization proposed as an a priori defined solution to provide interactive access to the different views @cite_28 . In our work, we rather dynamically optimize the coding modes and the scheduling of video frames for interactive multiview navigation. We extend our preliminary work in @cite_5 to include users' interactivity in the scheduling optimization. This allows the system to dynamically adapt the transmission of the coded frames to various system's dynamics and to outperform the above transmission policies of a priori encoded frames.
@cite_26 @cite_8 @cite_28 @cite_21 @cite_32 @cite_3 @cite_6 @cite_24 @cite_19 @cite_0 @cite_5 @cite_31
2098033563 2111229404 2131723117 2165659335 2013294225 2117382523 2093252776 2057415916 2142876645 2102152774 2046149460 2122883255 2357930737 2145419160 2033837429 2081893537 2077252365 2598021007 1485536093 2396399340
We derive an optimization framework for joint view and rate scalable coding of multi-view video content represented in the texture plus depth format. The optimization enables the sender to select the subset of coded views and their encoding rates such that the aggregate distortion over a continuum of synthesized views is minimized. We construct the view and rate embedded bitstream such that it delivers optimal performance simultaneously over a discrete set of transmission rates. In conjunction, we develop a user interaction model that characterizes the view selection actions of the client as a Markov chain over a discrete state-space. We exploit the model within the context of our optimization to compute user-action-driven coding strategies that aim at enhancing the client's performance in terms of latency and video quality. Our optimization outperforms the state-of-the-art H.264 SVC codec as well as a multi-view wavelet-based coder equipped with a uniform rate allocation strategy, across all scenarios studied in our experiments. Equally important, we can achieve an arbitrarily fine granularity of encoding bit rates, while providing a novel functionality of view embedded encoding, unlike the other encoding methods that we examined. Finally, we observe that the interactivity-aware coding delivers superior performance over conventional allocation techniques that do not anticipate the client's view selection actions in their operation. We present a novel client-driven multiview video streaming system that allows a user to watch 3D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his her head position. The user's head position is tracked and predicted into the future to select the views that best match the user's current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be prefetched in order to account for delays due to network transport and stream switching. The system allocates more bandwidth to the selected views in order to render the current viewing angle. Highly compressed, lower quality versions of some other views are also prefetched for concealment if the current user viewpoint differs from the predicted viewpoint. An objective measure based on the abruptness of the head movements and delays in the system is introduced to determine the number of additional lower quality views to be prefetched. The proposed system makes use of multiview coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compression efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions. While much of multiview video coding focuses on the rate-distortion performance of compressing all frames of all views for storage or non-interactive video delivery over networks, we address the problem of designing a frame structure to enable interactive multiview streaming, where clients can interactively switch views during video playback. Thus, as a client is playing back successive frames (in time) for a given view, it can send a request to the server to switch to a different view while continuing uninterrupted temporal playback. Noting that standard tools for random access (i.e., I-frame insertion) can be bandwidth-inefficient for this application, we propose a redundant representation of I-, P-, and “merge” frames, where each original picture can be encoded into multiple versions, appropriately trading off expected transmission rate with storage, to facilitate view switching. We first present ad hoc frame structures with good performance when the view-switching probabilities are either very large or very small. We then present optimization algorithms that generate more general frame structures with better overall performance for the general case. We show in our experiments that we can generate redundant frame structures offering a range of tradeoff points between transmission and storage, e.g., outperforming simple I-frame insertion structures by up to 45 in terms of bandwidth efficiency at twice the storage cost. The traffic of multi-view video is several times larger than traditional multimedia, which brings much more increment in the bandwidth requirement. To overcome this problem, Simulcast and MVC (Multi-view Video Coding) can be used to decrease the traffic for multi-view video. However, although Simulcast removes redundant information within each view, it still contains a large amount of inter-view redundant information. MVC further removes the correlation between the views, but all the views should be encoded and transmitted. So the bit-rates are still too high in these schemes. In this paper, in order to reduce the bit-rate for multi-view video transmission, we analyze the user's motion which is classified into random access model and successive motion model. We find that in the successive motion model only part of frames are possible to be displayed in a period of time. According to the analytical result, we propose a user dependent scheme to decrease the bit-rate of multi-view video transmission. The proposed scheme only encodes and transmits those frames that are possible to be displayed according to the user's motion. Further, in order to support the proposed scheme, we also propose a prediction structure as a substitute of the prediction structure in MVC. Performance evaluation proves that this proposed scheme reduces the average bit-rate for the transmission of multi-view video. This paper presents a rate-distortion (RD) optimized interactive streaming method for multiview video pre-compressed by H.264 Joint Multiview Video Model (JMVM). In the proposed method, multiple encodings are first used to facilitate the flexible server-client interaction. Second, a RD-optimized scheduling strategy is provided to guarantee the optimal view-dependent delivery of multiview video. In the RD-optimized scheduling strategy, a distortion model is proposed to estimate the expected end-to-end distortion by accounting for both coding and packet-loss-induced distortions, as well as rendering-induced distortion. With the end-to-end distortion model, the server can select the optimal encoding combination for transmission. Experimental results demonstrate that the proposed method can achieve a significant end-to-end RD performance improvement over the selective streaming methods with simulcast coding or scalable multiview coding. In addition, it has better error-resilience performance to combat with packet-losses over the Internet protocol (IP) networks. Multiview video is well known to involve interactions with audience and offer better view experience than conventional single-view video. The interactive nature of multiview video has made the service very delay-sensitive and thus imposes great challenges for providing multiview streaming service in Internet. Currently, almost none of existing works have addressed the delay issue in multiview streaming. To ensure the quality of service and offer preferable view experience for users, we propose a novel streaming framework to provide delay-guaranteed service for interactive multiview video. The basic tradeoff between consumed server bandwidth and the required delay is carefully studied. We leverage the features of MVC to keep the server bandwidth costs low while guarantee the delay constraint. In addition, we introduce a neighbor-assisted view switching scheme: peer's neighbors are involved in its switching process and their upload bandwidth resources are effectively utilized to reduce the bandwidth costs on server. Simulation results show that our proposed framework meet the delay-guaranteed requirement with restrained server bandwidth costs while maintaining scalability and resilience to users dynamics. We study the scenario of multicasting multi-view video content, recorded in the video plus depth format, to a collection of heterogeneous clients featuring Internet access links of diverse packet loss and transmission bandwidth values. We design a popularity-aware joint source-channel coding optimization framework that allocates source and channel coding rates to the captured content, such that the aggregate video quality of the reconstructed content across the client population is maximized, for the given packet loss and bandwidth characteristics of the clients and their view selection preferences. The source coding component of our framework features a procedure for generating a view and rate embedded bitstream that is optimally decodable at multiple data rates and accounts for the different popularity of diverse video perspectives of the scene of interest, among the clients. The channel coding component of our framework comprises an expanding-window rateless coding procedure that optimally allocates parity protection bits to the source encoded layers, in order to address packet loss across the unreliable client access links. We develop an optimization method that jointly computes the source and channel coding decisions of our framework, and also design a fast local-search-based solution that exhibits a negligible performance loss relative to the full optimization. We carry out comprehensive simulation experiments and demonstrate significant performance gains over competitive state-of-the-art methods (based on H.264 AVC and network coding, and H.264 SVC and our own channel coding procedure), across different scenario settings and parameter values. We consider the design of a scheduling policy for video streaming in a wireless network formed by several users and helpers (e.g., base stations). In such networks, any user is typically in the range of multiple helpers. Hence, an efficient policy should allow the users to dynamically select the helper nodes to download from and determine adaptively the quality level of the requested video segment. In order to obtain a tractable formulation, we follow a “divide and conquer” approach. First, we formulate a network utility maximization (NUM) problem where the network utility function is a concave and component-wise nondecreasing function of the time-averaged users' requested video quality index, and maximization is subject to the stability of all queues in the system. Second, we solve the NUM problem by using a Lyapunov drift plus penalty approach, obtaining a dynamic adaptive scheme that decomposes into two building blocks: 1) adaptive video quality and helper selection (run at the user nodes); and 2) dynamic allocation of the helper-to-user transmission rates (run at the help nodes). Our solution provably achieves NUM optimality in a strong per-sample path sense (i.e., without assumptions of stationarity and ergodicity). Third, we observe that, since all queues in the system are stable, all requested video chunks shall be eventually delivered. Fourth, in order to translate the requested video quality into the effective video quality at the user playback, it is necessary that the chunks are delivered within their playback deadline. This requires that the largest delay among all queues at the helpers serving any given user is less than the pre-buffering time of that user at its streaming session startup phase. In order to achieve this condition with high probability, we propose an effective and decentralized (albeit heuristic) scheme to adaptively calculate the pre-buffering and re-buffering time at each user. In this way, the system is forced to work in the “smooth streaming regime,” i.e., in the regime of very small playback buffer underrun rate. Through simulations, we evaluate the performance of the proposed algorithm under realistic assumptions of a network with densely deployed helper and user nodes, including user mobility, variable bit-rate video coding, and users joining or leaving the system at arbitrary times. An important issue of supporting multi-user video streaming over wireless networks is how to optimize the systematic scheduling by intelligently utilizing the available network resources while, at the same time, to meet each video's Quality of Service (QoS) requirement. In this work, we study the problem of video streaming over multi-channel multi-radio multihop wireless networks, and develop fully distributed scheduling schemes with the goals of minimizing the video distortion and achieving certain fairness. We first construct a general distortion model according to the network?s transmission mechanism, as well as the rate distortion characteristics of the video. Then, we formulate the scheduling as a convex optimization problem, and propose a distributed solution by jointly considering channel assignment, rate allocation, and routing. Specifically, each stream strikes a balance between the selfish motivation of minimizing video distortion and the global performance of minimizing network congestions. Furthermore, we extend the proposed scheduling scheme by addressing the fairness problem. Unlike prior works that target at users' bandwidth or demand fairness, we propose a media-aware distortion-fairness strategy which is aware of the characteristics of video frames and ensures max-min distortion-fairness sharing among multiple video streams. We provide extensive simulation results which demonstrate the effectiveness of our proposed schemes. We investigate coding tools for interactive multiview streaming (IMVS), where clients interactively request desired views for successive video frames, and in response the server sends the appropriate pre-compressed video data to the clients. Solution based on using only I-frames to support view switching would incur high transmission cost, while for that based on using only P-frames to encode every possible traversal, although it can minimize transmission cost, prohibitive server's storage may be required. Therefore, efficient solutions for IMVS need to consider the trade-off between transmission and storage cost. In this paper, we study the potential use of distributed source coding (DSC) in IMVS. Specifically, we propose two DSC constructions that could achieve good transmission-storage trade-offs. Central to these constructions is a method that can efficiently encode the least significant bits (LSB) of a frame to be decoded, leading to competitive storage and transmission requirements. Experiment results demonstrate these constructions compare favorably to existing tools, and could be valuable for interactive multiview streaming. In multiview video services, multiple cameras acquire the same scene from different perspectives, which results in correlated video streams. This generates large amounts of highly redundant data, which need to be properly handled during encoding and transmission of the multi-view data. In this work, we study coding and transmission strategies in multicamera sets, where correlated sources need to be sent to a central server through a bottleneck channel, and eventually delivered to interactive clients. We propose a dynamic correlation-aware packet scheduling optimization under delay, bandwidth, and interactivity constraints. A novel trellis-based solution permits to formally decompose the multivariate optimization problem, thereby significantly reducing the computation complexity. Simulation results show the gain of the proposed algorithm compared to baseline scheduling policies. In this paper, we concentrate on opportunistic scheduling for multicast information. We pose the problem as a multicast throughput optimization problem. As a solution we present how one can jointly utilize fixed-rate and rateless erasure coding along with simple rate adaptation techniques in order to achieve the optimal multicast throughput per user. We first investigate the performance of the proposed system under i.i.d. channel conditions. Our analysis shows a linear gain for the multicast capacity over i.i.d. Rayleigh fading channels with respect to the number of users. Since the established results require coding over large number of blocks and hence induce large decoding delays, we extend our analysis to the cases where we code over shorter block lengths and thus quantify the delay-capacity tradeoffs under a simple setting. We further look into non-i.i.d. channel conditions and show achievable gains by modifying a scheduling heuristic whose fairness is well- established for opportunistic scheduling of unicast flows. Our overall evaluations demonstrate that under both i.i.d. and non-i.i.d. channel conditions, opportunistic multicasting with erasure coding can significantly improve the performance over the traditional techniques used in today's communication systems. Although HTTP-based video streaming can easily penetrate firewalls and profit from Web caches, the underlying TCP may introduce large delays in case of a sudden capacity loss. To avoid an interruption of the video stream in such cases we propose the Backward-Shifted Coding (BSC). Based on Scalable Video Coding (SVC), BSC adds a time-shifted layer of redundancy to the video stream such that future frames are downloaded at any instant. This pre-fetched content maintains a fluent video stream even under highly variant network conditions and leads to high Quality of Experience (QoE). We characterize this QoE gain by analyzing initial buffering time, re-buffering time and content resolution using the Ballot theorem. The probability generating functions of the playback interruption and of the initial buffering latency are provided in closed form. We further compute the quasi-stationary distribution of the video quality, in order to compute the average quality, as well as temporal variability in video quality. Employing these analytic results to optimize QoE shows interesting trade-offs and video streaming at outstanding fluency. We present a novel framework for robustly delivering video data from distributed wireless camera networks that are characterized by packet drops. The main focus in this work is on robustness which is imminently needed in a wireless setting. We propose two alternative models to capture interview correlation among cameras with overlapping views. The view-synthesis-based correlation model requires at least two other camera views and relies on both disparity estimation and view interpolation. The disparity-based correlation model requires only one other camera view and makes use of epipolar geometry. With the proposed models, we show how interview correlation can be exploited for robustness through the use of distributed source coding. The proposed approach has low encoding complexity, is robust while satisfying tight latency constraints and requires no intercamera communication. Our experiments show that on bursty packet erasure channels, the proposed H.263+1 based method outperforms baseline methods such as H.263+ with forward error correction and H.263+ with intra refresh by up to 2.5 dB. Empirical results further support the relative insensitivity of our proposed approach to the number of additional available camera views or their placement density. In this paper, we present error-resilient Internet video transmission using path diversity and rate-distortion optimized reference picture selection. Under this scheme, the optimal packet dependency is determined adapting to network characteristics and video content, to achieve a better trade-off between coding efficiency and forming independent streams to increase error-resilience. The optimization is achieved within a rate-distortion framework, so that the expected end-to-end distortion is minimized under the given rate constraint. The expected distortion is calculated based on an accurate binary tree modeling with the effects of channel loss and error concealment taken into account. With the aid of active probing, packets are sent across multiple available paths according to a transmission policy which takes advantage of path diversity and seeks to minimize the loss rate. Experiments demonstrate that the proposed scheme provides significant diversity gain, as well as gains over video redundancy coding and the NACK mode of conventional reference picture selection. Consider media streaming in MIMO interference networks whereby multiple base stations (BS) simultaneously deliver media to their associated users using fixed data rates. The performance is fundamentally limited by the cross-link interference. We propose a cache-induced opportunistic cooperative MIMO (CoMP) for interference mitigation. By caching a portion of the media files, the BSs opportunistically employ CoMP to transform the cross-link interference into spatial multiplexing gain. We study a mixed-timescale optimization of MIMO precoding and cache control to minimize the transmit power under the rate constraint. The cache control is to create more CoMP opportunities and is adaptive to the long-term popularity of the media files. The precoding is to guarantee the rate requirement and is adaptive to the channel state information and cache state at the BSs. The joint stochastic optimization problem is decomposed into a short-term precoding and a long-term cache control problem. We propose a precoding algorithm which converges to a stationary point of the short-term problem. Based on this, we exploit the hidden convexity of the long-term problem and propose a low complexity and robust solution using stochastic subgradient. The solution has significant gains over various baselines and does not require explicit knowledge of the media popularity. Wired and wireless data networks have witnessed a rapid proliferation of multimedia applications such as Internet video streaming, video conferencing, etc. A desirable key feature for multimedia transmission over multiuser environments is the ability of adapting rate and quality of video stream to different QoS conditions. The most efficient approach to address the scalability of multimedia applications is to encode video streams in compliance with the scalable video coding (SVC) standard, which is proposed as an extension to H.264 AVC standard. This paper addresses the rate control and bandwidth sharing for multimedia applications that are relying on SVC-encoded video streams. In previous studies, idealistic utility functions, mainly in the form of a staircase function, were used to cast the rate allocation for SVC-encoded streams as an optimization problem. These utility functions make the optimization problem non-convex and even non-differentiable, for which achieving optimality through dual-based approaches proves quite challenging. Towards this goal, we introduce an accurate analytical model of the utility function for SVC-encoded video streams. Using the abovementioned utility model and adopting the utility-proportional optimization approach, we come up with a convex formulation and propose a dual-based distributed algorithm for rate allocation of SVC-encoded streams. To the best of our knowledge, this is the first work that focuses on an accurate utility function modeling for SVC-encoded streams. Simulation experiments show that the proposed algorithm is quite efficient in achieving the convergence towards the global optimality. We consider the content delivery problem in a fading multi-input single-output channel with cache-aided users. We are interested in the scalability of the equivalent content delivery rate when the number of users, @math , is large. Analytical results show that, using coded caching and wireless multicasting, without channel state information at the transmitter, linear scaling of the content delivery rate with respect to @math can be achieved in some different ways. First, if the multicast transmission spans over @math independent sub-channels, e.g., in quasi-static fading if @math , and in block fading or multi-carrier systems if @math , linear scaling can be obtained, when the product of the number of transmit antennas and the number of sub-channels scales logarithmically with @math . Second, even with a fixed number of antennas, we can achieve the linear scaling with a threshold-based user selection requiring only one-bit feedbacks from the users. When CSIT is available, we propose a mixed strategy that combines spatial multiplexing and multicasting. Numerical results show that, by optimizing the power split between spatial multiplexing and multicasting, we can achieve a significant gain of the content delivery rate with moderate cache size. A major fraction of multimedia stream contents tends to be redundant and leads to wastage of storage capacity and channel bandwidth. In order to eliminate surplus data, standard video compression algorithms exploit spatial and temporal correlation present in video sequence. However, in case of a multisensor network, intersensor statistical redundancy is the most significant factor in acquiring efficient link utilization as well as making resultant findings valuable to the end user. In this paper, an extension to our previously proposed scheme has been presented to accomplish performance goals of a multisensor environment. Standard MPEG codec has been used to accomplish distributed motion compensation in prespecified directions known as directional correlation. Video frame correlation has been estimated locally at the camera node as well as across different nodes, defined as node communication strategies. Further, receiver feedback assists in quality control after reconstitution by decoder assessment. Results estimated have been analyzed for saving ratios and multimedia quality. Results analysis illustrates increased gains in frame quality and compression saving, achieved through reducing node displacement from the reference node (NR). Humans see only a tiny region at the center of their visual field with the highest visual acuity, a behavior known as foveation. Visual acuity reduces drastically towards the visual periphery. 'Foveated' video coding compression techniques exploit this non-uniformity to gain significant efficiency by compressing more in the periphery and less in the center. We propose a practical and scalable method to use such a technique for video streaming service over the Internet. The essential idea is to use a commodity webcam on the user side to provide real-time gaze feedback to the server with the server sending appropriately coded video to the client player. We develop a multi-resolution video coding approach that is scalable in that it is possible to pre-code the video in a small number of copies for a given set of resolutions. The coding approach is designed to match the error performance of an eye tracker built using commodity webcams. We demonstrate that the technique is energy efficient and thus usable in mobile devices. We develop a methodology for performance evaluation of such a system when network budgets may vary and video quality may fluctuate. Finally, we present a comprehensive user study that demonstrates a bandwidth reduction of a factor of 2 for the same user satisfaction.