A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> This paper proposes a robust video streaming scheme for priority networks with path diversity, based on a combined approach of multiple description coding (MDC) with optimal picture classification into two priorities. A binary classification algorithm is proposed to define high (HP) and low (LP) priority network abstraction layer units (NALU), which in turn define the packet priorities. An optimisation algorithm is used to find HP pictures, based on dynamic programming and relying on minimisation of the packet loss concealment distortion. The paper shows that the proposed algorithm is able to effectively improve the decoded video without increasing the MDC stream redundancy. The overall performance evaluation, carried out by simulating MDC video streaming over lossy networks with path diversity, demonstrates that the proposed algorithm yields higher video quality for a wide range of packet loss rates (PLR). Comparing with no-priority MDC video streaming schemes, the simulation results show that the proposed algorithm can improve the average PSNR results in 0.7–3.2dB for packet loss rates between 3% and 15%. <s> BIB001 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> Datagram Congestion Control Protocol (DCCP) is an unreliable transport layer protocol with a congestion control mechanism used for multimedia streaming. In order to use multiple network interfaces to transmit streaming data smoothly, a Multi-Path Datagram Congestion Control Protocol (MP-DCCP) is proposed and presented in this paper. Video streaming transmission through multi-path faces three problems: (1) out-of-ordering packets at the receiver side, (2) conditions of paths are changed anytime, and (3) the importances of frames/packets are different. The first problem may let video streaming data be delivered to the application layer too late, especially for the live streaming data. The second and third problems may let a packet be scheduled to an unsuitable path for transmission. Since importances of video streaming frames/packets are different, it should consider how to schedule the transmission of frames/packets properly. For example, an important frame/packet should be transmitted in the more reliable path. In order to resolve these three problems, a QoS-aware Order Prediction Scheduling (QOPS) scheme for MP-DCCP is proposed in this paper. QOPS estimates packets' arrival orders at the receive side through these multiple paths before packets are scheduled into the transmission at the sender side. From simulation results, we proved that the packets' out-of-ordering problem in MPDCCP can be improved using our proposed QOPS scheme. <s> BIB002 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> The increasing number of interfaces using different access technologies in modern devices gives opportunities for enhancing the Quality of Service (QoS) delivered to multimedia and interactive data transfers involved in distributed applications. In the current Internet, the presence of "middleboxes" such as Network Address Translators (NAT) or firewalls hardly lets applications use any Transport protocol but the well-known Transmission Control Protocol (TCP). Currently under standardization at the IETF, the new Multipath-TCP (MPTCP) protocol uses several TCP flows to make use of the multiple interfaces available on the end terminals, thus improving both network availability and QoS, still being capable to cross over middle boxes. Although originally being fully reliable and fully ordered, its two sub-layers architecture gives opportunity to use QoS techniques over fully reliable paths. This paper studies the QoS benefits induced by the implementation of the "partial reliability" concept in MPTCP for interactive video applications. Two different mechanisms are experimented with the aim to enhance global quality of video transmission over paths close to 3G networks characteristics. The Internet Engineering Task Force (IETF) MPTCP working group's ns-2 implementation is used for the experimentations. The Evalvid toolkit over ns-2 is used to measure the QoS benefits expressed in terms of Peak Signal to Noise Ratio (PSNR). <s> BIB003 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> The Internet infrastructure often supports multiple routes between two communicating hosts and, today, especially mobile hosts usually offer multiple network interfaces, so that disjoint paths between the hosts can be constructed. Having a number of (partly or fully) disjoint paths available may allow applications to distribute their traffic, aggregate capacity of different paths, choose the most suitable subset of paths, and support failover if a path fails. Exploiting multipath characteristics has been explored for TCP, but the requirements for real-time traffic differs notably. In this paper, we devise a multipath communication model for Real-time Transport Protocol (RTP); present minimal set of required protocol extensions; develop algorithms for scheduling RTP traffic across multiple paths at the sender and a corresponding de-jittering algorithm at the receiver side; and evaluate our proposal in varying scenarios using media traffic across different emulated mobile access network setups. <s> BIB004 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> We consider the problem of scalable video streaming from a server to multinetwork clients over heterogeneous access networks, with the goal of minimizing the distortion of the received videos. This problem has numerous applications including: 1) mobile devices connecting to multiple licensed and ISM bands, and 2) cognitive multiradio devices employing spectrum bonding. In this paper, we ascertain how to optimally determine which video packets to transmit over each access network. We present models to capture the network conditions and video characteristics and develop an integer program for deterministic packet scheduling. Solving the integer program exactly is typically not computationally tractable, so we develop heuristic algorithms for deterministic packet scheduling, as well as convex optimization problems for randomized packet scheduling. We carry out a thorough study of the tradeoff between performance and computational complexity and propose a convex programming-based algorithm that yields good performance while being suitable for real-time applications. We conduct extensive trace-driven simulations to evaluate the proposed algorithms using real network conditions and scalable video streams. The simulation results show that the proposed convex programming-based algorithm: 1) outperforms the rate control algorithms defined in the Datagram Congestion Control Protocol (DCCP) by about 10-15 dB higher video quality; 2) reduces average delivery delay by over 90% compared to DCCP; 3) results in higher average video quality of 4.47 and 1.92 dB than the two developed heuristics; 4) runs efficiently, up to six times faster than the best-performing heuristic; and 5) does indeed provide service differentiation among users. <s> BIB005 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> Recent technological advancements in wireless infrastructures and handheld devices enable mobile users to concurrently receive multimedia contents with different radio interfaces (e.g., cellular and Wi-Fi). However, multipath video transport over the resource-limited and error-prone wireless networks is challenged with key technical issues: 1) conventional multipath protocols are throughput-oriented, and video data are scheduled in a content-agnostic fashion and 2) high-quality real-time video is bandwidth-intensive and delay-sensitive. To address these critical problems, this paper proposes a bandwidth-efficient multipath streaming (BEMA) protocol featured by the priority-aware data scheduling and forward error correction-based reliable transmission. First, we present a mathematical framework to formulate the delay-constrained distortion minimization problem for concurrent video transmission over multiple wireless access networks. Second, we develop a joint Raptor coding and data distribution framework to achieve target video quality with minimum bandwidth consumption. The proposed BEMA is able to effectively mitigate packet reordering and path asymmetry to improve network utilization. We conduct performance evaluation through extensive emulations in Exata involving real-time H.264 video streaming. Compared with the existing multipath protocols, BEMA achieves appreciable improvements in terms of video peak signal-to-noise ratio, end-to-end delay, bandwidth utilization, and goodput. Therefore, BEMA is recommended for streaming high-quality real-time video to multihomed terminals in heterogeneous wireless networks. <s> BIB006 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> One major concern when applying Multipath TCP (MPTCP) to the real-time multimedia applications is related to MPTCP's fully-reliable and fully-ordered service nature, which will inevitably degrade users' Quality of Experience (QoE) for multimedia streaming services in a heterogeneous wireless network environment because asymmetric wireless links are commonly with different transmission characteristics and sensitive to variations. In this paper, we first discuss the design considerations of partially reliable-MPTCP associated with the real-time constraint of multimedia streaming. Then we propose a context-aware QoE-oriented MPTCP Partial Reliability extension (PR-MPTCP+) for providing partially reliable multimedia streaming service to an upper layer protocol. Finally, we evaluate the proposed PR-MPTCP+ solution using a wide range of multimedia quality metrics. <s> BIB007 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> Delivering high-definition (HD) wireless video under stringent delay constraint is challenging with regard to the limited network resources and high transmission rate. Concurrent multipath transfer (CMT) using stream control transmission protocol (SCTP) exploits the multihoming feature of mobile devices to establish associations with different access networks. In this paper, we study the multihomed HD video communication with SCTP over heterogeneous wireless networks. The existing CMT schemes mainly treat the traffic data in a content-agnostic fashion, and thus cannot effectively leverage the scarce wireless resources to maximize the perceived video quality. To address this critical issue, we propose a content-aware CMT (CMT-CA) solution that featured by the unequal frame-level scheduling based on estimated video parameters and feedback channel status. First, we develop an analytical framework to model the total distortion of parallel video transmission over multiple wireless access networks. Second, we introduce a joint congestion control and data distribution scheme to minimize the total distortion based on online quality evaluation and Markov decision process (MDP). The performance of CMT-CA is evaluated through extensive semi-physical emulations in Exata involving HD video encoded with H.264 codec. Experimental results show that CMT-CA outperforms the reference schemes in terms of video peak signal-to-noise ratio (PSNR), end-to-end delay, and goodput. Or conversely, CMT-CA achieves the same video quality with $20$ percent bandwidth conservation. <s> BIB008 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> Transport protocols that can exploit multiple paths, especially MPTCP, do not match the requirements of video streaming: high average transmission delay, too strict reliability, and frequent head-of-line phenomenons resulting in abrupt throughput drops. In this paper, we address this mismatch by introducing a cross-layer scheduler, which leverages information from both application and transport layers to re-order the transmission of data and prioritize the most significant parts of the video. Our objective is to maximize the amount of video data that is received in time at the client. We show that current technologies enable the implementation of this cross-layer scheduler without much overhead. We then demonstrate the validity of our approach by studying the performance of an optimal cross-layer scheduler. The gap between the performance of the traditional scheduler versus the optimal scheduler justifies our motivation to implement a cross-layer scheduler in practice. We propose one implementation with basic cross-layer awareness. To evaluate the performance of our proposal, we aggregate a dataset of real MPTCP sessions and we use video stream encoded with HEVC. Our results show that our cross-layer proposal outperforms the traditional scheduler. Viewers not only benefit from the inherent advantages of using MPTCP (such as a better resilience to path failure) but also get a better QoE compared to the traditional scheduler. <s> BIB009 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> Video streaming has already become the main portion of data traffic over the Internet. However, it is hard to guarantee the user-perceived video quality in single access networks due to the limitation in available bandwidth. Multihomed systems enable concurrent multipath data transfer over multiple network paths. Path control can be implemented at the transport layer such as in Stream Control Transport Protocol (SCTP). SCTP has built-in support for multihoming, but offers only a limited standard path selection mechanism. In this paper a new method for video streaming over SCTP is proposed. Secondary paths are used to send redundant information to prevent degradation of viewer's perceived quality. The efficiency of the proposed method is demonstrated by computer simulations using video coded with H.264/MPEG-4 AVC (Advanced Video Coding). The results show that the proposed method can prevent degradation of video quality at the cost of small extra bandwidth usage. <s> BIB010 </s> A Holistic Survey of Wireless Multipath Video Streaming <s> 1) Content Awareness: <s> Bandwidth constraints and high end-to-end delays are real challenges for achieving and sustaining high quality mobile video streaming services. Diverse multipath transmission techniques are being investigated as possible solutions, since recent developments have enabled mobile devices users to receive video data simultaneously over multiple interfaces (e.g., LTE and WiFi). While some multipath protocols have been recently standardized for this purpose (e.g., MPTCP), being network layer protocols they cannot properly handle challenging transmission scenarios subject to packet losses and congestion, such as lossy wireless channels. In this work, we adopt the MPEG Media Transport (MMT) protocol to propose an improvement for mobile multipath video streaming solutions. MMT is an application layer protocol with inherent hybrid media delivery properties. We propose a novel path-and-content-aware scheduling strategy for MMT by means of full cooperation between network metrics and video content features. Our strategy provides better models to adaptively cope with unstable communication channel conditions and to improve the final user quality of experience (QoE). For the experimental evaluation, we used NS3-DCE to simulate a realistic multipath network scenario which includes channel error models and background traffic. Results for two video sequences are presented in terms of PSNR, SSIM, goodput, delay and packet loss rates. When compared with a simple scheduling strategy for the traditional multipath MMT, our approach yields significant packet loss rate reductions (~90%) and video quality improvements of around 12 dB for PSNR and 0.15 for SSIM. <s> BIB011
Considering video content features in the scheduling strategy helps to define the priority of each packet, and subsequently choose the frame packets with higher priority to send it first or via more qualified paths. In video streaming, some frames have higher effect on video quality, and large frame inter-dependency. For example, I-frames have highest priority among other frames. These strategies are generally referred to as content-aware scheduling strategies. In addition, a content-aware scheduling strategy could use stronger packet protection for higher priority packets than the less priority packets, for example, by applying adaptive FEC, which will be explained in next subsection. On the other hand, if the scheduler is unaware of the video content features, the sending buffer would transmit data packets in the same order as they arrived in the buffer (FIFO) without considering the priority of packets (e.g., MPTCP scheduler). Video content features are considered as inputs to the scheduling strategy in the following works: MPRTP BIB004 , BIB011 , BEMA BIB006 , BIB005 , BIB001 , MP-DCCP BIB002 , MPTCP-SD BIB003 , PR-MPTCP + BIB007 , CMA-CA BIB008 , BIB009 . In SRMT BIB010 , the primary path is used for all data while the secondary paths are used to send redundant packets, which are, in turn, chosen based on their priority (e.g. I-frame packets have highest priority). 2) Video Distortion (Frame Level): Video distortion impacts perceived video quality. Generally, video distortion is considered at both frame level and flow level. In this section, we study the frame level video distortion because it assesses inter-frame dependencies and analyzes each specific video frame, including the frame priority and decoding dependency BIB005 . We will discuss flow level video distortion in Section V-C5. In particular, frame level distortion refers to the quality degradation of each frame of GoP after data transmission and video decoding process BIB006 . This way, the frame level distortion is calculated as a total of truncation and drifting distortion. The truncation distortion refers to the video quality degradation caused by packet drops during transferring data, and the drifting distortion refers to the video quality distortion occurred by imperfect reconstruction of parent frames which are used for inter-frame prediction. In the surveyed works, frame level distortion is used by BEMA BIB006 for calculating FEC coding parameters (e.g., code rate and symbol size), and also it is used by BIB005 to assign higher priority values to the pictures which minimize the distortion of the decoded video affected by packet loss. Such information could also be used for path selection in CMT-CA BIB008 .