Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> The amount of video data stored on local devices or transmitted over the networks is permanently increased. The emerging of a more efficient next generation video coding standard is of a high demand at the moment. There seem to be two main contenders for the position of the next state-of-the-art video compression standard: JCT-VC H.265/HEVC and Google VP9. The announced aim of HEVC is to achieve twice more efficient compression compared to H.264/AVC, and VP9 was developed to get half the bit-rate of VP8 with royalty-free video 6804 M.P. Sharabayko, O.G. Ponomarev and R.I. Chernyak codec. Intra compression is one of the main features that determines the compression efficiency of the whole codec. In this paper we get into detailed overview of intra compression data-flow in HEVC and VP9. We describe common and unique stages of both standards. Then we carry out experiments with JCT-VC HM and WebM VP9 encoders on intra compression efficiency. We also turn some of the HEVC features off to get its dataflow as close to VP9 as possible. Finally we get into discussion of the efficiency of both codecs, the corresponding standards and their intra compression algorithms. Mathematics Subject Classification: 94A08 <s> BIB001 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> This work presents a performance comparison of the two latest video coding standards H.264/MPEG-AVC and H.265/MPEG-HEVC (High-Efficiency Video Coding) as well as the recently published proprietary video coding scheme VP9. According to the experimental results, which were obtained for a whole test set of video sequences by using similar encoding configurations for all three examined representative encoders, H.265/MPEG-HEVC provides significant average bit-rate savings of 43.3% and 39.3% relative to VP9 and H.264/MPEG-AVC, respectively. As a particular aspect of the conducted experiments, it turned out that the VP9 encoder produces an average bit-rate overhead of 8.4% at the same objective quality, when compared to an open H.264/MPEG-AVC encoder implementation - the x264 encoder. On the other hand, the typical encoding times of the VP9 encoder are more than 100 times higher than those measured for the x264 encoder. When compared to the full-fledged H.265/MPEG-HEVC reference software encoder implementation, the VP9 encoding times are lower by a factor of 7.35, on average. <s> BIB002 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> It is accepted that image compression and transmission are essential in the image processing system and that errors occurred may introduce the degradation on the quality of the received image over the wireless transmission for intelligent Internet of things (IoT). Image quality assessment metric (QAM) is of fundamental significance to various image processing systems, and the goal of studying the QAM is to design an algorithm that can automatically evaluate the quality of the received image at the terminal display equipment in a perceptible way under the ubiquitous network circumstances. In this paper, focusing on the image compression and transmission, a joint full-reference (FR) QAM ( JQAM ) for evaluating the quality of a 3-D image is proposed based on the state-of-the-art physiological and psychological properties of the human visual system (HVS) for image transmission under the wireless networks. The major technical contribution of this paper is that the binocular perception (depth perception) and local image properties are taken into consideration. First, the luminance masking property and local image content information are calculated to establish image QAM (IQAM) to improve the abilities of current quality evaluation algorithms. Meanwhile, the information abstracted from the depth map is also utilized as side information (SI) to evaluate the depth map on quality assessment. Finally, IQAM and SI are combined to construct the proposed JQAM. Experimental results show that the proposed metric could achieve better correlation with the subjective quality scores compared with the relevant existing IQAMs, and it could be used to evaluate the quality of the 3-D image signal of the intelligent equipment for multimedia communication systems in IoT. <s> BIB003 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> Nowadays, multimedia is considered to be the biggest big data as it dominates the traffic in the Internet and mobile phones. Currently symmetric encryption algorithms are used in IoT but when considering multimedia big data in IoT, symmetric encryption algorithms incur more computational cost. In this paper, we have designed and developed a resource-efficient encryption system for encrypting multimedia big data in IoT. The proposed system takes the advantages of the Feistel Encryption Scheme, an Advanced Encryption Standard (AES), and genetic algorithms. To satisfy high throughput, the GPU has also been used in the proposed system. This system is evaluated on real IoT medical multimedia data to benchmark the encryption algorithms such as MARS, RC6, 3-DES, DES, and Blowfish in terms of computational running time and throughput for both encryption and decryption processes as well as the avalanche effect. The results show that the proposed system has the lowest running time and highest throughput for both encryption and decryption processes and highest avalanche effect with compared to the existing encryption algorithms. To satisfy the security objective, the developed algorithm has better Avalanche Effect with compared to any of the other existing algorithms and hence can be incorporated in the process of encryption/decryption of any plain multimedia big data. Also, it has shown that the classical and modern ciphers have very less Avalanche Effect and hence cannot be used for encryption of confidential multimedia messages or confidential big data. The developed encryption algorithm has higher Avalanche Effect and for instance, AES in the proposed system has an Avalanche Effect of %52.50. Therefore, such system is able to secure the multimedia big data against real-time attacks. <s> BIB004 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> A common approach in the realization of wavelet-based compressors for ECG signals makes use of truncation methods, whereby wavelet coefficients are truncated (i.e., thrown away) if they're deemed insignificant. A popular truncation strategy applied in these compressors is to truncate based on Energy Packing Efficiency or EPE, which tends to favor coefficients at higher scales because their energy contribution (either in squared or absolute-value sense) is itself insignificant. In this paper, we present four rudimentary truncation strategies to analyze and demonstrate how the choice of truncation strategy can affect the signal in terms of compression ratio (CR) and signal fidelity. Of these, a truncation strategy we call ‘ScaleRelativeMAX’ is proposed, which exhibits some useful properties for sensitive biomedical applications. Simulation results are presented using representative select ECG records from PhysioNet's database to show that some truncation methods — in particular, our proposed truncation strategy — allows for fine-grained fidelity and CR control than others and offer nearly linear reconstruction error growth as a function of the truncation threshold in comparison to other strategies that are more aggressive in favoring CR over signal fidelity. Such fidelity-first strategies are useful in biomedical communication architectures for emerging Internet-of-Things (IoT) applications that employ compressors to minimize energy transmission costs in Body Area Sensor Networks (BASNs) and similar wearable devices. Such signals carry diagnostic information that are of clinical significance, and whose reconstruction of clinical features should take priority and is in stark contrast to ordinary multimedia class signals, which generally tend to favor CR over signal fidelity. <s> BIB005 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> In this paper, we present a compressed sensing-based approach, which combines the dictionary learning (DL) method and the approximate message passing (AMP) approach. The approach can be used for efficient communication in the multimedia Internet of Things (IoT). AMP is a signal reconstruction algorithm framework, which can be explained as an iterative denoising process. On the other hand, the DL method seeks an adaptive dictionary for realizing sparse signal representations, and provides good performance in signal denoising. We apply the DL-based denoising method within the AMP algorithm framework and propose a novel DL-AMP framework. We demonstrate our framework’s effectiveness for multimedia IoT devices by showing its capability in reducing required communication bandwidth for multimedia communication while improving reconstruction quality (by over 2 dB). <s> BIB006 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> Small-size and light-weight, as the modern design concept for the emerging wearable devices, has become a trend. However, such trend puts physical limitations to the battery, and the resulting short battery lifetime becomes the bottleneck for most wearable devices today. In this paper, we aim to optimize the energy usage through data compression and transmission rate control. We propose a novel joint compression–transmission approach, which not only minimizes the energy consumption of both compression and transmission, but also maintains the corresponding data distortion and transmission delay within a certain tolerant level. By adopting the Lyapunov framework, we develop an online algorithm to minimize the one-slot drift-plus-penalty function. We conduct numerical analysis and experimental study for our proposed approach. The results show that the size of queuing buffer has the significant impact on the energy cost. Next, we verify a fundamental tradeoff between the energy expenditure and transmission delay, and derive the theoretical performance bounds. After that, we show that the energy cost is also determined by the wireless channel gain and the data compression ratio. Finally, compared to a strategy without compression, our approach can save up to 92% of energy. <s> BIB007 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> Because of the co-existing of H.264/AVC and high efficiency video coding standard (HEVC) in the coming long period, video transcoding technology has become an essential part of multimedia communication in the field of the Internet of Vehicles (IoV). However, due to the huge computational complexity of re-encoding processes, traditionally cascaded transcoders greatly increase the computing burden of the embedded devices and impact the real-time capability of transportation communication systems. In order to address this problem, a fast transcoding solution is proposed in this paper. First, we exploit the mapping relationship among H.264/AVC decoding information and HEVC coding unit (CU) depth decision and prediction unit (PU) mode decision. Then, a three-output classification model is built for CU depth decision processes, and a two-output classification model is built for PU mode selection processes by using support vector machine method. Finally, the models are applied into the cascaded transcoder to accelerate the re-encoding process. The experimental results show that our proposal averagely achieves up to 53.7% and 52.3% complexity reductions under Lowdelay_P_main and Randomaccess_main configurations, respectively, with the negligible rate-distortion degradation, which show a great potential in improving the transcoding efficiency in the real-time video communication system of IoV. <s> BIB008 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> Internet of Things devices, such as video-based detectors or road side units are being deployed in emerging applications like sustainable and intelligent transportation systems. Oftentimes, stringent operation and energy cost constraints are exerted on this type of applications, necessitating a hybrid supply of renewable and grid energy. The key issue of a cost-constrained hybrid of renewable and grid power is its uncertainty in energy availability. The characteristic of approximate computation that accepts an approximate result when energy is limited and executes more computations yielding better results if more energy is available, can be exploited to intelligently handle the uncertainty. In this paper, we first propose an energy-adaptive task allocation scheme that optimally assigns real-time approximate-computation tasks to individual processors and subsequently enables a matching of the cost-constrained hybrid supply of energy with the energy demand of the resultant task schedule. We then present a quality of service (QoS)-driven task scheduling scheme that determines the optional execution cycles of tasks on individual processors for optimization of system QoS. A dynamic task scheduling scheme is also designed to adapt at runtime the task execution to the varying amount of the available energy. Simulation results show that our schemes can reduce system energy consumption by up to 29% and improve system QoS by up to 108% as compared to benchmarking algorithms. <s> BIB009 </s> Multimedia Internet of Things: A Comprehensive Survey <s> A. MULTIMEDIA CODING AND COMPRESSION <s> Abstract Today, wireless cameras are widely deployed in various applications such as in commercial and military domains (eg. patients monitored in smart hospitals, multimedia surveillance deployed in smart cities, intelligent multimedia monitoring of an ecological system, etc.). The design of such systems can be achieved by allowing the interaction between all their component objects ( i.e. , internet of things), which is one of the preoccupations of the researchers this last decade. However, using the multimedia sensor nodes for the internet of things (IoT) is problematic in term of power consumption, since these latter would be wireless and battery powered. Within the wider context of IoT, Wireless Multimedia Sensor Networks (WMSNs) can be a promising technology for the aforementioned domains. However, in a such network, the captured data is usually voluminous and requires local processing, and thus, entailing a higher energy requirement. Therefore, the most important issue is to balance the video data coding versus the perceived video quality at the sink, while maximizing the overall network lifetime. To this end, we propose a fully distributed algorithm, that considers the problem of allocating the resources in wireless video sensor network, in a way to ensure a trade-off between both: the encoding power and the source rate at each node, while serving the desired video quality requirement. In opposition to existing approaches, our algorithm, of distributed nature, is provided to find such a trade-off whatever the initial network configuration is. In addition, a dynamic change of links capacity is considered with the limited transmission and reception powers. A new problem is then formulated to cope with the channel transmission error and it is solved. <s> BIB010
In conventional multimedia data encoding techniques, data is compressed once and decoded whenever played. M-IoT devices are more concerned with uploading the data in uplink transmission, which poses challenges on computationally powered constrained M-IoT devices. Traditionally video encoding/compression is achieved by utilizing spatial and temporal redundancies. The techniques for multimedia encoding are known as High-Efficiency Video Coding (HEVC) jointly developed by ITU-T and International Organization for Standardization (ISO). The current standard jointly designed in 2013 is H.265, which shows efficient performance as compared to H.264 developed in 2003  . The major features of H.265 include maximum block size of 64 × 64, adaptive block sub splitting and prediction, and up to thirtyfive intra-frame prediction directions. Google developed VP9 and DAALA is designed by Mozilla Corporation, both compete with H.264 BIB001 . Comparative analysis of H.264, H.265, VP9, and DAALA has been presented in BIB002 . Results depict that H.265 outperforms other encoding techniques. Liu et al. BIB008 devised a computation complexity reduction transcoder solution for real-time video communication. After mapping the relationship among H.264/AAVC decoding information, the Coding Unit (CU), and the Prediction Unit (PU) decision process, the proposed model exploits SVM to classify either to select PU decision mode or CU depth decision process. The objective of the algorithm is to optimize the interframe prediction process of the HEVC reencoder. Energy consumption can be significantly minimized by applying compression techniques to avoid energy transmission costs. Santos et al. BIB005 proposed a truncation based compression technique named ScaleRelativeMax based on Energy Packing Efficiency (EPC) for sensitive biomedical applications. For a biomedical application, the reconstruction of the signal without losses is of great importance. The impact of different truncation strategies on the Compression Ratio (CR) has been discussed in detail and compared with the proposed strategy. Results illustrate that the proposed technique offers linear reconstruction growth with comparable CR to other techniques. Compressed Sensing (CS) uses sparse signal structures to diminish the size of the transmitted or stored data. Multimedia data is sparse in structure. CS is used for multimedia data acquisition and reconstruction. Li et al. BIB006 exploit DL and AMP for improving IoT multimedia data reconstruction quality and minimize bandwidth utilization. Performance comparison of the DL-AMP technique with EM-GM-GAMP, Gauss-AMP, fast-BM3D-AMP, and other proposed CS techniques in the literature validates the proposed framework is suitable for multimedia. A Joint full reference Quality Assessment Metric (JQAM) based on the Human Visual System (HVS) by considering binocular perception and image properties is projected in BIB003 . The proposed technique efficiently measures the image pixel, contrast, and structural distortion by considering the luminance masking of the image to access the quality of the image. Hu et al. in BIB007 proposed a joint compression and transmission technique to optimize energy usage based on the canonical Lyapunov optimization scheme for power-constrained multisensor wearable devices. M-IoT devices are energy and resource-constrained. M-IoT devices are required to be computationally efficient as most of the devices are battery operated. An efficient task allocation strategy to maintain the QoS is required to assign the task to process in parallel dynamically. Wei et al. BIB009 proposed a real-time approximate task computation allocation strategy for MultiProcessor System on Chips (MPSoC) to increase the number of executions in a hybrid energy environment. In a hybrid renewable energy system, the power supply is unpredictable. Authors designed a dynamic task scheduler to adapt real-time varying conditions in fluctuating energy availability. Khernane et al. BIB010 discussed the tradeoff between video data coding and quality at the user end while optimizing the energy for the maximum life expectancy of the network. An entirely distributed algorithm is presented to balance the power utilized to encode the video and video encoding rate. The authors considered the dynamic nature of link capacity. The transmission errors are minimized by incorporating retransmissions based on the two-state Markov chain. However, in this work, spectrum efficiency is not taken into account, which is the key requirement for multimedia data transmission. A detailed survey on M2M communication is highlighting the issue and challenges in distributed video cameras in  . Various video coding techniques, power consumption analysis, and energy harvesting approaches is detailed in this article. Aljawarneh et al. BIB004 devised a Graphics Processing Unit (GPU) encryption system based on Feistel Encryption Scheme (FES) and Advanced Encryption Standard (AES) to secure medical multimedia data. The multimedia data is divided into equal-sized blocks, which are subdivided into plain text and keys. Each key and plain text is then encrypted separately using FES and AES, respectively. The genetic algorithm integrates separately encrypted keys and plain text to secure the data from vulnerability. The system utilizes the GPU to process graphics efficiently and provides parallel processing to execute maximum tasks to increase system throughput.