Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> This paper describes the anatomy of a pilot surveillance system with a speech-based interface for content-based retrieval of video data. The proposed system relies on an ontology-based information sharing architecture and lets components of the system communicate among each other through TCP/IP communication channels. The aim of developing the pilot system was to explore dependencies between image analysis, event detection, video annotation, and speech- based retrieval of the video content in the context of a broader spoken dialogue system. <s> BIB001 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> Due to the rapid improvement in resolution and codec, the number of video sensing device grows fast in the recent years. Tremendous data need to be stored and processed. To meet such a need, we developed a video hosting and processing platform for near-real time applications. An intelligent device can obtain connection-less upload function from our client-side SDK. Video data are sliced into pieces, distributed over cloud-of-clouds and can be processed in parallel once stored. Various video processing algorithms can be mounted together and processed by multiple CPU cores. Performance evaluations show that our platform has the ability to host and process large-scale video data. <s> BIB002 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> This paper has designed and implemented a video surveillance system based on the smart Android terminal equipment. This system consists of two parts, including the server and the client. The server is responsible for the video acquisition and the H.264 video encoding. The video transmission uses RTP protocol. The client is responsible for receiving data, and completing the decoding and playback. Because of the limited storage capacity of mobile devices, the server implements the face detection function, which only stores the critical information. This monitoring system is applied to Android phones which are used for the elderly. With the arrival of the social trend of population aging in China, customers can get information about the old people at home by using this monitoring system. It can provide emergency alert about emergency situations. This system has a good mobile performance and transmission stability. It is intelligent, convenient and practical. <s> BIB003 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> Battery lifetime is critical for wireless video sensors. To enable battery-powered wireless video sensors, low-power design is required. In this paper, we consider applying multi-view summarization to wireless video sensors to remove redundant contents such that the compression and transmission power can be reduced. A low-complexity online multi-view video summarization scheme is proposed. Experiments show that the proposed summarization method successfully reduces the video content while keeping important events. A power analysis of the system also shows that a significant amount of energy can be saved. <s> BIB004 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> In order to reduce the required transmission bandwidth of a video sensor network for Internet-of-Things applications, a video sensor node with distributed video summary capability is developed, where the video summarization algorithm is developed based on a scene model and a frame color descriptor. Experimental results show that by exchanging information between video sensors, the required transmission bandwidth is drastically reduced. Moreover, real-time performance is achieved on an embedded platform. <s> BIB005 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> Internet-enabled cameras pervade daily life, generating a huge amount of data, but most of the video they generate is transmitted over wires and analyzed offline with a human in the loop. The ubiquity of cameras limits the amount of video that can be sent to the cloud, especially on wireless networks where capacity is at a premium. In this paper, we present Vigil, a real-time distributed wireless surveillance system that leverages edge computing to support real-time tracking and surveillance in enterprise campuses, retail stores, and across smart cities. Vigil intelligently partitions video processing between edge computing nodes co-located with cameras and the cloud to save wireless capacity, which can then be dedicated to Wi-Fi hotspots, offsetting their cost. Novel video frame prioritization and traffic scheduling algorithms further optimize Vigil's bandwidth utilization. We have deployed Vigil across three sites in both whitespace and Wi-Fi networks. Depending on the level of activity in the scene, experimental results show that Vigil allows a video surveillance system to support a geographical area of coverage between five and 200 times greater than an approach that simply streams video over the wireless network. For a fixed region of coverage and bandwidth, Vigil outperforms the default equal throughput allocation strategy of Wi-Fi by delivering up to 25% more objects relevant to a user's query. <s> BIB006 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> Deploying an IoT (Internet of Things) system over the next-generation mobile telecommunication could lead to an issue of “Low Spectrum Utilization”. To cope with this challenge caused by employing applications of “Environmental Monitoring” using a NB (Narrow Band) radio spectrum. This paper proposes a B4G (Beyond 4G) automatic TBS (Transport Block Size) index selection scheme to optimize the spectrum utilization for video-stream transmissions between IoT devices. Potential performance advantages in terms of coding scheme selection and radio spectrum utilization were evaluated. Moreover, variations of available bandwidth caused by noise upsets during packet transmissions were analyzed to validate the practice of our proposed TBS index selection scheme. <s> BIB007 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> With the increasing importance of information processing in industries arises the need for efficient data logging systems that are compatible with existing measuring devices such as LCD and LED meters. This paper presents the framework for an Internet of Things (IoT) device as an automated industrial meter reader that uploads the collected numeral data to a cloud storage for centralized data processing. The implementation of the device is done using Raspberry Pi as the platform. The device follows a four-step process-Image Acquisition using Raspberry Pi camera module, Optical Character Recognition using feature extraction technique, Internet Upload Mechanism using Google Forms and Online Data Processing using Google Spreadsheet. The performance of the device and techniques for debugging are also discussed. <s> BIB008 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> Image or video exchange over the Internet of Things (IoT) is a requirement in diverse applications, including smart health care, smart structures, and smart transportations. This paper presents a modular and extensible quadrotor architecture and its specific prototyping for automatic tracking applications. The architecture is extensible and based on off-the-shelf components for easy system prototyping. A target tracking and acquisition application is presented in detail to demonstrate the power and flexibility of the proposed design. Complete design details of the platform are also presented. The designed module implements the basic proportional–integral–derivative control and a custom target acquisition algorithm. Details of the sliding-window-based algorithm are also presented. This algorithm performs $20\times $ faster than comparable approaches in OpenCV with equal accuracy. Additional modules can be integrated for more complex applications, such as search-and-rescue, automatic object tracking, and traffic congestion analysis. A hardware architecture for the newly introduced Better Portable Graphics (BPG) compression algorithm is also introduced in the framework of the extensible quadrotor architecture. Since its introduction in 1987, the Joint Photographic Experts Group (JPEG) graphics format has been the de facto choice for image compression. However, the new compression technique BPG outperforms the JPEG in terms of compression quality and size of the compressed file. The objective is to present a hardware architecture for enhanced real-time compression of the image. Finally, a prototyping platform of a hardware architecture for a secure digital camera (SDC) integrated with the secure BPG (SBPG) compression algorithm is presented. The proposed architecture is suitable for high-performance imaging in the IoT and is prototyped in Simulink. To the best of our knowledge, this is the first ever proposed hardware architecture for SBPG compression integrated with an SDC. <s> BIB009 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> In recent years, video surveillance technology has become ubiquitous in every sphere of our life. But automated video surveillance generates huge quantities of data, which ultimately does rely upon manual inspection at some stage. The present work aims to address this ever increasing gap between the volumes of actual data generated and the volume that can be reasonably inspected manually. It is laborious and time consuming to scrutinize the salient events from the large video databases. We introduce smart surveillance by using video summarization for various applications. Techniques like video summarization epitomizes the vast content of a video in a succinct manner. In this paper, we give an overview how to use an optimal summarization framework for surveillance videos. In addition to reduce the search time we propose to convert content based video retrieval problem into a content based image retrieval problem. We have performed several experiments on different data sets to validate our proposed approach for smart surveillance. <s> BIB010 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> In order to address the high transmission bandwidth requirement of an Internet-of-Video-Things (IoVT), an object-based on-line video summarization algorithm is proposed to summarize the captured video information at the sensor nodes before being transmitted to the server. It is composed of two stages: intra-view and inter-view stages. In the intra-view stage, human object detector is employed with the proposed human object descriptor. In the inter-view stage, an on-line clustering algorithm with a two-layer K-nearest-neighbor model is also proposed for object clustering. Experimental results show that significant improvement can be achieved when compared with state-of-the-art works. <s> BIB011 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> In this paper, we propose a video-based indoor positioning system (VIPS) to provide centimeter-grade localization for the Internet of Things (IoT). The VIPS system integrates installed surveillance cameras with the IoT to provide highly-accurate positioning information to indoor people, which efficiently enables microlocation-aware IoT applications and services in smart environments. VIPS can estimate the precise position of an individual and detect the facing direction for each individual. To the best of our knowledge, VIPS is the first indoor positioning system which provides the following features: 1) it achieves the centimeter-grade positioning accuracy for the IoT that can detect the facing direction of individuals, 2) it employs existing surveillance cameras that can keep additional construction cost low, and 3) it establishes an innovative infrastructure for next-generation IoT applications that can make centimeter-accuracy individual-based path planning, group-based emergency evacuation, and microlocation-based geofencing possible. With the IoT, the estimated positions can be transmitted to corresponding individuals for interacting with nearby smart objects. With surveillance cameras, the facing direction of an individual can be detected even if her/his current position is not changed. This paper demonstrates our current prototype. <s> BIB012 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> In this paper, we design a novel multiview video encoding/decoding architecture for wirelessly multiview video streaming applications, e.g., 360 degrees video, Internet of Things (IoT) multimedia sensing, among others, based on distributed video coding and compressed sensing principles. Specifically, we focus on joint decoding of independently encoded compressively sampled multiview video streams. We first propose a novel side-information (SI) generation method based on a new interview motion compensation algorithm for multiview video joint reconstruction at the decoder end. Then, we propose a technique to fuse the received measurements with resampled measurements from the generated SI to perform the final recovery. Based on the proposed joint reconstruction method, we also derive a blind video quality estimation technique that can be used to adapt online the video encoding rate at the sensors to guarantee desired quality levels in multiview video streaming. Extensive simulation results of real multiview video traces show the effectiveness of the proposed fusion reconstruction method with the assistance of SI generated by an interview motion compensation method. Moreover, they also illustrate that the blind quality estimation algorithm can accurately estimate the reconstruction quality. <s> BIB013 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> Supporting city-wide exchange of information in Urban Internet of Things (IoT) systems using existing communication infrastructures is extremely challenging especially when traditional services operate in the same network resource. Additionally, the most advanced Urban IoT services focus on real-time data processing, which shifts the perspective and goal of the network when transporting data. In this paper, the notion of Quality of Computing (QoC) is introduced to capture the level of support the communication infrastructure provides to this family of computation applications. In this context, we propose a dynamic network selection mechanism based on Software Defined Networks (SDN) designed to provide QoC in Urban IoT scenarios where the heterogeneous network resources are shared. The proposed mechanism dynamically assigns portions of data from IoT streams over licensed and unlicensed bands to guarantee QoC while minimizing cost of operations and licensed band occupation. Instrumental to our technique is the recently proposed edge-computing architecture, where computational resources placed at the edge of wireless access networks enable the interconnection of network management to processing. We consider a real-time monitoring scenario, where sensors transmit a video stream which is processed to identify and classify objects. The supporting wireless infrastructure consists of WiFi that operates in unlicensed frequency bands and cellular communication technology, Long Term Evolution (LTE) operating in licensed bands. We demonstrate the performance by means of real-world experiments on a testbed with WiFi and LTE networks built with hostapd and OpenAirInterface. <s> BIB014 </s> Multimedia Internet of Things: A Comprehensive Survey <s> d: SURVEILLANCE FOR PUBLIC SAFETY <s> Cloud Computing (CL) and Internet of Things (IoT) solutions inspire various applications by providing connectivity, storage, and computational capabilities. Those solution are essential in providing 5G services. One of the common application of IoT systems are Mobile Video Surveillance Systems where the thing are cameras. Mobile edge computing (MEC) technologies aim to provide CL services, completely or partially, at the network edge to reduce service delay. Integrating CL and MEC into a Mobile Video Surveillance System (MVSS) offloads management and computation burdens from the MSS into the MEC and CL environments. However, an MVSS would still need to provide the minimal functional and security requirements. This paper proposes a seamless key management and distribution system among MEC and MVSS devices. In our proposed system, secure channels are established between all the communicating entities: Camera-to-MEC, MEC-to-MEC, and MEC-to-CL. One-time-key establishment between a camera and a MEC is required. Afterwards, the authentication process takes place with the same key. The key is forwarded between the MEC devices through their pre-established secure channels. This framework helps develop secure and efficient CL-based applications. <s> BIB015
With the increasing deployment of IoT in our daily life and decline in the implementation costs, video surveillance systems became a key requirement for the smart city to ensure public safety for crime detection, home security system, industrial surveillance, and other localization-related features. The immobility of the PC based surveillance system encourages the researcher to propose IoT based security systems. Multimedia communication over IoT enables us to achieve this objective. Basri et al. in  proposed IoT residential surveillance system. The proposed scheme is an Android-based application and hardware which includes Raspberry Pi, Pi camera, PIR sensor, and ultrasonic sensors to alert the owner of a house with an intruder. PIR sensor detects the suspicious movement within the specified range, and the image taken by Pi camera is stored in the memory card. An email or alert message is sent to the owner by Raspberry Pi. To detect emergency and generate alerts for the older person at home, an Android application is designed and implemented based on the Real-Time Stream Protocol (RTSP) and Real-time Transport Protocol (RTP) BIB003 . The video is captured, processed, and encoded using H.264 at the server end. Client-end includes smartphones, tablets, or PC, can access live streaming using RTSP and video is decoded using FFMPEG. Security is further enhanced for the elderly by face detection feature using OpenCV library for intrusion detection, and critical screenshots are saved for emergency alerts. A natural language processing interface combined with ontologies and image analysis based video surveillance approach is proposed in BIB001 for content-based retrieval of visual data. The proposal aims to achieve passenger safety in the public transport system. Four ontologies are utilized, i.e., DAML time, properties, object, and event. PCA and SVM are used for face detection and gender classification. Sphinx is used for audio mapping with semantics formed by ontology vocabularies. Infants baby monitoring system for the detection of sudden infant death syndrome (SIDS) is presented in  . A video camera exploiting the Eulerian motion magnification technique developed by MIT to monitor and detect infant chest motion during breathing is proposed. When an emergency is identified, an alert message is sent to a parent's smartphone via Twilio, which is a cloud-based communication platform  . Video summarization to reduce the search time from big multimedia data generated by the surveillance system and content image-based retrieval techniques for smart surveillance is detailed in BIB010 . Aggregated Channel Feature (ACF) extraction and bounding box approach for moving objects are used, and the cost is determined if any critical situation is observed. Image with a high-cost box is summarized and utilized for video retrieval on search, which reduces the time, memory, and computation requirements of the system. Lin et al. BIB011 proposed a multi-view video summarization method to address the high bandwidth transmission requirement for the Internet of Video Things (IoVT). The proposed scheme is based on the K-Nearest Neighbour (KNN) model. The performance evaluation is measured by making a comparison with the GMM based summarization model based on precision metrics, i.e., removal of redundant data, and the security of critical information. An online multi-view summarization algorithm and RPi based distributed video summarization sensor node is developed in BIB004 and BIB005 . Al-Saleh et al. BIB015 presented a secure framework for Mobile Video Surveillance Systems (MVSS) by considering a key management system to secure channels between all the entities, i.e., camera, Mobile Edge Computing (MEC), MEC to MEC, and MEC to cloud. An optimized spectrum utilization scheme for video streaming in environment monitoring by automatic Transport Block Size (TBS) index selection using narrow-band Beyond 4G (B4G) is proposed in BIB007 . Video-based Indoor Positioning System (VIPS) estimating the precise face detection of each individual with centimeter grade accuracy is presented in BIB012 . The prototype comprises two units, i.e., an Indoor Positioning System (IPS) and Mobile Handled Unit (MHU). IPS is responsible for capturing the frame of individuals approaching an area, the face is detected for all individuals and mapped to the area location, after which the positioning information is broadcasted to MHU which compares the individual information in its database with the received information. Cen et al. BIB013 proposed an efficient video compression and encoding/decoding technique for wireless multiview video streaming applications in IoT to enhance the energy efficiency of the system. The researcher in BIB006 presents the concept of distributed edge computing to minimize the video processing load at the cloud for real-time wireless multiview surveillance systems. Authors name this approach as Vigil, a camera with Edge Computing Node (ECN), which priorities the video frames from multi-camera deployed in the same region for multiview and intelligently schedule them for efficient resource utilization. It also provides a user input interface for a specific query. Slicing and parallel processing approaches to efficiently handle the bulky nature of IoVT are proposed in BIB002 . sTune architecture is followed in this approach, in which a metadata manager manages several cloud storage endpoints. The processing unit is composed of a master node and several slave nodes. The master node is capable of retrieving data from sTune and assigns the slave nodes to process the sliced video data in parallel. After processing master nodes stores the processed data back to the cloud. CPU and memory usage are considered as performance metrics to evaluate the performance. A multimedia application requesting the same content from M-IoT is delivered separately over the core network that increases the overhead. To cope with this issue, Silva and Neto  propose SCORPIO, an SDN control plane to provide multimedia multipart transport services to duplicate the multimedia packets at the edge of the network and map them with the applications of common interest in the data plane path. The performance metrics to evaluate the proposed methodology are throughput, jitter, and packet dropping ratio. VOLUME 8, 2020  ), combustion quality maitenance  , and industrial meter reading BIB008 . (b) Agricultural application of M-IoT for crop monitoring for production control ( [134]-  ). BIB014 to determine the occupancy of the network. The article presents an SDN based network selection approach to maintain QoC in HetNet. The proposed approach selects between the licensed band (LTE) or unlicensed (WiFi) for real-time services. The data aggregated at the sensor is transmitted to the edge processor over upstream links, i.e., LTE and WiFi. The Edge processor evaluates the received packets according to the data path and computational goals, and feedback is sent on a reverse path. Kougianos et al. BIB009 proposed a novel concept of using quadrotors for IoT real-time automatic tracking applications.