A survey of visual sensor network platforms <s> Compression and coding <s> Video-based sensor networks can provide important visual information in a number of applications including: environmental monitoring, health care, emergency response, and video security. This paper describes the Panoptes video-based sensor networking architecture, including its design, implementation, and performance. We describe a video sensor platform that can deliver high-quality video over 802.11 networks with a power requirement of approximately 5 watts. In addition, we describe the streaming and prioritization mechanisms that we have designed to allow it to survive long-periods of disconnected operation. Finally, we describe a sample application and bitmapping algorithm that we have implemented to show the usefulness of our platform. Our experiments include an in-depth analysis of the bottlenecks within the system as well as power measurements for the various components of the system. <s> BIB001 </s> A survey of visual sensor network platforms <s> Compression and coding <s> Distributed coding is a new paradigm for video compression, based on Slepian and Wolf's and Wyner and Ziv's information-theoretic results from the 1970s. This paper reviews the recent development of practical distributed video coding schemes. Wyner-Ziv coding, i.e., lossy compression with receiver side information, enables low-complexity video encoding where the bulk of the computation is shifted to the decoder. Since the interframe dependence of the video sequence is exploited only at the decoder, an intraframe encoder can be combined with an interframe decoder. The rate-distortion performance is superior to conventional intraframe coding, but there is still a gap relative to conventional motion-compensated interframe coding. Wyner-Ziv coding is naturally robust against transmission errors and can be used for joint source-channel coding. A Wyner-Ziv MPEG encoder that protects the video waveform rather than the compressed bit stream achieves graceful degradation under deteriorating channel conditions without a layered signal representation. <s> BIB002 </s> A survey of visual sensor network platforms <s> Compression and coding <s> This paper addresses the important aspect of compressing and transmitting video signals generated by wireless broadband networks while heeding the architectural demands imposed by these networks in terms of energy constraints as well as the channel uncertainty related to the wireless communication medium. Driven by the need to develop light, robust, energy-efficient, and low delay video delivery schemes, a distributed video coding based framework dubbed PRISM is introduced. PRISM addresses the wireless video sensor network requirements far more effectively than current state-of-the-art standards like MPEG. This paper focuses on the case of a single video camera and use it as a platform to describe the theoretical principles and practical aspects underlying distributed video coding. <s> BIB003 </s> A survey of visual sensor network platforms <s> Compression and coding <s> We introduce Meerkats, a wireless network of batteryoperated camera nodes that can be used for monitoring and surveillance of wide areas. One distinguishing feature of Meerkats (when compared, for example, with systems like Cyclops [10]) is that our nodes are equipped with sufficient processing and storage capabilities to be able to run relatively sophisticated vision algorithms (e.g., motion estimation) locally and/or collaboratively. In previous work [9, 8, 7] we analyzed the energy consumption characteristics of the Meerkats nodes under different duty cycles, involving different power states of the system’s components. In this paper we present an analysis of the performance of the surveillance system as a function of the image acquisition rate and of the synchronization between cameras nodes. Our ultimate goal is to optimally balance the trade-off between application-specific performance requirements (e.g., event miss rate) and network lifetime (as a function of the energy consumption characteristics of each node). <s> BIB004 </s> A survey of visual sensor network platforms <s> Compression and coding <s> Compressive sampling is an emerging theory that is based on the fact that a relatively small number of random projections of a signal can contain most of its salient information. In this paper, we introduce the concept of compressive wireless sensing for sensor networks in which a fusion center retrieves signal field information from an ensemble of spatially distributed sensor nodes. Energy and bandwidth are scarce resources in sensor networks and the relevant metrics of interest in our context are 1) the latency involved in information retrieval; and 2) the associated power-distortion trade-off. It is generally recognized that given sufficient prior knowledge about the sensed data (e.g., statistical characterization, homogeneity etc.), there exist schemes that have very favorable power-distortion-latency trade-offs. We propose a distributed matched source-channel communication scheme, based in part on recent results in compressive sampling theory, for estimation of sensed data at the fusion center and analyze, as a function of number of sensor nodes, the trade-offs between power, distortion and latency. Compressive wireless sensing is a universal scheme in the sense that it requires no prior knowledge about the sensed data. This universality, however, comes at the cost of optimality (in terms of a less favorable power-distortion-latency trade-off) and we quantify this cost relative to the case when sufficient prior information about the sensed data is assumed. <s> BIB005 </s> A survey of visual sensor network platforms <s> Compression and coding <s> Suppose x is an unknown vector in Ropfm (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural classes of images with m pixels need only n=O(m1/4log5/2(m)) nonadaptive nonpixel samples for faithful recovery, as opposed to the usual m pixel samples. More specifically, suppose x has a sparse representation in some orthonormal basis (e.g., wavelet, Fourier) or tight frame (e.g., curvelet, Gabor)-so the coefficients belong to an lscrp ball for 0<ples1. The N most important coefficients in that expansion allow reconstruction with lscr2 error O(N1/2-1p/). It is possible to design n=O(Nlog(m)) nonadaptive measurements allowing reconstruction with accuracy comparable to that attainable with direct knowledge of the N most important coefficients. Moreover, a good approximation to those N important coefficients is extracted from the n measurements by solving a linear program-Basis Pursuit in signal processing. The nonadaptive measurements have the character of "random" linear combinations of basis/frame elements. Our results use the notions of optimal recovery, of n-widths, and information-based complexity. We estimate the Gel'fand n-widths of lscrp balls in high-dimensional Euclidean space in the case 0<ples1, and give a criterion identifying near- optimal subspaces for Gel'fand n-widths. We show that "most" subspaces are near-optimal, and show that convex optimization (Basis Pursuit) is a near-optimal way to extract information derived from these near-optimal subspaces <s> BIB006 </s> A survey of visual sensor network platforms <s> Compression and coding <s> This paper develops a new framework for distributed coding and compression in sensor networks based on distributed compressed sensing (DCS). DCS exploits both intra-signal and inter-signal correlations through the concept of joint sparsity; just a few measurements of a jointly sparse signal ensemble contain enough information for reconstruction. DCS is well-suited for sensor network applications, thanks to its simplicity, universality, computational asymmetry, tolerance to quantization and noise, robustness to measurement loss, and scalability. It also requires absolutely no inter-sensor collaboration. We apply our framework to several real world datasets to validate the framework. <s> BIB007 </s> A survey of visual sensor network platforms <s> Compression and coding <s> Wireless video sensor networks (WVSNs) have been envisioned for a wide range of important applications, including battlefield intelligence, security monitoring, emergency response, and environmental tracking. Compared to traditional communication system, the WVSN operates under a set of unique resource constraints, including limitations with respect to energy supply, on-board computational capability, and transmission bandwidth. The objective of this paper is to study the resource utilization behavior of a wireless video sensor and analyze its performance under the resource constraints. More specifically, we develop an analytic power-rate-distortion (P-R-D) model to characterize the inherent relationship between the power consumption of a video encoder and its rate-distortion performance. Based on the P-R-D analysis and a simplified model for wireless transmission power, we study the optimum power allocation between video encoding and wireless transmission and introduce a measure called achievable minimum distortion to quantify the distortion under a total power constraint. We consider two scenarios in wireless video sensing, small-delay wireless video monitoring and large-delay wireless video surveillance, and analyze the performance limit of the wireless video sensor in each scenario. The analysis and results obtained in this paper provide an important guideline for practical wireless video sensor design. <s> BIB008 </s> A survey of visual sensor network platforms <s> Compression and coding <s> With the advent of CMOS cameras, it is now possible to make compact, cheap and low-power image sensors capable of on-board image processing. These embedded vision sensors provide a rich new sensing modality enabling new classes of wireless sensor networking applications. In order to build these applications, system designers need to overcome challenges associated with limited bandwidth, limited power, group coordination and fusing of multiple camera views with various other sensory inputs. Real-time properties must be upheld if multiple vision sensors are to process data, communicate with each other and make a group decision before the measured environmental feature changes. In this paper, we present FireFly Mosaic, a wireless sensor network image processing framework with operating system, networking and image processing primitives that assist in the development of distributed vision-sensing tasks. Each FireFly Mosaic wireless camera consists of a FireFly (Rowe et al., 2006) node coupled with a CMUcam3 (Rowe et al., 2007) embedded vision processor. The FireFly nodes run the nano-RK (Eswaran et al., 2005) real-time operating system and communicate using the RT-link (Rowe et al., 2006) collision-free TDMA link protocol. Using FireFly Mosaic, we demonstrate an assisted living application capable of fusing multiple cameras with overlapping views to discover and monitor daily activities in a home. Using this application, we show how an integrated platform with support for time synchronization, a collision-free TDMA link layer, an underlying RTOS and an interface to an embedded vision sensor provides a stable framework for distributed real-time vision processing. To the best of our knowledge, this is the first wireless sensor networking system to integrate multiple coordinating cameras performing local processing. <s> BIB009 </s> A survey of visual sensor network platforms <s> Compression and coding <s> INTRODUCTION Surveillance isoneofthepromising applications towhichsmart Distributed smartcameras havereceived increased focus inthe cameramotesforming a vision-enabled network canadd research community overthepastseveral months. Thenotion of increasing levels ofintelligence. We seeahighdegree ofin-node camerascombined withembeddedcomputation powerand processing incombination withdistributed reasoning algorithmsinterconnected through radio links opensupa newrealmof asthekeyenablers forsuchintelligent surveillance systems. To intelligent vision-enabled applications. Real-time image putthesesystems intopractice still requires a considerableprocessing anddistributed reasoning madepossible bysmart amountofresearch ranging frommotearchitectures, pixel- cameras cannotonlyenhance existing applications butalso processing algorithms, uptodistributed reasoning engines. This motivate newapplications. Potential application areas range from paperintroduces MeshEye, anenergy-efficient smartcamera homemonitoring, elderly care, andsmartenvironments to motearchitecture thathasbeendesigned withintelligent security andsurveillance inpublic orcorporate buildings. Critical surveillance asthetarget application inmind.Special attention is issues influencing thesuccess ofsmartcameradeployments for given toMeshEye's unique vision system: alow-resolution stereo suchapplications include reliable androbust operation withas vision system continuously determines position, range, andsize of little maintenance aspossible. movingobjects entering itsfield ofview.Thisinformation triggers acolor cameramodule toacquire ahigh-resolution image ncmpaisonito alar sensors, shsm r e pess sub-array cotinn th obet whic ca.eefiinlhumidity, velocity, andacceleration sensors, vision sensors pub-arocessedainysubseq staeIt offerredced complextly generate muchhigher bandwidth dataduetothetwo-dimensional processed insubsequent stages. Itoffers reduced complexity, naueothipxlary.Tesermutofawda response~~~~~~~~ tie.n oe osmto vrcnetoa nature oftheir pixel array. Thesheeramountofrawdata <s> BIB010 </s> A survey of visual sensor network platforms <s> Compression and coding <s> In this paper we propose MicrelEye, a wireless video node for cooperative distributed video processing applications that involve image classification. The node is equipped with a low-cost VGA CMOS image sensor, a reconfigurable processing engine (FPGA, Microcontroller, SRAM) and a Bluetooth 100-m transceiver. It has a size of few cubic centimeters and its typical power consumption is approximately ten times less than that of typical commercial DSP-based solutions. As regards classification, a highly optimized hardware-oriented support vector machine-like (SVM-like) algorithm called ERSVM is proposed and implemented. We describe our hardware and software architecture, its performance and power characteristics. The case study considered in this paper is people detection. The obtained results suggest that the present technology allows for the design of simple intelligent video nodes capable of performing classification tasks locally. <s> BIB011 </s> A survey of visual sensor network platforms <s> Compression and coding <s> To meet the requirements of resource-limited video sensors, low-complexity video encoding technique is highly desired. In this paper, a low-complexity multi-view distributed video encoding scheme by using the correlations among video frames from adjacent video sensor nodes (VSNs) via robust media hashing at encoder and the global motion parameters estimated and fed back from the decoder is proposed. The frames from adjacent VSNs are warped into the same view-direction based on the global motion parameters. Then, the significant differences between the warped key frame and the non-key frame from adjacent VSNs are efficiently extracted based on robust media hashing for non-key frame compression. The key is that few data (hash information) exchanges among adjacent VSNs are allowed to efficiently exploit the correlations among VSNs. The coding performance and energy consumption of the proposed encoder have been verified through simulations and comparisons with existing low-complexity video encoders. <s> BIB012 </s> A survey of visual sensor network platforms <s> Compression and coding <s> Currently most energy-constrained wireless sensor networks are designed with the object of minimizing the communication power at the cost of more computation. To achieve high compression efficiency, the main image compression algorithms used in wireless sensor networks are the high-complexity, state-of-the-art image compression standards, such as JPEG2000. These algorithms require complex hardware and make the energy consumption for computation comparable to communication energy dissipation. To reduce the hardware cost and the energy consumption of the sensor network, a low-complexity and energy efficient image compression scheme is proposed. The compression algorithm in the proposed scheme greatly lowers the computational complexity and reduces the required memory, while it still achieves required PSNR. The proposed implementation scheme of the image compression algorithm overcomes the computation and energy limitation of individual nodes by sharing the processing of tasks. And, it applies transmission range adjustment to save communication energy dissipation. Performance of the proposed scheme is investigated with respect to image quality and energy consumption. Simulation results show that it greatly prolongs the lifetime of the network under a specific image quality requirement. <s> BIB013 </s> A survey of visual sensor network platforms <s> Compression and coding <s> In this paper, we propose and demonstrate a novel wireless camera network system, called CITRIC. The core component of this system is a new hardware platform that integrates a camera, a frequency-scalable (up to 624 MHz) CPU, 16MB FLASH, and 64MB RAM onto a single device. The device then connects with a standard sensor network mote to form a camera mote. The design enables in-network processing of images to reduce communication requirements, which has traditionally been high in existing camera networks with centralized processing. We also propose a back-end client/server architecture to provide a user interface to the system and support further centralized processing for higher-level applications. Our camera mote enables a wider variety of distributed pattern recognition applications than traditional platforms because it provides more computing power and tighter integration of physical components while still consuming relatively little power. Furthermore, the mote easily integrates with existing low-bandwidth sensor networks because it can communicate over the IEEE 802.15.4 protocol with other sensor network platforms. We demonstrate our system on three applications: image compression, target tracking, and camera localization. <s> BIB014 </s> A survey of visual sensor network platforms <s> Compression and coding <s> This paper considers the problem of communicating correlated information from multiple source nodes over a network of noiseless channels to multiple destination nodes, where each destination node wants to recover all sources. The problem involves a joint consideration of distributed compression and network information relaying. Although the optimal rate region has been theoretically characterized, it was not clear how to design practical communication schemes with low complexity. This work provides a partial solution to this problem by proposing a low-complexity scheme for the special case with two sources whose correlation is characterized by a binary symmetric channel. Our scheme is based on a careful combination of linear syndrome-based Slepian-Wolf coding and random linear mixing (network coding). It is in general suboptimal; however, its low complexity and robustness to network dynamics make it suitable for practical implementation. <s> BIB015 </s> A survey of visual sensor network platforms <s> Compression and coding <s> Video is an important medium for the observation of a variety of phenomena in the physical world such as for water conservancy engineering. Video sensor networks (VSN) consists of a large number of sensor nodes with cheap CMOS cameras, therefore, it has the ability of large scale visual monitoring. However, a kind of powerful and scalable platform is urgent for simulation and development of VSN. This paper proposed a novel and scalable video sensor networks platform named as Vision Mesh for water conservancy engineering. Vision Mesh is composed of a mass of image or video sensor nodes - Vision Motes, with which multi-view image or video information of FOV (Field of View) can be acquired simply. Vision Mote is built with Atmel ARM9 CPU and operated on Linux operating system, with TI CC2430 Zigbee module taken as wireless transceiver. Furthermore, OpenCV machine vision lib is migrated to Vision Mesh platform so as to improve video processing ability, therefore, Vision Mesh has the ability of image and video processing and is of strong scalability to extend performance. In this paper, we provide an overview of Vision Mesh architecture and provide an insightful platform for VSN. <s> BIB016
Transform coding also known as intra-coding (e.g., Discrete Cosine Transform-DCT and Wavelet Transforms-DWT) is a method used for lossy compression of still images and video. Intra-coding typically achieves poor compression for video since it does not exploit temporal correlation although it has high robustness. For video compression, transform coding is combined with motion-compensated prediction (inter-coding). One of the most used techniques is block-based predictive motion compensation technique. Inter-coding achieves high compression at the cost of high complexity. This means that one or more frames should be stored at the encoder and the decoder. Furthermore, predictive motion compensation BIB003 suffers from sensitivity to synchronization between coder and decoder. Error prone wireless channels often cause packet losses affecting the integrity of data and inducing prediction mismatches. Thus, the use of predictive motion-compensated coding techniques necessitates powerful computation and ample storage capabilities, which are not well suited for VSN platforms. A full search block motion estimation algorithm incurs around 65,000 operations per pixel per second for a 30 fps (frames per second) video as reported in BIB003 . To illustrate the communication and computation energy dissipation terms we use the example presented in  1 : processing with QCIF (176×144) resolution and 8-bit/pixel coding would cost the energy equivalent of transmission of approximately 4,400 Kb and processing with CIF (352×288) resolution and 8-bit/pixel coding would cost the energy equivalent of transmission of approximately 17,600 Kb. Color provides multiple measurements at a single pixel of the image. The 24-bit RGB encoding uses 3 bytes for each basic color (red-green-blue) enabling (2 8 ) BIB004 colors. Other schemes use 15-bit RGB (5 bits for each color component) or 16-bit RGB with emphasized green sensitivity. YCbCr encoding, typically used in JPEG and MPEG coders, allows a better representation for image storage and transmission. The amount of data to be sent becomes larger as the color-coding technique becomes richer. For example, a sensor using CIF (352×288) resolution with 512 KB FLASH memory can store 41.37 1-bit (black and white) frames, 5.17 8-bit color frames or 2.58 16-bit color frames. There is, thus, a clear trade-off between sensor capabilities (i.e., buffering, computation, and energy) and resolution, coding, compression, and video frame rate (see Fig. 1 ). Most of the cameras and imagers allow the choice of a wide range of resolutions and color encodings. Table 2 shows image resolutions and encoding and Table 3 shows compression algorithms used in VSN platforms. Below, we provide a general discussion and comparison of VSN platforms based on compression and coding techniques. Detailed descriptions of VSN platforms are specified in Section 4. The behavior of JPEG compression is investigated in most of the platforms. For example, in BIB009 it is shown that by using the FireFly Mosaic platform, the JPEG processing time varies little with the quality level but greatly with the resolution of the image. Other platforms, like Panoptes BIB001 are designed with more powerful hardware (Intel ARM 206 MHz) and able to support more complex compression algorithms. Panoptes designers compare a standard JPEG compression algorithm (ChenDCT) with their own algorithm optimized for the Panoptes platform. They showed that optimized compression could be performed with significantly lower energy (i.e., they reported only execution times, but assuming similar power per unit time, longer execution time implies more energy dissipation). Cyclops designers also report the implementation of their own JPEG compression algorithm, showing that their JPEG implementation consumes only 105.5 μJ to compress the test images, while the standard integer implementation from the Independent JPEG Group takes 133.15 μJ to compress the image with the same amount of Peak Signal to Noise Ratio performance. BIB005 Cyclops, Panoptes, Meerkats, FireFly Mosaic, CITRIC and Vision Mote (see Table 3 ) use JPEG or modified JPEG compression algorithms to perform intra-frame coding. No data compression or coding results are reported for either MeshEye BIB010 or MicrelEye BIB011 platforms. There is an evident need of low-complex and high efficient image compression mechanisms able to offer acceptable QoS in terms of power-rate-distortion parameters or The source voltage is not reported and assumed to be 3.6 V  BIB016 other quality metrics. The only prototypes that implement and report a scheme different than JPEG compression or its versions are XYZ-ALOHA and CITRIC. CITRIC implements compressed sensing; a new approach for simultaneous sensing and compression explained in this subsection. XYZ-ALOHA uses AER (Address Event Representation). AER only outputs a few features of interest of the visual scene by detecting intensity differences (motion) information. These sensors are sensitive to light and motion so that only pixels that include the brightest one generate the events first and more periodically than others. Thus, only pixels which realize a difference in the light intensity generate the events. Since, the sensor sends metadata and not images, there is a great saving in number of bits and thus in energy. There are alternatives other than modifying standard JPEG compression such as Distributed Video Coding (DVC) BIB002   BIB003 based on Slepian-Wolf and Wyner-Ziv theorems. DVC exploits the possibility to compress two statistically dependent signals using a separate encoding and a joint decoding approach, which is different from the classical predictive encoding that jointly encodes and decodes. This approach moves the complexity from the source node to the end node, giving room to design low-complexity and low-power sensors at the cost of the sink node. As reported in  , sink complexity may be a drawback if high node density sensor networks are deployed in terms of how many sensors such a sink could support and what DVC compression efficiency may be achieved with low encoding complexity. Nevertheless, the potential benefits of DVC in VSNs are due to the higher coding efficiency allowing (a) reduction in transmission rates; (b) low power consumption; (c) improved error resilience; (d) multi-view correlation exploitation. None of the existing VSN platforms and prototypes introduced in Section 4 has implemented and tested DVC. However, at the academic/research level there is a growing base of information about the use of DVC in VSNs. For instance, PRISM BIB003 aims to bring block-motion block-DCT to the decoder. In BIB013 , the use of Lapped Biorthogonal Transform (LBT) instead of DCT/DWT (Discrete Cosine/Wavelet Transforms) and Golomb+Multiple Quantization (MQ) methods instead of Huffman coding or arithmetic coding is proposed. To further reduce the compression complexity the authors propose sharing the processing tasks between nodes belonging to the same cluster. The result is a low-complexity and lowmemory demanding image compression algorithm that reduces hardware cost and energy consumption. In BIB012 , a DVC scheme is proposed for VSNs which exploits the correlations among video frames from adjacent nodes via robust media hashing. The global motion parameters are estimated by the decoder (the sink) and sent to the encoders. Compressed Sensing or Compressive Sampling (CS)  BIB006 BIB013 aims to compress data before storing the whole information to be compressed. CS works on the sparseness representation of the signal-instead of acquiring the whole signal  the complete set of coefficients are computed, the largest ones are encoded and the rest of the coefficients are discarded. CS assumes data is in y=Ax form, where A is a kxn sensing sparseness matrix (k«n). CS guarantees that for a certain matrix A, x may be reconstructed from y whenever x is compressible in some domain. Vector y is easy to compute (even in a distributed manner), has small size and does not require a prior knowledge about the data, being therefore a good candidate for compressing in VSN applications. Distributed Compressed Sensing (DCS) BIB007 exploits both intra and inter-signal correlation structures and is based on the concept of joint sparseness of a signal ensemble. Compressive Wireless Sensing (CWS) BIB005 reconstructs the sensed data from noisy random projections of the sensor network data and obtains good power-distortion trade-offs given the lack of prior knowledge about the signal. Network Coding  can help to minimize the amount of energy required per packet. Furthermore, the use of joint network coding BIB015 and distributed source coding to send multiple correlated sources to a common receiver is also proposed. CS and other related techniques are promising emerging techniques for coding and compression of data. At this moment, current platforms and prototypes focus on well-known intra-frame compression algorithms and rely on in-network processing for data reduction. An exception is CITRIC platform BIB014 where image compression via CS is implemented and tested. In CITRIC's CS implementation different sensors measure sparse but correlated signals and send them to a sink that is able to jointly reconstruct all the signals precisely. Data processing in VSN is more computationally intensive than scalar sensors. That makes energy dissipation for video coding/compression an important feature to be analyzed and researched. In scalar sensors, there is a clear objective in analyzing connectivity energy consumption (transmission and reception) of individual sensors and global network lifetime in terms of protocol energy performance. Nevertheless, visual sensors consume higher data processing energy than scalar sensors and therefore, there is a need to optimize energy consumption for data processing in individual visual sensors. In this sense, an analytical model BIB008 is introduced to characterize the relationship between encoding power consumption and its quality performance. The objective is to extend the traditional RateDistortion analysis to include energy constraints. Rate-Distortion and Time-Distortion trade-offs also are evaluated in CITRIC BIB014 , where it is shown that CS Time-Distortion tends to become zero while JPEG needs a residual computation time of 70 ms per image for the lowest distortion performance. Incorporating power consumption and rate-distortion analysis to video sensors is a promising research avenue since most of the current studies related to video coders in VSNs are using intra-and inter-frame compression techniques. The next subsection is dedicated to in-network vision computing mechanisms.