A survey of visual sensor network platforms <s> Vision computing <s> This paper argues that a camera sensor network containing heterogeneous elements provides numerous benefits over traditional homogeneous sensor networks. We present the design and implementation of senseye---a multi-tier network of heterogeneous wireless nodes and cameras. To demonstrate its benefits, we implement a surveillance application using senseye comprising three tasks: object detection, recognition and tracking. We propose novel mechanisms for low-power low-latency detection, low-latency wakeups, efficient recognition and tracking. Our techniques show that a multi-tier sensor network can reconcile the traditionally conflicting systems goals of latency and energy-efficiency. An experimental evaluation of our prototype shows that, when compared to a single-tier prototype, our multi-tier senseye can achieve an order of magnitude reduction in energy usage while providing comparable surveillance accuracy. <s> BIB001 </s> A survey of visual sensor network platforms <s> Vision computing <s> Although imaging is an information-rich sensing modality, the use of cameras in sensor networks is very often prohibited by factors such as power, computation cost, storage, communication bandwidth and privacy. In this paper we consider information selective and privacy-preserving address-event imagers for sensor networks. Instead of providing full images with a high degree of redundancy, our efforts in the design of these imagers specialize on selecting a handful of features from a scene and outputting these features in address-event representation. In this paper we present our initial results in modeling and evaluating address-event sensors in the context of sensor networks. Using three different platforms that we have developed, we illustrate how to model address-event cameras and how to build an emulator using these models. We also present a lightweight classification scheme to illustrate the computational advantages of address-event sensors. The paper concludes with an evaluation of the classification algorithm and a feasibility study of using COTS components to emulate address-event inside a sensor network. <s> BIB002 </s> A survey of visual sensor network platforms <s> Vision computing <s> We introduce Meerkats, a wireless network of batteryoperated camera nodes that can be used for monitoring and surveillance of wide areas. One distinguishing feature of Meerkats (when compared, for example, with systems like Cyclops [10]) is that our nodes are equipped with sufficient processing and storage capabilities to be able to run relatively sophisticated vision algorithms (e.g., motion estimation) locally and/or collaboratively. In previous work [9, 8, 7] we analyzed the energy consumption characteristics of the Meerkats nodes under different duty cycles, involving different power states of the system’s components. In this paper we present an analysis of the performance of the surveillance system as a function of the image acquisition rate and of the synchronization between cameras nodes. Our ultimate goal is to optimally balance the trade-off between application-specific performance requirements (e.g., event miss rate) and network lifetime (as a function of the energy consumption characteristics of each node). <s> BIB003 </s> A survey of visual sensor network platforms <s> Vision computing <s> In this paper we propose MicrelEye, a wireless video node for cooperative distributed video processing applications that involve image classification. The node is equipped with a low-cost VGA CMOS image sensor, a reconfigurable processing engine (FPGA, Microcontroller, SRAM) and a Bluetooth 100-m transceiver. It has a size of few cubic centimeters and its typical power consumption is approximately ten times less than that of typical commercial DSP-based solutions. As regards classification, a highly optimized hardware-oriented support vector machine-like (SVM-like) algorithm called ERSVM is proposed and implemented. We describe our hardware and software architecture, its performance and power characteristics. The case study considered in this paper is people detection. The obtained results suggest that the present technology allows for the design of simple intelligent video nodes capable of performing classification tasks locally. <s> BIB004 </s> A survey of visual sensor network platforms <s> Vision computing <s> With the advent of CMOS cameras, it is now possible to make compact, cheap and low-power image sensors capable of on-board image processing. These embedded vision sensors provide a rich new sensing modality enabling new classes of wireless sensor networking applications. In order to build these applications, system designers need to overcome challenges associated with limited bandwidth, limited power, group coordination and fusing of multiple camera views with various other sensory inputs. Real-time properties must be upheld if multiple vision sensors are to process data, communicate with each other and make a group decision before the measured environmental feature changes. In this paper, we present FireFly Mosaic, a wireless sensor network image processing framework with operating system, networking and image processing primitives that assist in the development of distributed vision-sensing tasks. Each FireFly Mosaic wireless camera consists of a FireFly (Rowe et al., 2006) node coupled with a CMUcam3 (Rowe et al., 2007) embedded vision processor. The FireFly nodes run the nano-RK (Eswaran et al., 2005) real-time operating system and communicate using the RT-link (Rowe et al., 2006) collision-free TDMA link protocol. Using FireFly Mosaic, we demonstrate an assisted living application capable of fusing multiple cameras with overlapping views to discover and monitor daily activities in a home. Using this application, we show how an integrated platform with support for time synchronization, a collision-free TDMA link layer, an underlying RTOS and an interface to an embedded vision sensor provides a stable framework for distributed real-time vision processing. To the best of our knowledge, this is the first wireless sensor networking system to integrate multiple coordinating cameras performing local processing. <s> BIB005
Techniques aimed to manipulate image content and, thus, allowing the transmission of a reduced amount of information are generally named as in-network processing techniques within VSN domain. Most of these techniques come from the vision computing field. Vision computing can reduce the amount of data to be sent to the sink. Well-known visual computing techniques, such as object recognition or object tracking  , can reduce the amount of raw data to be transmitted. Although there are many techniques for object recognition depending on the applications, general object recognition methods can be categorized as in Fig. 2 . Object/event detection and recognition can be done by using the identified features of the object. Object tracking can be done by localization algorithms (e.g., active contour-based, feature-based, and model-based). Vision-computing techniques implemented in VSN platforms are summarized in Table 3 . It is important to mention that most of these techniques require powerful processors and large memory resources if they are not customized for implementation in VSN platforms. Vision-computing techniques optimized for VSN platforms are implemented, tested and Most of the platforms utilize background subtraction, frame differencing, and edge detection as the main in-network processing techniques. Background subtraction is based on extracting the object of interest from the image and removing the background. It must be robust against changes in illumination and should avoid detecting non-stationary background objects (moving leaves, rain, snow, etc). Moving average filters smooth backgrounds with illumination changes or moving objects. Frame differencing detects changes between two consecutive frames. Differencing the current frame with a background frame averaged over past frames adds robustness against illumination changes. For example, Meerkats, Cyclops, MeshEye, MicrelEye, and CITRIC perform background subtraction and frame differencing as object detection mechanisms. MicrelEye takes a fixed background frame at the beginning and then performs pixel-by-pixel differencing between each frame and the background frame. Cyclops use moving average filters to smooth background changes. Firefly Mosaic uses a Gaussian Mixture Model (GMM) to separate foreground from background. GMM is able to detect objects even when the object stands still. Edges could be considered as boundaries between dissimilar regions in an image. Computation of edges are fairly cheap and recognition of an object is easy since it provides strong visual clues, however, edge detection can be affected by the noise in an image. Although there are several different methods to perform edge detection, generally they can be categorized into two classes: (a) gradient-based (e.g., Sobel algorithm); (b) Laplacianbased. While the first one detects the edges by looking for the maximum and minimum in the first derivative of the image, the second one searches for zero crossings in the second derivative of the image to find edges. Cyclops supports Sobel libraries to perform edge detection. Delay incurred by edge detection as a function of image size is investigated for XYZ platform BIB002 . For example, 8-bit Sobel edge detection lasts respectively 3,560 ms, 248 ms, 65 ms and 15 ms for 320×240, 256×64, 64×64 and 32×32 resolutions. However, the use of an FPGA block can decrease delay at reasonable energy costs. The results reported on energy dissipation and efficiency of the vision algorithms suggests that vision-computing is an essential component of the VSN paradigm. It is shown in BIB003 that processing energy dissipation is no longer negligible and is comparable to (even higher than) the energy dissipation for networking. Meerkats performs object detection based on background subtraction that simply detects motion taken between two snapshots at different times. Nodes wake-up periodically, perform object detection, and send a wakeup message to neighbors which, in response, start taking pictures. This duty-cycle method consumes more batteries than using the n-tier architecture, proposed in BIB001 . Power consumption and performance of object classification tasks are reported for MicrelEye BIB004 . The authors implement three hardware and software architectures (serial, parallelized and optimized) and demonstrated a power consumption of 430 mW at 5 fps for the serial implementation, 500 mW at 9 fps for the parallel implementation and 500 mW at 15 fps for the optimized implementation, showing that simple in-network processing tasks can be performed at moderate frame rates. Energy consumption will be one of the main challenges in the design of future in-network processing techniques. Exploiting spatial correlation between the images obtained by neighboring nodes can reduce power consumption. Collaborative image coding  computes differences between neighbor camera frames and local frames. Then, using edge detection (e.g.; Sobel operator) dominant features that are to be sent to the sink are computed. Before initiating the process it is necessary to obtain the background to later reconstruct the image, where synchronization of the cameras is the critical part. Experiments in  are performed using an Intel StrongARM SA 1100 as processor and LMX3162 as transceiver. An application in which, after an initial training phase, multiple overlapping cameras merge regions and collaborate in object activity detection is proposed for Firefly Mosaic platform BIB005 . However, the trade-off between the increase in signal processing for target detection and the transmission of data in light-weight sensors still remains as an active research issue. Collaborative image coding is a promising technique for VSNs; however, it has also drawbacks. In networks with scalar sensor nodes, the sensing area and the transmitting node's coverage area are basically the same. In visual sensor networks, sensing nodes are cameras with a FoV and a DoV (Depth of View). FoV is the maximum volume visible from a camera while DoV is the amount of distance between the nearest and farthest objects that appear in an image  . Overlapping cameras can cover the same sensing area depending on the FoV and the DoV; however, if the transmission radius is not large enough then VSN nodes with correlated data cannot directly reach each other. In this situation, collaborative image coding needs specific sensor coordination algorithms to reach out of range nodes.