Action monitoring in a home environment provides important information for health monitoring and may serve as input into a smart home environment. Visual analysis using cameras can recognise actions in a complex scene, such as someones living room. However, although there the huge potential benefits and importance, specifically for health, cameras are not widely accepted because of privacy concerns. This paper recognises human activities using a sensor that retains privacy. The sensor is not only different by being thermal, but it is also of low resolution: 8x8 pixels. The combination of the thermal imaging, and the low spatial resolution ensures the privacy of individuals. We present an approach to recognise daily activities using this sensor based on a discrete cosine transform. We evaluate the proposed method on a state-of-the-art dataset and experimentally confirm that our approach outperforms the baseline method. We also introduce a new dataset, and evaluate the method on it. Here we show that the sensor is considered better at detecting the occurrence of falls and Activities of Daily Living. Our method achieves an overall accuracy of 87.50 across 7 activities with a fall detection sensitivity of 100 and specificity of 99.21 .
An infrared sensor array is a device composed of a small number of discrete infrared sensors. It represents the spatial distribution of temperature as a low-resolution image. Unlike colour cameras, infrared sensor arrays only capture the shape of the human body, therefore making individual identification harder. Additionally, the low spatial resolution also makes identification of individuals difficult. As this is more comfortable for users, it becomes more acceptable for installation in residential environments. Such infrared sensor arrays can be applied in many scenarios. A 4x4 sensor array has been used to recognise hand motion directions @cite_5 , although the extremely low resolution of this sensor renders it unsuitable for more complex visual tasks. A 8x8 pixel sensor array has been successfully used to detect, count and track people indoors @cite_6 . Human movements has also be inferred by using the subject's location and moving trajectory using a 16x16 sensor array @cite_10 . Most recently, a multi-sensor system has been designed for human movement detection and activity recognition @cite_1 , which our method will be compared against.
@cite_1 @cite_5 @cite_10 @cite_6
2899374650 2287367556 957840212 2601917592
We propose a human body tracking method using a far-infrared sensor array. A far-infrared sensor array captures the spatial distribution of temperature as a low-resolution image. Since it is difficult to identify a person from the low-resolution thermal image, we can avoid privacy issues. Therefore, it is expected to be applied for the analysis of human behaviors in various places. However, it is difficult to accurately track humans because of the lack of information sufficient to describe the feature of the target human body in the low-resolution thermal image. In order to solve this problem, we propose a thermo-spatial sensitive histogram suitable to represent the target in the low-resolution thermal image. Unlike the conventional histograms, in case of the thermo-spatial sensitive histogram, a voting value is weighted depending on the distance to the target’s position and the difference from the target’s temperature. This histogram allows the accurate tracking by representing the target with multiple histograms and reducing the influence of the background pixels. Based on this histogram, the proposed method tracks humans robustly to occlusions, pose variations, and background clutters. We demonstrate the effectiveness of the method through an experiment using various image sequences. In this paper we present our work towards a hand gesture recognition system realised with a passive thermal infrared sensor array. In contrast with the majority of recent research activities into gesture recognition, which focus on the complex analysis of video sequences, our approach shows that the functionality of a simple pyroelectric movement sensor can be expanded to detect differing hand gestures at short range. We show that blob detection from a hand waving over a 16 element passive infrared sensor array provides sufficient information to discriminate four directions of hand stroke. This sensor system is unique and lends itself to low cost, low profile and low power applications. Keywords-touchless input device; dynamic hand gesture; infrared motion sensor; infrared sensor array; pyroelectricity; Low Resolution Thermal Array Sensors are widely used in several applications in indoor environments. In particular, one of these cheap, small and unobtrusive sensors provides a low-resolution thermal image of the environment and, unlike cameras; it is capable to detect human heat emission even in dark rooms. The obtained thermal data can be used to monitor older seniors while they are performing daily activities at home, to detect critical situations such as falls. Most of the studies in activity recognition using Thermal Array Sensors require human detection techniques to recognize humans passing in the sensor field of view. This paper aims to improve the accuracy of the algorithms used so far by considering the temperature environment variation. This method leverages an adaptive background estimation and a noise removal technique based on Kalman Filter. In order to properly validate the system, a novel installation of a single sensor has been implemented in a smart environment: the obtained results show an improvement in human detection accuracy with respect to the state of the art, especially in case of disturbed environments. Indoor tracking has all-pervasive applications beyond mere surveillance, for example in education, health monitoring, marketing, energy management and so on. Image and video based tracking systems are intrusive. Thermal array sensors on the other hand can provide coarse-grained tracking while preserving privacy of the subjects. The goal of the project is to facilitate motion detection and group proxemics modeling using an 8 x 8 infrared sensor array. Each of the 8 x 8 pixels is a temperature reading in Fahrenheit. We refer to each 8 x 8 matrix as a scene. We collected approximately 902 scenes with different configurations of human groups and different walking directions. We infer direction of motion of a subject across a set of scenes as left-to-right, right-to-left, up-to-down and down-to-up using cross-correlation analysis. We used features from connected component analysis of each background subtracted scene and performed Support Vector Machine classification to estimate number of instances of human subjects in the scene. A prototype real-time, smart sensor array for measuring soil moisture and soil temperature that uses off-the-shelf components was developed and evaluated for scheduling irrigation in cotton. The array consists of a centrally located receiver connected to a laptop computer and multiple sensor nodes installed in the field. The sensor nodes consist of sensors (up to three Watermark^(R) soil moisture sensors and up to four thermocouples), a specially designed circuit board, and a Radio Frequency IDentification (RFID) tag which transmits data to the receiver. The smart sensor array described here offers real potential for reliably monitoring spatially variable soil water status in crop fields. The relatively low cost of the system ( USD 2400 for a 20-sensor node system) allows for installation of a dense population of soil moisture sensors that can adequately represent the inherent soil variability present in fields. This paper describes the smart sensor array and testing in a cotton crop. Integration of the sensors with precision irrigation technologies will provide a closed loop irrigation system where inputs from the smart sensor array will determine timing and amounts for real-time site-specific irrigation applications. New vision sensors, such as the Dynamic and Active-pixel Vision sensor (DAVIS), incorporate a conventional camera and an event-based sensor in the same pixel array. These sensors have great potential for robotics because they allow us to combine the benefits of conventional cameras with those of event-based sensors: low latency, high temporal resolution, and high dynamic range. However, new algorithms are required to exploit the sensor characteristics and cope with its unconventional output, which consists of a stream of asynchronous brightness changes (called “events”) and synchronous grayscale frames. In this paper, we present a low-latency visual odometry algorithm for the DAVIS sensor using event-based feature tracks. Features are first detected in the grayscale frames and then tracked asynchronously using the stream of events. The features are then fed to an event-based visual odometry algorithm that tightly interleaves robust pose optimization and probabilistic mapping. We show that our method successfully tracks the 6-DOF motion of the sensor in natural scenes. This is the first work on event-based visual odometry with the DAVIS sensor using feature tracks. The measurement rate of cameras that take spatially multiplexed measurements by using spatial light modulators (SLM) is often limited by the switching speed of the SLMs. This is especially true for single-pixel cameras where the photodetector operates at a rate that is many orders-of-magnitude greater than the SLM. We study the factors that determine the measurement rate for such spatial multiplexing cameras (SMC) and show that increasing the number of pixels in the device improves the measurement rate, but there is an optimum number of pixels (typically, few thousands) beyond which the measurement rate does not increase. This motivates the design of LiSens, a novel imaging architecture, that replaces the photodetector in the single-pixel camera with a 1D linear array or a line-sensor. We illustrate the optical architecture underlying LiSens, build a prototype, and demonstrate results of a range of indoor and outdoor scenes. LiSens delivers on the promise of SMCs: imaging at a megapixel resolution, at video rate, using an inexpensive low-resolution sensor. FlatCam is a thin form-factor lensless camera that consists of a coded mask placed on top of a bare, conventional sensor array. Unlike a traditional, lens-based camera, where an image of the scene is directly recorded on the sensor pixels, each pixel in FlatCam records a linear combination of light from multiple scene elements. A computational algorithm is then used to demultiplex the recorded measurements and reconstruct an image of the scene. FlatCam is an instance of a coded aperture imaging system; however, unlike the vast majority of related work, we place the coded mask extremely close to the image sensor that enables thin and flat form-factor imaging devices. We employ a separable mask to ensure that both calibration and image reconstruction are scalable in terms of memory requirements and computational complexity. We demonstrate the potential of the FlatCam design using two prototypes: one at visible wavelengths and one at infrared wavelengths. Laser devices have received increasing attention in numerous computer-aided applications such as automatic control, 3D modeling and virtual reality. In this paper, aiming at people counting, we propose a novel people detection and tracking method based on the multipoint infrared laser, which can further facilitate intelligent scene modeling and analysis. In our method, a camera with the infrared lens filter is utilized to capture the monitored scene where an array of infrared spots is produced by the multipoint infrared laser. We build a spatial background model based on locations of spots. Pedestrians are detected by clustering of foreground spots. Then, our method tracks and counts the detected pedestrians via inferring the forward–backward motion consistency. Both quantitative and qualitative evaluation and comparison are conducted, and the experimental results demonstrate that the proposed method achieves excellent performance in challenging scenarios. We analyze the resolution of imaging functionals that migrate the cross-correlation matrices of passive sensor arrays. These matrices are obtained by cross-correlating signals emitted by ambient noise sources and recorded by the passive sensor array. They contain information about reflectors in the surrounding medium. Therefore, travel time or Kirchhoff migration of the cross-correlations can, under favorable circumstances, produce images of such reflectors. However, migration should be carried out appropriately depending on the type of illumination provided by the ambient noise sources. We present here a detailed resolution analysis of these functionals in a homogeneous medium. Resolution depends on the sensor array diameter, the distance from the array to the reflector and the central frequency, as is the case in active array imaging. When imaging with passive sensor arrays and ambient noise, resolution also depends on the space and time coherence of the noise sources because it determines an effective noise bandwidth. Very large format video or wide-area motion imagery (WAMI) acquired by an airborne camera sensor array is characterized by persistent observation over a large field-of-view with high spatial resolution but low frame rates (i.e. one to ten frames per second). Current WAMI sensors have sufficient coverage and resolution to track vehicles for many hours using just a single airborne platform. We have developed an interactive low frame rate tracking system based on a derived rich set of features for vehicle detection using appearance modeling combined with saliency estimation and motion prediction. Instead of applying subspace methods to very high-dimensional feature vectors, we tested the performance of feature fusion to locate the target of interest within the prediction window. Preliminary results show that fusing the feature likelihood maps improves detection but fusing feature maps combined with saliency information actually degrades performance. The goal of this work is to build video cameras whose spatial and temporal resolutions can be changed post-capture depending on the scene. Building such cameras is difficult due to two reasons. First, current video cameras allow the same spatial resolution and frame rate for the entire captured spatio-temporal volume. Second, both these parameters are fixed before the scene is captured. We propose different components of video camera design: a sampling scheme, processing of captured data and hardware that offer post-capture variable spatial and temporal resolutions, independently at each image location. Using the motion information in the captured data, the correct resolution for each location is decided automatically. Our techniques make it possible to capture fast moving objects without motion blur, while simultaneously preserving high-spatial resolution for static scene parts within the same video sequence. Our sampling scheme requires a fast per-pixel shutter on the sensor-array, which we have implemented using a co-located camera-projector system. This paper presents a fiber optic based tactile array sensor that can be employed in magnetic resonance environments. In contrast to conventional sensing approaches, such as resistive or capacitive-based sensing methods, which strongly rely on the generation and transmission of electronics signals, here electromagnetically isolated optical fibers were utilized to develop the tactile array sensor. The individual sensing elements of the proposed sensor detect normal forces; fusing the information from the individual elements allows the perception of the shape of probed objects. Applied forces deform a micro-flexure inside each sensor tactel, displacing a miniature mirror which, in turn, modulates the light intensity introduced by a transmitting fiber connected to a light source at its proximal end. For each tactel, the light intensity is read by a receiving fiber connected directly to a 2-D vision sensor. Computer software, such as MATLAB, is used to process the images received by the vision sensor. The calibration process was conducted by relating the applied forces to the number of activated pixels for each image received from a receiving fiber. The proposed approach allows the concurrent acquisition of data from multiple tactile sensor elements using a vision sensor such as a standard video camera. Test results of force responses and shape detection have proven the viability of this sensing concept. We present a novel application of the Kinect™, an input device designed for the Microsoft® Xbox 360® video game system. The device can be used by Earth scientists as a low-cost, high-resolution, short-range 3D 4D camera imaging system producing data similar to a terrestrial light detection and ranging (LiDAR) sensor. The Kinect contains a structured light emitter, an infrared camera (the combination of these two produce a distance image), a visual wavelength camera, a three-axis accelerometer, and four microphones. The cost is US $100, frame rate is 30 Hz, spatial and depth resolutions are mm to cm depending on range, and the optimal operating range is 0.5 to 5 m. The resolution of the distance measurements decreases with distance and is ≤1 mm at 0.5 m and 75 mm at 5 m. We illustrate data collection and basic data analysis routines in three experiments designed to demonstrate the breadth and utility of this new sensor in domains of glaciology, stream bathymetry, and geomorphology, although the device is applicable to a number of other Earth science fields. Copyright © 2012 John Wiley & Sons, Ltd. We present PalmType, which uses palms as interactive keyboards for smart wearable displays, such as Google Glass. PalmType leverages users' innate ability to pinpoint specific areas of their palms and fingers without visual attention (i.e. proprioception), and provides visual feedback via the wearable displays. With wrist-worn sensors and wearable displays, PalmType enables typing without requiring users to hold any devices and does not require visual attention to their hands. We conducted design sessions with 6 participants to see how users map QWERTY layout to their hands based on their proprioception. To evaluate typing performance and preference, we conducted a 12-person user study using Google Glass and Vicon motion tracking system, which showed that PalmType with optimized QWERTY layout is 39 faster than current touchpad-based keyboards. In addition, PalmType is preferred by 92 of the participants. We demonstrate the feasibility of wearable PalmType by building a prototype that uses a wrist-worn array of 15 infrared sensors to detect users' finger position and taps, and provides visual feedback via Google Glass. The increased power densities of multi-core processors and the variations within and across workloads lead to runtime thermal hot spots locations of which change across time and space. Thermal hot spots increase leakage, deteriorate timing, and reduce the mean time to failure. To manage runtime thermal variations, circuit designers embed within-die thermal sensors that acquire temperatures at few selected locations. The acquired temperatures are then used to guide runtime thermal management techniques. The capabilities of these techniques are essentially bounded by the spatial thermal resolution of the sensor measurements. In this paper we characterize temperature signals of real processors and demonstrate that on-chip thermal gradients lead to sparse signals in the frequency domain. We exploit this observation to (1) devise thermal sensor allocation techniques, and (2) devise signal reconstruction techniques that fully characterize the thermal status of the processor using the limited number of measurements from the thermal sensors. To verify the accuracy of our methods, we compare our temperature characterization results against thermal measurements acquired from a state-of-the-art infrared camera that captures the mid-band infrared emissions from the back of the die of a 45 nm dual-core processor. Our results show that our techniques are capable of accurately characterizing the temperatures of real processors. We present the architecture and three applications of the largest resolution image sensor based on single-photon avalanche diodes (SPADs) published to date. The sensor, fabricated in a high-voltage CMOS process, has a resolution of 512 × 128 pixels and a pitch of 24 ?m. The fill-factor of 5 can be increased to 30 with the use of microlenses. For precise control of the exposure and for time-resolved imaging, we use fast global gating signals to define exposure windows as small as 4 ns. The uniformity of the gate edges location is ?140 ps (FWHM) over the whole array, while in-pixel digital counting enables frame rates as high as 156 kfps. Currently, our camera is used as a highly sensitive sensor with high temporal resolution, for applications ranging from fluorescence lifetime measurements to fluorescence correlation spectroscopy and generation of true random numbers. We present the development of a polyimide-based two-dimensional tactile sensing array realized using a novel inverted fabrication technique. Thermal silicon oxide or Pyrex® substrates are treated such that their surfaces are OH group terminated, allowing good adhesion between such substrates and a spun-on polyimide film during processing through what are suspected to be hydrogen bonds that can be selectively broken when release is desired. The release of the continuous polyimide film is rapidly accomplished by breaking these bonds. This process results in robust, low-cost and continuous polymer-film devices. The developed sensor skin contains an array of membrane-based tactile sensors (taxels). Micromachined thin-film met al strain gauges are positioned on the edges of polyimide membranes. The change in resistance from each strain gauge resulting from normal forces applied to tactile bumps on the top of the membranes is used to image force distribution. Response of an individual taxel is characterized. The effective gauge factor of the taxels is found to be approximately 1.3. Sensor array output is experimentally obtained. The demonstrated devices are robust enough for direct contact with humans, everyday objects and contaminants without undue care. Accurate detection and localization of vehicles in aerial images has a wide range of applications including urban planning, military reconnaissance, visual surveillance, and realtime traffic management. Automated detection of vehicles in aerial imagery is a challenging task, due to the density of vehicles on the road, the complexity of the surrounding environment in urban areas, and low spatial resolution of the image sensor array. We propose an automated method for detecting vehicles of varying sizes in low-resolution aerial imagery. First, we develop a new vehicle enhancement filter involving multiscale Hessian analysis. After thresholding, we refine the candidate vehicle detections based on analysis of bilateral symmetry. We show that our proposed method provides improved detection accuracy compared with existing vehicle detection algorithms for various low-resolution aerial images. The problems of sensor configuration and activation for the detection of correlated random fields using large sensor arrays are considered. Using results that characterize the large-array performance of sensor networks in this application, the detection capabilities of different sensor configurations are analyzed and compared. The dependence of the optimal choice of configuration on parameters such as sensor signal-to-noise ratio (SNR), field correlation, etc., is examined, yielding insights into the most effective choices for sensor selection and activation in various operating regimes.