Hyperspectral cameras can provide unique spectral signatures for consistently distinguishing materials that can be used to solve surveillance tasks. In this paper, we propose a novel real-time hyperspectral likelihood maps-aided tracking method (HLT) inspired by an adaptive hyperspectral sensor. A moving object tracking system generally consists of registration, object detection, and tracking modules. We focus on the target detection part and remove the necessity to build any offline classifiers and tune a large amount of hyperparameters, instead learning a generative target model in an online manner for hyperspectral channels ranging from visible to infrared wavelengths. The key idea is that, our adaptive fusion method can combine likelihood maps from multiple bands of hyperspectral imagery into one single more distinctive representation increasing the margin between mean value of foreground and background pixels in the fused map. Experimental results show that the HLT not only outperforms all established fusion methods but is on par with the current state-of-the-art hyperspectral target tracking frameworks.
Our work is inspired by Yin, Porikli and Collins @cite_22 who proposed a single target tracker for aerial imagery and @cite_25 who used different set of visual features to extract feature likelihood maps and adaptively fuse them to obtain a single fusion map. While hyperspectral imagery has shown to have potential for vehicle tracking @cite_32 @cite_16 @cite_24 @cite_30 @cite_27 @cite_17 @cite_9 , the sensors are rare and expensive and hence most of the aerial tracking work has been done in infrared, single-band or RGB. In this section, we review the efforts done in the above sensor domains to solve vehicle detection and tracking and how hyperspectral imagery could possibly solve some of the problems.
@cite_30 @cite_22 @cite_9 @cite_32 @cite_24 @cite_27 @cite_16 @cite_25 @cite_17
2087143641 2232829127 2399469309 2484071357
Hyperspectral imagery has the capability of capturing spectral features of interest that can be used to differentiate among similar materials. While hyperspectral imaging has been demonstrated to provide data that enable classification of relatively broad categories, there remain open questions as to how fine of discrimination is possible. An application of this fine discrimination question is the potential that spectral features exist in the surface reflectance of ordinary civilian vehicles that would enable tracking of a particular vehicle across repeated hyperspectral images in a cluttered urban area. To begin to explore this question a vehicle tracking experiment was conducted in the summer of 2005 on the Rochester Institute of Technology (RIT) campus in Rochester, New York. Several volunteer vehicles were moved around campus at specific times coordinated with over flights of RIT's airborne Modular Imaging Spectrometer Instrument (MISI). MISI collected sequential images of the campus in 70 spectral channels from 0.4 to 1.0 microns with a ground resolution of approximately 2.5 meters. Ground truth spectra and photographs were collected for the vehicles. These data are being analyzed to determine the ability to uniquely associate a vehicle in one image with its location in a subsequent image. Initial results have demonstrated that the spectral measurement of a specific vehicle can be used to find the same vehicle in a subsequent image, although this is not always possible and is very dependent upon the specifics of the situation. Additionally, efforts are presented that explore predicted performance for variations in scene and sensor parameters through an analytical performance prediction model. Abstract Vehicle tracking in adverse environments is a challenging problem because of the high number of factors constraining their motion and possibility of frequent occlusion. In such conditions, identification rates drop dramatically. Hyperspectral imaging is known to improve the robustness of target identification by recording extended data in many wavelengths. However, it is impossible to transmit such a high rate data in real time with a conventional full hyperspectral sensor. Thus, we present a persistent ground-based target tracking system, taking advantage of a state-of-the-art, adaptive, multi-modal sensor controlled by Dynamic Data Driven Applications Systems (DDDAS) methodology. This overcomes the data challenge of hyperspectral tracking by only using spectral data as required. Spectral features are inserted in a feature matching algorithm to identify spectrally likely matches and simplify multidimensional assignment algorithm. The sensor is tasked for spectra acquisition by the prior estimates from the Gaussian Sum Filter and foreground mask generated by the background subtraction. Prior information matching the target features is used to tackle false negatives in the background subtraction output. The proposed feature-aided tracking system is evaluated in a challenging scene with a realistic vehicular simulation. Tracking vehicles through dense environments is an important and challenging task that is mostly tackled using visible and near IR wavelengths. Hyperspectral imaging is known to improve the robustness of target identification, but the massive increase in data created is usually prohibitive for tracking many targets. We present a persistent real-time aerial target tracking system, taking advantage of an adaptive, multimodal sensor concept and blending the hyperspectral likelihoods with kinematic likelihoods in a multidimensional assignment framework. The adaptive sensor is capable of providing wide field of view panchromatic images as well as the spectra of small number of pixels. The proposed system does not require large amount of hyperspectral data collection as we focus on tracking fewer number of targets with higher persistency. This overcomes the data challenge of hyperspectral tracking by following dynamic data-driven application systems (DDDAS) principles to control hyperspectral data collection where most beneficial. The DDDAS framework for controlling hyperspectral data collection is developed by incorporating prior information from the filter movement predictions and information from motion detection. The proposed multidimensional hyperspectral feature-aided tracker is compared to a 2-D hyperspectral feature-aided tracker and another cascaded hyperspectral data based tracker by generating a synthetic, realistic, aerial video on a dense scene. Vehicle tracking from a moving aerial platform poses a number of unique challenges including the small number of pixels representing a vehicle, large camera motion, and parallax error. This paper considers a multi-modal sensor to design a real-time persistent aerial tracking system. Wide field of view (FOV) panchromatic imagery is used to remove global camera motion whereas narrow FOV hyperspectral image is used to detect the target of interest (TOI). Hyperspectral features provide distinctive information to reject objects with different reflectance characteristics from the TOI. This way the density of detected vehicles is reduced, which increases tracking consistency. Finally, we use a spatial data based classifier to remove spurious detections. With such framework, parallax effect in non-planar scenes is avoided. The proposed tracking system is evaluated in a dense, synthetic scene and outperforms other state-of-theart traditional and aerial object trackers.