We explore the use of convolutional neural networks for the semantic classification of remote sensing scenes. Two recently proposed architectures, CaffeNet and GoogLeNet, are adopted, with three different learning modalities. Besides conventional training from scratch, we resort to pre-trained networks that are only fine-tuned on the target data, so as to avoid overfitting problems and reduce design time. Experiments on two remote sensing datasets, with markedly different characteristics, testify on the effectiveness and wide applicability of the proposed solution, which guarantees a significant performance improvement over all state-of-the-art references.
Most of the features proposed in the current literature are extracted only from the gray-scale image. However, Yang and Newsam @cite_48 showed that color histogram descriptors, evaluated on hue, lightness and saturation (Color-HLS), may provide a very good performance. The interactions among RGB color bands is exploited also in mCENTRIST @cite_58 , an extension of the CENTRIST algorithm @cite_53 , based in turn on LBP histograms and principal component analysis. LBP features are used also in @cite_17 , where the maximal conditional mutual information (MCMI) scheme is proposed to select an optimal subset of these features. In @cite_12 , instead, feature selection is performed separately for each class, with both one-versus-all and one-versus-one strategies.
@cite_48 @cite_53 @cite_58 @cite_12 @cite_17
2113855951 2071309849 2064125725 2132633146
CENsus TRansform hISTogram (CENTRIST), a new visual descriptor for recognizing topological places or scene categories, is introduced in this paper. We show that place and scene recognition, especially for indoor environments, require its visual descriptor to possess properties that are different from other vision domains (e.g., object recognition). CENTRIST satisfies these properties and suits the place and scene recognition task. It is a holistic representation and has strong generalizability for category recognition. CENTRIST mainly encodes the structural properties within an image and suppresses detailed textural information. Our experiments demonstrate that CENTRIST outperforms the current state of the art in several place and scene recognition data sets, compared with other descriptors such as SIFT and Gist. Besides, it is easy to implement and evaluates extremely fast. mCENTRIST, a new multichannel feature generation mechanism for recognizing scene categories, is proposed in this paper. mCENTRIST explicitly captures the image properties that are encoded jointly by two image channels, which is different from popular multichannel descriptors. In order to avoid the curse of dimensionality, tradeoffs at both feature and channel levels have been executed to make mCENTRIST computationally practical. As a result, mCENTRIST is both efficient and easy to implement. In addition, a hyperopponent color space is proposed by embedding Sobel information into the opponent color space for further performance improvements. Experiments show that mCENTRIST outperforms established multichannel descriptors on four RGB and RGB-near infrared data sets, including aerial orthoimagery, indoor, and outdoor scene category recognition tasks. Experiments also verify that the hyper opponent color space enhances descriptors' performance effectively. Generally, some object-based features are more relevant to a thematic class than other features. These strongly relevant features, termed as class-specific features, would significantly contribute to thematic information extraction for very high resolution (VHR) images. However, many existing feature selection methods have been designed to select a good feature subset for all classes, rather than an independent feature subset for the thematic class. The latter might better meet the requirement of thematic information extraction than the former. In addition, the lack of quantitative evaluation of the contribution of the selected features to thematic classes also weakens our understandability of these features. To address the problems, class-specific feature selection methods are developed to measure the effectiveness of features for extracting thematic information from VHR images. First, the one-versus-all scheme is combined with traditional feature selection methods, such as ReliefF and LeastC. Also, one-versus-one scheme is utilized for alleviating the negative impact of a class imbalance problem arising from the one-versus-all scheme. Then, the relative contributions of features to thematic classes are obtained by the class-specific feature selection methods to describe the effectiveness of features for thematic information extraction. Finally, the class-specific feature selection methods are compared with the original methods on three different VHR image data sets by the nearest neighbor and support vector machine. Experimental results show that the class-specific feature selection methods outperform the corresponding conventional methods, and the one-versus-one scheme surpasses one-versus-all scheme. Additionally, many features are evaluated by the class-specific feature selection methods, to provide end users advice on effectiveness of the features. Local binary patterns of more bits extracted in a large structure have shown promising results in visual recognition applications. This results in very high-dimensional data so that it is not feasible to directly extract features from the LBP histogram, especially for a large-scale database. Instead of extracting features from the LBP histogram, we propose a new approach to learn discriminative LBP structures for a specific application. Our objective is to select an optimal subset of binarized-pixel-difference features to compose the LBP structure. As these features are strongly correlated, conventional feature-selection methods may not yield a desirable performance. Thus, we propose an incremental Maximal-Conditional-Mutual-Information scheme for LBP structure learning. The proposed approach has demonstrated a superior performance over the state-of-the-arts results on classifying both spatial patterns such as texture classification, scene recognition and face recognition, and spatial-temporal patterns such as dynamic texture recognition. HighlightsWe propose a new approach to tackle high-dimensional LBP features.It discovers optimal LBP structure to generate discriminative features.We propose a MCMI scheme for LBP structure learning to handle pixel correlation.It demonstrates a superior performance to SOTA on various visual applications.