Abstract Designing discriminative powerful texture features robust to realistic imaging conditions is a challenging computer vision problem with many applications, including material recognition and analysis of satellite or aerial imagery. In the past, most texture description approaches were based on dense orderless statistical distribution of local features. However, most recent approaches to texture recognition and remote sensing scene classification are based on Convolutional Neural Networks (CNNs). The de facto practice when learning these CNN models is to use RGB patches as input with training performed on large amounts of labeled data (ImageNet). In this paper, we show that Local Binary Patterns (LBP) encoded CNN models, codenamed TEX-Nets, trained using mapped coded images with explicit LBP based texture information provide complementary information to the standard RGB deep models. Additionally, two deep architectures, namely early and late fusion, are investigated to combine the texture and color information. To the best of our knowledge, we are the first to investigate Binary Patterns encoded CNNs and different deep network fusion architectures for texture recognition and remote sensing scene classification. We perform comprehensive experiments on four texture recognition datasets and four remote sensing scene classification benchmarks: UC-Merced with 21 scene categories, WHU-RS19 with 19 scene classes, RSSCN7 with 7 categories and the recently introduced large scale aerial image dataset (AID) with 30 aerial scene types. We demonstrate that TEX-Nets provide complementary information to standard RGB deep model of the same network architecture. Our late fusion TEX-Net architecture always improves the overall performance compared to the standard RGB network on both recognition problems. Furthermore, our final combination leads to consistent improvement over the state-of-the-art for remote sensing scene classification.
A variety of texture recognition approaches have been proposed in literature @cite_44 @cite_29 . The work of @cite_84 proposes a statistical approach to model textures based on the joint probability distribution of filter responses. The work of @cite_75 proposes an approach based on Weber's law which consists of two components: differential excitation and orientation. An image is represented by the concatenation of these two components in a single representation. The work of @cite_2 introduces an approach that uses lookup-table based vector quantization for texture description. A set of low and mid-level perceptually inspired visual features are introduced by @cite_69 for texture recognition. A multi-resolution framework based on LBP is proposed by @cite_67 for rotation invariant texture recognition. As discussed earlier, LBP is one of the most successful approaches for texture recognition with several variants existing in literature @cite_18 @cite_62 @cite_26 .
@cite_69 @cite_67 @cite_18 @cite_62 @cite_26 @cite_29 @cite_84 @cite_44 @cite_2 @cite_75
1985636418 1975959625 61677869 2028735463 2103496373 1516748092 2154209944 2132633146 1482080565 2049310893
Local or global rotation invariant feature extraction has been widely used in texture classification. Local invariant features, e.g. local binary pattern (LBP), have the drawback of losing global spatial information, while global features preserve little local texture information. This paper proposes an alternative hybrid scheme, globally rotation invariant matching with locally variant LBP texture features. Using LBP distribution, we first estimate the principal orientations of the texture image and then use them to align LBP histograms. The aligned histograms are then in turn used to measure the dissimilarity between images. A new texture descriptor, LBP variance (LBPV), is proposed to characterize the local contrast information into the one-dimensional LBP histogram. LBPV does not need any quantization and it is totally training-free. To further speed up the proposed matching scheme, we propose a method to reduce feature dimensions using distance measurement. The experimental results on representative databases show that the proposed LBPV operator and global matching scheme can achieve significant improvement, sometimes more than 10 in terms of classification accuracy, over traditional locally rotation invariant LBP method. This paper presents a novel approach for texture classification, generalizing the well-known local binary pattern (LBP) approach. In the proposed approach, two different and complementary types of features (pixel intensities and differences) are extracted from local patches. The intensity-based features consider the intensity of the central pixel (CI) and those of its neighbors (NI); while for the difference-based feature, two components are computed: the radial-difference (RD) and the angular-difference (AD). Inspired by the LBP approach, two intensity-based descriptors CI-LBP and NI-LBP, and two difference-based descriptors RD-LBP and AD-LBP are developed. All four descriptors are in the same form as conventional LBP codes, so they can be readily combined to form joint histograms to represent textured images. The proposed approach is computationally very simple: it is totally training-free, there is no need to learn a texton dictionary, and no tuning of parameters. We have conducted extensive experiments on three challenging texture databases (Outex, CUReT and KTHTIPS2b). Outex results show significant improvements over the classical LBP approach, which clearly demonstrates the great power of the joint distributions of these proposed descriptors for gray-scale and rotation invariant texture classification. The proposed method produces the best classification results on KTHTIPS2b, and results comparable to the state-of-the-art on CUReT. This work presents a novel image appearance description method based on the highly popular local binary pattern (LBP) texture features. The key idea consists of introducing a dense sampling encoding strategy for extracting more stable and discriminative texture patterns in local regions. Compared to the conventional "sparse" sampling scheme commonly used in basic LBP, our proposed dense sampling aims to generate, through a form of up-sampling, more neighboring pixels so that more stable LBP codes, carrying out richer information, are computed. This yields in significantly enhanced image description which is less prone to noise and to sparse and unstable histograms. Another interesting property of the dense sampling scheme is that it can be easily integrated with many existing LBP variants. Extensive experiments on three different classification problems namely face recognition, texture classification and age group estimation on various challenging benchmark databases clearly demonstrate the efficiency of the proposed scheme, showing very promising results compared not only to original LBP but also to state-of-the-art especially in the very demanding task of human age estimation. In this paper, we propose a simple, efficient, yet robust multiresolution approach to texture classification—binary rotation invariant and noise tolerant (BRINT). The proposed approach is very fast to build, very compact while remaining robust to illumination variations, rotation changes, and noise. We develop a novel and simple strategy to compute a local binary descriptor based on the conventional local binary pattern (LBP) approach, preserving the advantageous characteristics of uniform LBP. Points are sampled in a circular neighborhood, but keeping the number of bins in a single-scale LBP histogram constant and small, such that arbitrarily large circular neighborhoods can be sampled and compactly encoded over a number of scales. There is no necessity to learn a texton dictionary, as in methods based on clustering, and no tuning of parameters is required to deal with different data sets. Extensive experimental results on representative texture databases show that the proposed BRINT not only demonstrates superior performance to a number of recent state-of-the-art LBP variants under normal conditions, but also performs significantly and consistently better in presence of noise due to its high distinctiveness and robustness. This noise robustness characteristic of the proposed BRINT is evaluated quantitatively with different artificially generated types and levels of noise (including Gaussian, salt and pepper, and speckle noise) in natural texture images. This paper proposes a novel approach to extract image features for texture classification. The proposed features are robust to image rotation, less sensitive to histogram equalization and noise. It comprises of two sets of features: dominant local binary patterns (DLBP) in a texture image and the supplementary features extracted by using the circularly symmetric Gabor filter responses. The dominant local binary pattern method makes use of the most frequently occurred patterns to capture descriptive textural information, while the Gabor-based features aim at supplying additional global textural information to the DLBP features. Through experiments, the proposed approach has been intensively evaluated by applying a large number of classification tests to histogram-equalized, randomly rotated and noise corrupted images in Outex, Brodatz, Meastex, and CUReT texture image databases. Our method has also been compared with six published texture features in the experiments. It is experimentally demonstrated that the proposed method achieves the highest classification accuracy in various texture databases and image conditions. The recent emergence of Local Binary Patterns (LBP) has led to significant progress in applying texture methods to various computer vision problems and applications. The focus of this research has broadened from 2D textures to 3D textures and spatiotemporal (dynamic) textures. Also, where texture was once utilized for applications such as remote sensing, industrial inspection and biomedical image analysis, the introduction of LBP-based approaches have provided outstanding results in problems relating to face and activity analysis, with future scope for face and facial expression recognition, biometrics, visual surveillance and video analysis. Computer Vision Using Local Binary Patterns provides a detailed description of the LBP methods and their variants both in spatial and spatiotemporal domains. This comprehensive reference also provides an excellent overview as to how texture methods can be utilized for solving different kinds of computer vision and image analysis problems. Source codes of the basic LBP algorithms, demonstrations, some databases and a comprehensive LBP bibliography can be found from an accompanying web site. Topics include: local binary patterns and their variants in spatial and spatiotemporal domains, texture classification and segmentation, description of interest regions, applications in image retrieval and 3D recognition - Recognition and segmentation of dynamic textures, background subtraction, recognition of actions, face analysis using still images and image sequences, visual speech recognition and LBP in various applications. Written by pioneers of LBP, this book is an essential resource for researchers, professional engineers and graduate students in computer vision, image analysis and pattern recognition. The book will also be of interest to all those who work with specific applications of machine vision. Inspired by theories of sparse representation and compressed sensing, this paper presents a simple, novel, yet very powerful approach for texture classification based on random projection, suitable for large texture database applications. At the feature extraction stage, a small set of random features is extracted from local image patches. The random features are embedded into a bag--of-words model to perform texture classification; thus, learning and classification are carried out in a compressed domain. The proposed unconventional random feature extraction is simple, yet by leveraging the sparse nature of texture images, our approach outperforms traditional feature extraction methods which involve careful design and complex steps. We have conducted extensive experiments on each of the CUReT, the Brodatz, and the MSRC databases, comparing the proposed approach to four state-of-the-art texture classification methods: Patch, Patch-MRF, MR8, and LBP. We show that our approach leads to significant improvements in classification accuracy and reductions in feature dimensionality. Local binary patterns of more bits extracted in a large structure have shown promising results in visual recognition applications. This results in very high-dimensional data so that it is not feasible to directly extract features from the LBP histogram, especially for a large-scale database. Instead of extracting features from the LBP histogram, we propose a new approach to learn discriminative LBP structures for a specific application. Our objective is to select an optimal subset of binarized-pixel-difference features to compose the LBP structure. As these features are strongly correlated, conventional feature-selection methods may not yield a desirable performance. Thus, we propose an incremental Maximal-Conditional-Mutual-Information scheme for LBP structure learning. The proposed approach has demonstrated a superior performance over the state-of-the-arts results on classifying both spatial patterns such as texture classification, scene recognition and face recognition, and spatial-temporal patterns such as dynamic texture recognition. HighlightsWe propose a new approach to tackle high-dimensional LBP features.It discovers optimal LBP structure to generate discriminative features.We propose a MCMI scheme for LBP structure learning to handle pixel correlation.It demonstrates a superior performance to SOTA on various visual applications. Research in texture recognition often concentrates on the problem of material recognition in uncluttered conditions, an assumption rarely met by applications. In this work we conduct a first study of material and describable texture at- tributes recognition in clutter, using a new dataset derived from the OpenSurface texture repository. Motivated by the challenge posed by this problem, we propose a new texture descriptor, D-CNN, obtained by Fisher Vector pooling of a Convolutional Neural Network (CNN) filter bank. D-CNN substantially improves the state-of-the-art in texture, mate- rial and scene recognition. Our approach achieves 82.3 accuracy on Flickr material dataset and 81.1 accuracy on MIT indoor scenes, providing absolute gains of more than 10 over existing approaches. D-CNN easily trans- fers across domains without requiring feature adaptation as for methods that build on the fully-connected layers of CNNs. Furthermore, D-CNN can seamlessly incorporate multi-scale information and describe regions of arbitrary shapes and sizes. Our approach is particularly suited at lo- calizing stuff categories and obtains state-of-the-art re- sults on MSRC segmentation dataset, as well as promising results on recognizing materials and surface attributes in clutter on the OpenSurfaces dataset. Local binary pattern from three orthogonal planes (LBP-TOP) has been widely used in emotion recognition in the wild. However, it suffers from illumination and pose changes. This paper mainly focuses on the robustness of LBP-TOP to unconstrained environment. Recent proposed method, spatiotemporal local monogenic binary pattern (STLMBP), was verified to work promisingly in different illumination conditions. Thus this paper proposes an improved spatiotemporal feature descriptor based on STLMBP. The improved descriptor uses not only magnitude and orientation, but also the phase information, which provide complementary information. In detail, the magnitude, orientation and phase images are obtained by using an effective monogenic filter, and multiple feature vectors are finally fused by multiple kernel learning. STLMBP and the proposed method are evaluated in the Acted Facial Expression in the Wild as part of the 2014 Emotion Recognition in the Wild Challenge. They achieve competitive results, with an accuracy gain of 6.35 and 7.65 above the challenge baseline (LBP-TOP) over video.