A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> In the field of pattern recognition, multiple classifier systems based on the combination of outputs of a set of different classifiers have been proposed as a method for the development of high performance classification systems. Previous work clearly showed that multiple classifier. Systems are effective only if the classifiers forming them make independent errors. Therefore, the fundamental need for methods aimed to design "error-independent" classifiers is currently acknowledged. In the paper, an approach to the automatic design of multiple classifier systems is proposed. Given an initial large set of classifiers, our approach is aimed at selecting the subset formed by the most error-independent classifiers. Reported results on the classification of multisensor remote-sensing images show that this approach allows to design effective multiple classifier systems. <s> BIB001 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> Abstract Various experimental comparisons of algorithms for supervised classification of remote-sensing images have been reported in the literature. Among others, a comparison of neural and statistical classifiers has previously been made by the authors in (Serpico, S.B., Bruzzone, L., Roli, F., 1996. Pattern Recognition Letters 17, 1331–1341). Results of reported experiments have clearly shown that the superiority of one algorithm over another cannot be claimed. In addition, they have pointed out that statistical and neural algorithms often require expensive design phases to attain high classification accuracy. In this paper, the combination of neural and statistical algorithms is proposed as a method to obtain high accuracy values after much shorter design phases and to improve the accuracy–rejection tradeoff over those allowed by single algorithms. <s> BIB002 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> Abstract In the field of pattern recognition, the combination of an ensemble of neural networks has been proposed as an approach to the development of high performance image classification systems. However, previous work clearly showed that such image classification systems are effective only if the neural networks forming them make different errors. Therefore, the fundamental need for methods aimed to design ensembles of ‘error-independent’ networks is currently acknowledged. In this paper, an approach to the automatic design of effective neural network ensembles is proposed. Given an initial large set of neural networks, our approach is aimed to select the subset formed by the most error-independent nets. Reported results on the classification of multisensor remote-sensing images show that this approach allows one to design effective neural network ensembles. <s> BIB003 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> Random Forests are considered for classification of multisource remote sensing and geographic data. Various ensemble classification methods have been proposed in recent years. These methods have been proven to improve classification accuracy considerably. The most widely used ensemble methods are boosting and bagging. Boosting is based on sample re-weighting but bagging uses bootstrapping. The Random Forest classifier uses bagging, or bootstrap aggregating, to form an ensemble of classification and regression tree (CART)-like classifiers. In addition, it searches only a random subset of the variables for a split at each CART node, in order to minimize the correlation between the classifiers in the ensemble. This method is not sensitive to noise or overtraining, as the resampling is not based on weighting. Furthermore, it is computationally much lighter than methods based on boosting and somewhat lighter than simple bagging. In the paper, the use of the Random Forest classifier for land cover classification is explored. We compare the accuracy of the Random Forest classifier to other better-known ensemble methods on multisource remote sensing and geographic data. <s> BIB004 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> Abstract Detailed land use/land cover classification at ecotope level is important for environmental evaluation. In this study, we investigate the possibility of using airborne hyperspectral imagery for the classification of ecotopes. In particular, we assess two tree-based ensemble classification algorithms: Adaboost and Random Forest, based on standard classification accuracy, training time and classification stability. Our results show that Adaboost and Random Forest attain almost the same overall accuracy (close to 70%) with less than 1% difference, and both outperform a neural network classifier (63.7%). Random Forest, however, is faster in training and more stable. Both ensemble classifiers are considered effective in dealing with hyperspectral data. Furthermore, two feature selection methods, the out-of-bag strategy and a wrapper approach feature subset selection using the best-first search method are applied. A majority of bands chosen by both methods concentrate between 1.4 and 1.8 μm at the early shortwave infrared region. Our band subset analyses also include the 22 optimal bands between 0.4 and 2.5 μm suggested in Thenkabail et al. [Thenkabail, P.S., Enclona, E.A., Ashton, M.S., and Van Der Meer, B. (2004). Accuracy assessments of hyperspectral waveband performance for vegetation analysis applications. Remote Sensing of Environment, 91, 354–376.] due to similarity of the target classes. All of the three band subsets considered in this study work well with both classifiers as in most cases the overall accuracy dropped only by less than 1%. A subset of 53 bands is created by combining all feature subsets and comparing to using the entire set the overall accuracy is the same with Adaboost, and with Random Forest, a 0.2% improvement. The strategy to use a basket of band selection methods works better. Ecotopes belonging to the tree classes are in general classified better than the grass classes. Small adaptations of the classification scheme are recommended to improve the applicability of remote sensing method for detailed ecotope mapping. <s> BIB005 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> Ensemble learning techniques are increasingly applied for species and vegetation distribution modelling, often resulting in more accurate predictions. At the same time, uncertainty assessment of distribution models is gaining attention. In this study, Random Forests, an ensemble learning technique, is selected for vegetation distribution modelling based on environmental variables. The impact of two important sources of uncertainty, that is the uncertainty on spatial interpolation of environmental variables and the uncertainty on species clustering into vegetation types, is quantified based on sequential Gaussian simulation and pseudo-randomization tests, respectively. An empirical assessment of the uncertainty propagation to the distribution modelling results indicated a gradual decrease in performance with increasing input uncertainty. The test set error ranged from 30.83% to 52.63% and from 30.83% to 83.62%, when the uncertainty ranges on spatial interpolation and on vegetation clustering, respectively, were fully covered. Shannon’s entropy, which is proposed as a measure for uncertainty of ensemble predictions, revealed a similar increasing trend in prediction uncertainty. The implications of these results in an empirical distribution modelling framework are further discussed with respect to monitoring setup, spatial interpolation and species clustering. <s> BIB006 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> SAR data are almost independent from weather conditions, and thus are well suited for mapping of seasonally changing variables such as land cover. In regard to recent and upcoming missions, multitemporal and multi-frequency approaches become even more attractive. In the present study, classifier ensembles (i.e., boosted decision tree and random forests) are applied to multi-temporal C-band SAR data, from different study sites and years. A detailed accuracy assessment shows that classifier ensembles, in particularly random forests, outperform standard approaches like a single decision tree and a conventional maximum likelihood classifier by more than 10% independently from the site and year. They reach up to almost 84% of overall accuracy in rural areas with large plots. Visual interpretation confirms the statistical accuracy assessment and reveals that also typical random noise is considerably reduced. In addition the results demonstrate that random forests are less sensitive to the number of training samples and perform well even with only a small number. Random forests are computationally highly efficient and are hence considered very well suited for land cover classifications of future multifrequency and multitemporal stacks of SAR imagery. <s> BIB007 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> The accuracy attained in the mapping of underwater areas is limited by the effect of variations in the water column, which degrade the signal received by the orbital sensor, creating interclasses confusion that introduce errors into the final result of the classification process. In this article we will describe a hybrid classifier ensembles; the classification is done by progressive refining in three stages. At the end of this process, a combining unit links the various partial classifications generated and achieve the accuracy level desired. At the end, the result obtained by the ensemble is compared to the results achieved by the application of multi-class voting scheme methods based on support vector machine: One-Against-the-Rest and One-Against-One. Classification accuracy showed the viability and the potential of using the proposed ensemble to classify images. <s> BIB008 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> In this article, we present a semisupervised support vector machine that uses self-training approach. We then construct an ensemble of semisupervised SVM classifiers to address the problem of pixel classification of remote sensing images. Semisupervised support vector machines (S^3VMs) are based on applying the margin maximization principle to both labeled and unlabeled samples. The ensemble of SVM classifiers recognizes the conceptual similarity between component classifiers from the same data source. The effectiveness of the proposed technique is first demonstrated for two numeric remote sensing data described in terms of feature vectors and then identifying different land cover regions in remote sensing imagery. Experimental results on these datasets show that employing this learning scheme can increase the accuracy level. The performance of the ensemble is compared with one of its component classifier and conventional SVM in terms of accuracy and quantitative cluster validity indices. <s> BIB009 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> Land cover monitoring using remotely sensed data requires robust classification methods which allow for the accurate mapping of complex land cover and land use categories. Random forest (RF) is a powerful machine learning classifier that is relatively unknown in land remote sensing and has not been evaluated thoroughly by the remote sensing community compared to more conventional pattern recognition techniques. Key advantages of RF include: their non-parametric nature; high classification accuracy; and capability to determine variable importance. However, the split rules for classification are unknown, therefore RF can be considered to be black box type classifier. RF provides an algorithm for estimating missing values; and flexibility to perform several types of data analysis, including regression, classification, survival analysis, and unsupervised learning. ::: ::: In this paper, the performance of the RF classifier for land cover classification of a complex area is explored. Evaluation was based on several criteria: mapping accuracy, sensitivity to data set size and noise. Landsat-5 Thematic Mapper data captured in European spring and summer were used with auxiliary variables derived from a digital terrain model to classify 14 different land categories in the south of Spain. Results show that the RF algorithm yields accurate land cover classifications, with 92% overall accuracy and a Kappa index of 0.92. RF is robust to training data reduction and noise because significant differences in kappa values were only observed for data reduction and noise addition values greater than 50 and 20%, respectively. Additionally, variables that RF identified as most important for classifying land cover coincided with expectations. A McNemar test indicates an overall better performance of the random forest model over a single decision tree at the 0.00001 significance level. <s> BIB010 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> The amounts and types of remote sensing data have increased rapidly, and the classification of these datasets has become more and more overwhelming for a single classifier in practical applications. In this paper, an ensemble algorithm based on Diversity Ensemble Creation by Oppositional Relabeling of Artificial Training Examples (DECORATEs) and Rotation Forest is proposed to solve the classification problem of remote sensing image. In this ensemble algorithm, the RBF neural networks are employed as base classifiers. Furthermore, interpolation technology for identical distribution is used to remold the input datasets. These remolded datasets will construct new classifiers besides the initial classifiers constructed by the Rotation Forest algorithm. The change of classification error is used to decide whether to add another new classifier. Therefore, the diversity among these classifiers will be enhanced and the accuracy of classification will be improved. Adaptability of the proposed algorithm is verified in experiments implemented on standard datasets and actual remote sensing dataset. <s> BIB011 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> An IDL implementation for the classification and regression analysis of remote sensing images with Random Forests is introduced. The tool, called imageRF, is platform and license independent and uses generic image file formats. It works well with default parameterization, yet all relevant parameters can be defined in intuitive GUIs. This makes it a user-friendly image processing tool, which is implemented as an add-on in the free EnMAP-Box and may be used in the commercial IDL/ENVI software. <s> BIB012 </s> A survey of multiple classifier systems as hybrid systems <s> Land cover mapping <s> Abstract Polarimetric SAR data contains a large amount of potential information that may be used to characterize forested scenes. However, the large number of PolSAR parameters and discriminators cannot all be used in most classification problems. Some form of feature selection will improve classification results and improve the efficiency of the system. In addition, classification of PolSAR data may be improved with an ensemble of classifiers, each tuned to a different class. Our research is in the Petawawa experimental forest, in the boreal forest northwest of Ottawa, Ontario, Canada. We employ Radarsat-2 fine-quad image data acquired in August (leaf-on) and November (leaf-off) of 2009. We present two system designs in this paper. The first system consists of a feature selector based on a non-parametric evaluation function and a support vector machine for classification. We demonstrate that the feature selection step improves classification accuracy significantly over a baseline classifier. We then present a system consisting of an ensemble of SVM classifiers, each with its own feature selection component and trained on an individual class. The classifier likelihoods are combined in a final step. We demonstrate that this system improves classification accuracy significantly over a single-classifier system. Finally, we demonstrate that classification accuracies are significantly higher when leaf-on and leaf-off images are combined over a single season image. <s> BIB013
Early application of MCS to land cover mapping consisted in overproducing a large set of classifiers and searching for the optimal subset BIB001 BIB003 BIB002 . To avoid the combinatorial complexity, the approach performs clustering of classifier error, aggregating similar classifiers. The approach was proven to be optimal under some conditions on the classifiers. Interestingly, testing was performed on multi-source data, composing the pixel's feature vector of joining multi-spectral with radar data channels, to compute the land cover map. The MCS was heterogenous, composed of MLP, RBF, and PNN. The application of RF to processing remote sensing data has been abundant in the literature. It has been applied to estimate land cover on Landsat data over Granada, Spain BIB010 and multisource data in a Colorado mountainous area BIB004 . Specifically, Landsat Multi-Spectral, elevation, slope and aspect data are used as input features. The RF approach is able to successfully fuse these inhomogeneous informations. Works on hyperspectral images acquired by the HyMap sensor have been addressed to build vegetation thematic maps BIB005 , comparing RF and decision tree-based Adaboost, as well as two feature selection methods: the out-ofbag and a best-first search wrapper feature subset selection method. Diverse feature subsets are tested, and the general conclusion is that tree ecotopes are better discriminated than grass ecotopes. Further work with RF has been done assessing the uncertainty in modeling the distribution of vegetation types BIB006 , performing classification on the basis of environmental variables, in an approach that combines spatial distribution modeling by spatial interpolation, using sequential Gaussian simulation and the clustering of species into vegetation types. Dealing with labeled data scarcity, there are methods BIB011 based on the combination of RF and the enrichment of the training dataset with artificially generated samples in order to increase classifier diversity, which is applied to Landsat multispectral data. Artificial data is generated from the Gaussian modeling of the data distribution. The application of RF to SAR multitemporal data aims to achieve season invariant detection of several classes of land cover, i.e. grassland, ceral, forest, etc. BIB007 . RF performed best, with lowest spatial variability. Images were coregistered and some model portability was tested, where the model trained on one SAR image was applied on other SAR images of the same site obtained at different times. The success of RF for remote sensing images has prompted the proposal of an specific computational environment BIB012 . Ensambles of SVM have been also applied to land cover map. Indeed, the ground truth data scarcity has been attacked by an active learning approach to semi-supervised SVM training BIB009 . The active learning approach is based on the clustering of the unlabeled data samples according to the clustering of the SVM outputs on the current training dataset. Samples with higher membership coefficient are added to the corresponding class data, and the classifier is retrained in an iterative process. These semi-supervised SVM are combined in a majority voting ensemble and applied to the classification SPOT and Landsat optical data. Land cover classification in the specific context of shallow waters has the additional difficulties of the scattering, refraction and reflection effects introduced by the water cover. A robust process combines a parallel and a serial architecture BIB008 , where initial classification results obtained by SVM are refined in a second SVM classifier and the final result is given by a linear combination of two ensembles of SVM classifiers and a minimum distance classifier. Besides, the system estimates the water depth by a bathymetry estimation process. The approach is applied to Landsat images for the estimation of coral population in coastal waters. Polarimetric SAR data used for the classification of Boreal forests require an ensemble of SVM BIB013 . Each of the SVM is specifically tuned to a class, with specific feature selection process. Best results are obtained when multitemporal data is used, joining two images from two different seasons (summer and winter) and performing the feature selection and training on the joint data vectors.