A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> The intensive use of memory to recall specific episodes from the past—rather than rules—should be the foundation of machine reasoning. <s> BIB001 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> Indexing of cases is an important topic for MemoryBased Reasoning(MBR). One key problem is how assign weights to attributes of cases. Although several weighting methods have been proposed, some methods cannot handle numeric attributes directly, so it is necessary to discretize numeric values by classification. Furthermore, existing methods have no theoretical background, so little can be said about optimality. We propose a new weighting method based on a statistical technique called Quantification Method II. It can handle both numeric and symbolic attributes in the same framework. Generated attribute weights are optimal in the sense that they maximize the ratio of variance between classes to variance of all cases. Experiments on several benchmark tests show that in many cases, our method obtains higher accuracies than some other weighting methods. The results also indicate that it can distinguish relevant attributes from irrelevant ones, and can tolerate noisy data. <s> BIB002 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> In the context of machine learning from examples this paper deals with the problem of estimating the quality of attributes with and without dependencies among them. Kira and Rendell (1992a,b) developed an algorithm called RELIEF, which was shown to be very efficient in estimating attributes. Original RELIEF can deal with discrete and continuous attributes and is limited to only two-class problems. In this paper RELIEF is analysed and extended to deal with noisy, incomplete, and multi-class data sets. The extensions are verified on various artificial and one well known real-world problem. <s> BIB003 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> Previous work on feature weighting for case-based learning algorithms has tended to use either global weights or weights that vary over extremely local regions of the case space. This paper examines the use of coarsely local weighting schemes, where feature weights are allowed to vary but are identical for groups or clusters of cases. We present a new technique, called class distribution weighting (CDW), that allows weights to vary at the class level. We further extend CDW into a family of related techniques that exhibit varying degrees of locality, from global to local. The class distribution techniques are then applied to a set of eleven concept learning tasks. We find that one or more of the CDW variants significantly improves classification accuracy for nine of the eleven tasks. In addition, we find that the relative importance of classes, features, and feature values in a particular domain determines which variant is most successful. <s> BIB004 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> In this article we propose a case-base maintenance methodology based on the idea of transferring knowledge between knowledge containers in a case-based reasoning (CBR) system. A machine-learning technique, fuzzy decision-tree induction, is used to transform the case knowledge to adaptation knowledge. By learning the more sophisticated fuzzy adaptation knowledge, many of the redundant cases can be removed. This approach is particularly useful when the case base consists of a large number of redundant cases and the retrieval efficiency becomes a real concern of the user. The method of maintaining a case base from scratch, as proposed in this article, consists of four steps. First, an approach to learning feature weights automatically is used to evaluate the importance of different features in a given case base. Second, clustering of cases is carried out to identify different concepts in the case base using the acquired feature-weights knowledge. Third, adaptation rules are mined for each concept using fuzzy decision trees. Fourth, a selection strategy based on the concepts of case coverage and reachability is used to select representative cases. In order to demonstrate the effectiveness of this approach as well as to examine the relationship between compactness and performance of a CBR system, experimental testing is carried out using the Traveling and the Rice Taste data sets. The results show that the testing case bases can be reduced by 36 and 39 percent, respectively, if we complement the remaining cases by the adaptation rules discovered using our approach. The overall accuracies of the two smaller case bases are 94 and 90 percent, respectively, of the originals. <s> BIB005 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> Many lazy learning algorithms are derivatives of the k-nearest neighbor (k-NN) classifier, which uses a distance function to generate predictions from stored instances. Several studies have shown that k-NN‘s performance is highly sensitive to the definition of its distance function. Many k-NN variants have been proposed to reduce this sensitivity by parameterizing the distance function with feature weights. However, these variants have not been categorized nor empirically compared. This paper reviews a class of weight-setting methods for lazy learning algorithms. We introduce a framework for distinguishing these methods and empirically compare them. We observed four trends from our experiments and conducted further studies to highlight them. Our results suggest that methods which use performance feedback to assign weight settings demonstrated three advantages over other methods: they require less pre-processing, perform better in the presence of interacting features, and generally require less training data to learn good settings. We also found that continuous weighting methods tend to outperform feature selection algorithms for tasks where some features are useful but less important than others. <s> BIB006 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> In this paper, we identify two issues involved in developing an automated feature subset selection algorithm for unlabeled data: the need for finding the number of clusters in conjunction with feature selection, and the need for normalizing the bias of feature selection criteria with respect to dimension. We explore the feature selection problem and these issues through FSSEM (Feature Subset Selection using Expectation-Maximization (EM) clustering) and through two different performance criteria for evaluating candidate feature subsets: scatter separability and maximum likelihood. We present proofs on the dimensionality biases of these feature criteria, and present a cross-projection normalization scheme that can be applied to any criterion to ameliorate these biases. Our experiments show the need for feature selection, the need for addressing these two issues, and the effectiveness of our proposed solutions. <s> BIB007 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: Relevant issues in the context of environmental data mining 645 <s> The major hypothesis that we will be prove in this paper is that unsupervised learning techniques of feature weighting are not significantly worse than supervised methods, as is commonly believed in the machine learning community. ::: ::: This paper tests the power of unsupervised feature weighting techniques for predictive tasks within several domains. The paper analyses several unsupervised and supervised feature weighting techniques, and proposes new unsupervised feature weighting techniques. Two unsupervised entropy-based weighting algorithms are proposed and tested against all other techniques. The techniques are evaluated in terms of predictive accuracy on unseen instances, measured by a ten-fold cross-validation process. The testing has been done using thirty-four data sets from the UCI Machine Learning Database Repository and other sources. Unsupervised weighting methods assign weights to attributes without any knowledge about class labels, so this task is considerably more difficult. It has commonly been assumed that unsupervised methods would have a substantially worse performance than supervised ones, as they do not use any domain knowledge to bias the process. The major result of the study is that unsupervised methods really are not so bad. Moreover, one of the new unsupervised learning method proposals has shown a promising behaviour when faced against domains with many irrelevant features, reaching similar performance as some of the supervised methods. <s> BIB008
In recent years, many researchers are focusing on supervised feature weighting, i.e. where a response variable is available and relevance towards it is evaluated: • Some research in filter global methods has been done like the Mutual Information technique (MI)  BIB006 , the QM2 method by BIB002 , and  about their introduced Cross-Category Feature importance (CCF) method. On the other hand, filter local weighting methods have been proposed in the literature such as the value difference metric (VDM) of BIB001 , the Class Distribution Weighting method (CDW) BIB004 , and the Per-Category Feature importance criterion (PCF)  . • Some wrapper methods are: the RELIEF-F method  BIB003 and the DIET algorithm  , and some approaches based on the use of Genetic Algorithms   . However, less work has been done in the field on unsupervised feature weighting, based on the sensitivity of instance similarities to that variable. Unsupervised scenario is clearly the required one when facing a new unknown database, which we want to mine to discover new knowledge, and usually no reference classification is available. In the literature, one of the few works is BIB005 on a Gradient Descent technique (GD) and feature selection approach on unsupervised entropy-based method  . In BIB007 a feature weighting method for supervised learning is presented. Derived from it, in BIB008 two new unsupervised feature weighting methods were proposed (UEB-1 and UEB-2) with promising results. Empirical works BIB006 and theoretical ones  , suggest that the learning complexity is exponential regarding the number of irrelevant variables. Therefore, the failures in the data mining process could be related to a similarity model, and in particular, with an incorrect weight assignment methodology. A comprehensive review of feature weighting can be found in  . Indeed, the use of pure feature weighting is quite marginal in real applications. It is more frequent to use them as a previous step of feature selection by choosing a relevance threshold to keep a variable for the analysis.