A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> Sequential search methods characterized by a dynamically changing number of features included or eliminated at each step, henceforth "floating" methods, are presented. They are shown to give very good results and to be computationally more effective than the branch and bound method. <s> BIB001 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> A comparison of multiple linear regression (MLR) with partial least-squares (PLS) regression is presented, for the multivariate modeling of hydroxyl number in a certain polymer of a heterogeneous near-IR spectroscopic data set. The MLR model was performed by selecting the variables with a genetic algorithm. A good model could be obtained with both methods. It was shown that the MLR and PLS solutions are very similar. The two models include the same number of variables, and the first variables in each model have similar, chemically understandable functions. It is concluded that both solutions are equivalent and that each has some advantages and disadvantages. This also means that even in very complex situations such as here, MLR can replace PLS in certain cases. <s> BIB002 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> From the Publisher: ::: With advanced computer technologies and their omnipresent usage, data accumulates in a speed unmatchable by the human's capacity to process data. To meet this growing challenge, the research community of knowledge discovery from databases emerged. The key issue studied by this community is, in layman's terms, to make advantageous use of large stores of data. In order to make raw data useful, it is necessary to represent, process, and extract knowledge for various applications. Feature Selection for Knowledge Discovery and Data Mining offers an overview of the methods developed since the 1970's and provides a general framework in order to examine these methods and categorize them. This book employs simple examples to show the essence of representative feature selection methods and compares them using data sets with combinations of intrinsic properties according to the objective of feature selection. In addition, the book suggests guidelines for how to use different methods under various circumstances and points out new challenges in this exciting area of research. Feature Selection for Knowledge Discovery and Data Mining is intended to be used by researchers in machine learning, data mining, knowledge discovery, and databases as a toolbox of relevant tools that help in solving large real-world problems. This book is also intended to serve as a reference book or secondary text for courses on machine learning, data mining, and databases. <s> BIB003 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> Abstract A new suboptimal search strategy for feature selection is presented. It represents a more sophisticated version of “classical” floating search algorithms ( Pudil et al., 1994 ), attempts to remove some of their potential deficiencies and facilitates finding a solution even closer to the optimal one. <s> BIB004 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> Feature selection plays an important role in classifying systems such as neural networks (NNs). We use a set of attributes which are relevant, irrelevant or redundant and from the viewpoint of managing a dataset which can be huge, reducing the number of attributes by selecting only the relevant ones is desirable. In doing so, higher performances with lower computational effort is expected. In this paper, we propose two feature selection algorithms. The limitation of mutual information feature selector (MIFS) is analyzed and a method to overcome this limitation is studied. One of the proposed algorithms makes more considered use of mutual information between input attributes and output classes than the MIFS. What is demonstrated is that the proposed method can provide the performance of the ideal greedy selection algorithm when information is distributed uniformly. The computational load for this algorithm is nearly the same as that of MIFS. In addition, another feature selection algorithm using the Taguchi method is proposed. This is advanced as a solution to the question as to how to identify good features with as few experiments as possible. The proposed algorithms are applied to several classification problems and compared with MIFS. These two algorithms can be combined to complement each other's limitations. The combined algorithm performed well in several experiments and should prove to be a useful method in selecting features for classification problems. <s> BIB005 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> This paper addresses a common methodological flaw in the comparison of variable selection methods. A practical approach to guide the search or the selection process is to compute cross-validation performance estimates of the different variable subsets. Used with computationally intensive search algorithms, these estimates may overfit and yield biased predictions. Therefore, they cannot be used reliably to compare two selection methods, as is shown by the empirical results of this paper. Instead, like in other instances of the model selection problem, independent test sets should be used for determining the final performance. The claims made in the literature about the superiority of more exhaustive search algorithms over simpler ones are also revisited, and some of them infirmed. <s> BIB006 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> This paper presents a comparison of feature selection methods for a unified detection of breast cancers in mammograms. A set of features, including curvilinear features, texture features, Gabor features, and multi-resolution features, were extracted from a region of 512×512 pixels containing normal tissue or breast cancer. Adaptive floating search and genetic algorithm were used for the feature selection, and a linear discriminant analysis (LDA) was used for the classification of cancer regions from normal regions. The performance is evaluated using A z , the area under ROC curve. On a dataset consisting 296 normal regions and 164 cancer regions (53 masses, 56 spiculated lesions, and 55 calcifications), adaptive floating search achieved A z =0.96 with comparison to A z =0.93 of CHC genetic algorithm and A z =0.90 of simple genetic algorithm. <s> BIB007 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> This paper introduces concepts and algorithms of feature selection, surveys existing feature selection algorithms for classification and clustering, groups and compares different algorithms with a categorizing framework based on search strategies, evaluation criteria, and data mining tasks, reveals unattempted combinations, and provides guidelines in selecting feature selection algorithms. With the categorizing framework, we continue our efforts toward-building an integrated system for intelligent feature selection. A unifying platform is proposed as an intermediate step. An illustrative example is presented to show how existing feature selection algorithms can be integrated into a meta algorithm that can take advantage of individual algorithms. An added advantage of doing so is to help a user employ a suitable algorithm without knowing details of each algorithm. Some real-world applications are included to demonstrate the use of feature selection in data mining. We conclude this work by identifying trends and challenges of feature selection research and development. <s> BIB008 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> A new improved forward floating selection (IFFS) algorithm for selecting a subset of features is presented. Our proposed algorithm improves the state-of-the-art sequential forward floating selection algorithm. The improvement is to add an additional search step called ''replacing the weak feature'' to check whether removing any feature in the currently selected feature subset and adding a new one at each sequential step can improve the current feature subset. Our method provides the optimal or quasi-optimal (close to optimal) solutions for many selected subsets and requires significantly less computational load than optimal feature selection algorithms. Our experimental results for four different databases demonstrate that our algorithm consistently selects better subsets than other suboptimal feature selection algorithms do, especially when the original number of features of the database is large. <s> BIB009 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> Choosing a small subset of genes that enables a good classification of diseases on the basis of microarray data is a difficult optimization problem. This paper presents a memetic algorithm, called MAGS, to deal with gene selection for supervised classification of microarray data. MAGS is based on an embedded approach for attribute selection where a classifier tightly interacts with the selection process. The strength of MAGS relies on the synergy created by combining a problem specific crossover operator and a dedicated local search procedure, both being guided by relevant information from a SVM classifier. Computational experiments on 8 well-known microarray datasets show that our memetic algorithm is very competitive compared with some recently published studies. <s> BIB010 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> Data and knowledge management systems employ feature selection algorithms for removing irrelevant, redundant, and noisy information from the data. There are two well-known approaches to feature selection, feature ranking (FR) and feature subset selection (FSS). In this paper, we propose a new FR algorithm, termed as class-dependent density-based feature elimination (CDFE), for binary data sets. Our theoretical analysis shows that CDFE computes the weights, used for feature ranking, more efficiently as compared to the mutual information measure. Effectively, rankings obtained from both the two criteria approximate each other. CDFE uses a filtrapper approach to select a final subset. For data sets having hundreds of thousands of features, feature selection with FR algorithms is simple and computationally efficient but redundant information may not be removed. On the other hand, FSS algorithms analyze the data for redundancies but may become computationally impractical on high-dimensional data sets. We address these problems by combining FR and FSS methods in the form of a two-stage feature selection algorithm. When introduced as a preprocessing step to the FSS algorithms, CDFE not only presents them with a feature subset, good in terms of classification, but also relieves them from heavy computations. Two FSS algorithms are employed in the second stage to test the two-stage feature selection idea. We carry out experiments with two different classifiers (naive Bayes' and kernel ridge regression) on three different real-life data sets (NOVA, HIVA, and GINA) of the”Agnostic Learning versus Prior Knowledge” challenge. As a stand-alone method, CDFE shows up to about 92 percent reduction in the feature set size. When combined with the FSS algorithms in two-stages, CDFE significantly improves their classification accuracy and exhibits up to 97 percent reduction in the feature set size. We also compared CDFE against the winning entries of the challenge and found that it outperforms the best results on NOVA and HIVA while obtaining a third position in case of GINA. <s> BIB011 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> Advances in computer technologies have enabled corporations to accumulate data at an unprecedented speed. Large-scale business data might contain billions of observations and thousands of features, which easily brings their scale to the level of terabytes. Most traditional feature selection algorithms are designed and implemented for a centralized computing architecture. Their usability significantly deteriorates when data size exceeds tens of gigabytes. High-performance distributed computing frameworks and protocols, such as the Message Passing Interface (MPI) and MapReduce, have been proposed to facilitate software development on grid infrastructures, enabling analysts to process large-scale problems efficiently. This paper presents a novel large-scale feature selection algorithm that is based on variance analysis. The algorithm selects features by evaluating their abilities to explain data variance. It supports both supervised and unsupervised feature selection and can be readily implemented in most distributed computing environments. The algorithm was implemented as a SAS High-Performance Analytics procedure, which can read data in distributed form and perform parallel feature selection in both symmetric multiprocessing mode (SMP) and massively parallel processing mode (MPP). Experimental results demonstrated the superior performance of the proposed method for large scale feature selection. <s> BIB012 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> K. Gibert et al. / A survey on pre-processing techniques: <s> Biofilm development in drinking water distribution systems (DWDSs) is a real problem negatively affecting service and water quality, and, thus, the satisfaction of the final consumers. It is the direct and indirect responsible for many of the DWDSs’ problems, and a lot of resources are invested to mitigate its effects. Addressing this problem has been a concern of researchers and DWDS managers for years. However, it is only recently that both technology and data have been available to support the new approach presented in this work. Our proposal is based on the combination of various existing data sets from similar studies to conduct a meta-data analysis of biofilm development. The approach lies on an intensive data pre-processing. Having a complete and extensive database on biofilm development in DWDSs allows applying Machine Learning techniques to develop a practical model. It is based on a multidisciplinary research vision to formulate effective biofilm control strategies. This work presents the basis for the development of a useful decision-making tool to assist in DWDS management. The negative effects on service and consumers caused by biofilm would be mitigated maintaining it at the lowest level. The performance of the suggested models is tested with data coming from two different case-studies: the DWDSs of the city of Thessaloniki (Greece) and the Pennine Water Group experimental facility (UK). The results obtained validate this methodology as an excellent approach to studying biofilm development in DWDSs. <s> BIB013
Relevant issues in the context of environmental data mining * Forward methods: starting with an empty set of features, they add one feature at each step until getting the optimal subset of features BIB001 BIB006 . * Backward methods: starting from a set containing all the features, they discard one feature at each step until getting the optimal set of features BIB009 BIB004  . * Random methods: they can try several subsets of features in a random way, trying to avoid to be trapped in local optima (i.e. anytime algorithms)  BIB002 BIB003  BIB007  . • An intermediate type of methods are the embedded methods: they perform feature selection in the process of training of the ML technique and are usually specific, and included within the given ML techniques BIB010 , such as in decision trees or classification problems BIB005 and artificial neural networks techniques  . Some works give common frames for different modeling methods BIB008 . Most recent works tackle the problem of computational cost in big data scenarios BIB011 BIB012 . As an example, in BIB013 up to 15 variables associated with biofilm development in water distribution systems were available, including sampling and incubation methods and physical, hydraulic and physico-chemical characteristics of water. However, a random wrapper suggested an adequate model using only eight of those variables.