A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> Two algorithms for producing multiple imputations for missing data are evaluated with simulated data. Software using a propensity score classifier with the approximate Bayesian bootstrap produces badly biased estimates of regression coefficients when data on predictor variables are missing at random or missing completely at random. On the other hand, a regression-based method employing the data augmentation algorithm produces estimates with little or no bias. <s> BIB001 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> The study is concerned with the fundamentals of granular computing. Granular computing, as the name itself stipulates, deals with representing information in the form of some aggregates (that embrace a number of individual entities) and their ensuing processing. We elaborate on the rationale behind granular computing. Next, a number of formal frameworks of information granulation are discussed including several alternatives such as fuzzy sets, interval analysis, rough sets, and probability. The notion of granularity itself is defined and quantified. A design agenda of granular computing is formulated and the key design problems are raised. A number of granular architectures are also discussed with an objective of delineating the fundamental algorithmic, and conceptual challenges. It is shown that the use of information granules of different size (granularity) lends itself to general pyramid architectures of information processing. The role of encoding and decoding mechanisms visible in this setting is also discussed in detail, along with some particular solutions. We raise an issue of interoperability of granular environments. The intent of the paper is to elaborate on the fundamentals and put the entire area in a certain perspective while not moving into specific algorithmic details. <s> BIB002 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> In KDD procedure, to fill in missing data typically requires a very large investment of time and energy – often 80% to 90% of a data analysis project is spent in making the data reliable enough so that the results can be trustful. In this paper, we propose a SVM regression based algorithm for filling in missing data, i.e. set the decision attribute (output attribute) as the condition attribute (input attribute) and the condition attribute as the decision attribute, then use SVM regression to predict the condition attribute values. SARS data set experimental results show that SVM regression method has the highest precision. The method with which the value of the example that has the minimum distance to the example with missing value will be taken to fill in the missing values takes the second place, and the mean and median methods have lower precision. <s> BIB003 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> The Generative Topographic Mapping (GTM) was originally conceived as a probabilistic alternative to the well-known, neural network-inspired, Self-Organizing Maps. The GTM can also be interpreted as a constrained mixture of distribution models. In recent years, much attention has been directed towards Student t-distributions as an alternative to Gaussians in mixture models due to their robustness towards outliers. In this paper, the GTM is redefined as a constrained mixture of t-distributions: the t-GTM, and the Expectation-Maximization algorithm that is used to fit the model to the data is modified to carry out missing data imputation. Several experiments show that the t-GTM successfully detects outliers, while minimizing their impact on the estimation of the model parameters. It is also shown that the t-GTM provides an overall more accurate imputation of missing values than the standard Gaussian GTM. <s> BIB004 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> Abstract Individual-based models, which permit a fine description of the dynamics of a population are costly in computation time and need many simulations for outlining global laws. Conversely aggregate models take less time for simulation and provide analytical understanding of the global evolution. Therefore they can sometimes profitably replace individual-based models. However, in many cases these models are linked to a model of environment which evolves over time. In this paper we show that the relevance of using an aggregate vs. an individual based model of water consumption highly depends on the information available on the resource. We use two versions of a consumption model (individual-based and aggregate), simulating a reversible diffusion process which depends on the information available on the resource. We compare the results of the two models for various kinds links between the resource and consumption models and corresponding to the information about the resource provided to the consumers. We propose and test different properties of this information which might lead to difference of the results. <s> BIB005 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> This review presents a practical summary of the missing data literature, including a sketch of missing data theory and descriptions of normalmodel multiple imputation (MI) and maximum likelihood methods. Practical missing data analysis issues are discussed, most notably the inclusion of auxiliary variables for improving power and reducing bias. Solutions are given for missing data challenges such as handling longitudinal, categorical, and clustered data with normal-model MI; including interactions in the missing data model; and handling large numbers of variables. The discussion of attrition and nonignorable missingness emphasizes the need for longitudinal diagnostics and for reducing the uncertainty about the missing data mechanism under attrition. Strategies suggested for reducing attrition bias include using auxiliary variables, collecting follow-up data on a sample of those initially missing, and collecting data on intent to drop out. Suggestions are given for moving forward with research on missing data and attrition. <s> BIB006 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> This work presents advances in the design of a hybrid methodology that combines artificial intelligence and statistical tools to induce a model of explicit knowledge in relation to the dynamics of a wastewater treatment plant. The methodology contributes to problem solving under the paradigm of knowledge discovery from data in which the pre-process, the automatic interpretation of results and the explicit production of knowledge play a role as important as the analysis itself. The data mining step is performed using clustering based on rules by states, which integrates the knowledge discovered separately at each step of the process into a single model of global operation of the phenomenon. This provides a more accurate model for the dynamics of the system than one obtained by analyzing the whole dataset with all the steps taken together. <s> BIB007 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> Using five medical datasets we detected the influence of missing values on true positive rates and classification accuracy. We randomly marked more and more values as missing and tested their effects on classification accuracy. The classifications were performed with nearest neighbour searching when none, 10, 20, 30% or more values were missing. We also used discriminant analysis and naive Bayesian method for the classification. We discovered that for a two-class dataset, despite as high as 20---30% missing values, almost as good results as with no missing value could still be produced. If there are more than two classes, over 10---20% missing values are probably too many, at least for small classes with relatively few cases. The more classes and the more classes of different sizes, a classification task is the more sensitive to missing values. On the other hand, when values are missing on the basis of actual distributions affected by some selection or non-random cause and not fully random, classification can tolerate even high numbers of missing values for some datasets. <s> BIB008 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> Imputation of missing values is one of the major tasks for data pre-processing in many areas. Whenever imputation of data from official statistics comes into mind, several (additional) challenges almost always arise, like large data sets, data sets consisting of a mixture of different variable types, or data outliers. The aim is to propose an automatic algorithm called IRMI for iterative model-based imputation using robust methods, encountering for the mentioned challenges, and to provide a software tool in R. This algorithm is compared to the algorithm IVEWARE, which is the ''recommended software'' for imputations in international and national statistical institutions. Using artificial data and real data sets from official statistics and other fields, the advantages of IRMI over IVEWARE-especially with respect to robustness-are demonstrated. <s> BIB009 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> Multivariate imputation by chained equations (MICE) has emerged as a principled method of dealing with missing data. Despite properties that make MICE particularly useful for large imputation procedures and advances in software development that now make it accessible to many researchers, many psychiatric researchers have not been trained in these methods and few practical resources exist to guide researchers in the implementation of this technique. This paper provides an introduction to the MICE method with a focus on practical aspects and challenges in using this method. A brief review of software programs available to implement MICE and then analyze multiply imputed data is also provided. <s> BIB010 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> The division of a water supply network (WSN) into isolated supply clusters aims at improving the management of the whole system. This paper deals with the application of spectral clustering to achieve this aim. A semi-supervised approach can take into account the graph structure of a network and incorporate the corresponding hydraulic constraints and the other available vector information from the WSN. Several of the disadvantages of these methodologies stem from the largeness of the most WSN and the associated computational complexity. To solve these problems, we propose subsampling graph data to run successive weak clusters and build a single robust cluster configuration. The resulting methodology has been tested in a real network and can be used to successfully partition large WSNs. <s> BIB011 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> The field of missing data analysis has evolved quickly; outdated strategies have been replaced by missing data procedures that make use of partial data to provide unbiased and efficient parameter estimates. Here, we describe an eight-step process for performing basic missing data procedures (single and multiple imputation) using NORM (Version 2.03; Schafer, 1997) and SAS (Version 9.2; SAS, 2011a). We make use of the automation utility, MIAutomate, to facilitate the analysis of NORM-imputed data with SPSS. We also perform a basic multiple regression to compare the performance of the following five missing data approaches: (1) complete cases analysis, (2) mean substitution, (3) FIML-based Amos, (4) multiple imputation with no auxiliary variables (MI), and (5) multiple imputation with auxiliary variables (MI+). Mean substitution showed clear bias in parameter estimates. Although the complete cases model yielded parameter estimates similar to those obtained with MI and FIML, the standard errors were consistently larger because the estimation was based on fewer cases. The MI and FIML models yielded reasonably unbiased estimates of the b-weights; however, they had noticeably lower t-values compared to the MI+ analysis. Based on prior research and the empirical results shown in this chapter, including auxiliary variables to the missing data model that are highly correlated with variables containing missingness is a good way to reduce the standard errors of estimated parameters. ::: ::: ::: ::: The prescribed missing data procedures will always perform at least as well as more traditional approaches (e.g., complete cases analysis), and in most circumstances, there will be a clear advantage of the prescribed procedures, in terms of estimation bias and/or statistical power. Useful and accessible missing data procedures are now an integral part of many mainstream statistical packages, making it increasingly difficult to justify not using them. ::: ::: ::: Keywords: ::: ::: multiple imputation; ::: SAS Proc MI; ::: Amos; ::: FIML; ::: regression analysis <s> BIB012 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Treatment <s> The impact of missing data on quantitative research can be serious, leading to biased estimates of parameters, loss of information, decreased statistical power, increased standard errors, and weakened generalizability of findings. In this paper, we discussed and demonstrated three principled missing data methods: multiple imputation, full information maximum likelihood, and expectation-maximization algorithm, applied to a real-world data set. Results were contrasted with those obtained from the complete data set and from the listwise deletion method. The relative merits of each method are noted, along with common features they share. The paper concludes with an emphasis on the importance of statistical assumptions, and recommendations for researchers. Quality of research will be enhanced if (a) researchers explicitly acknowledge missing data problems and the conditions under which they occurred, (b) principled methods are employed to handle missing data, and (c) the appropriate treatment of missing data is incorporated into review standards of manuscripts submitted for publication. <s> BIB013
Nowadays, most of the data mining softwares claim that they can deal with missing values. Apparently, this seems to mean that one can forget about pre-processing them. However, one needs to know well which kind of missing data treatment is going to be automatically performed by the software in each different data mining method and properly evaluate whether this treatment fits well the data and goals of the analysis or not. In most cases, the pre-processing software tool is using the listwise deletion strategy or a complete case analysis strategy, both meaning that rows of data matrix containing missing data are excluded from the analysis. Removing rows with missing cells from a dataset may cause serious problems if the missing values are not randomly distributed BIB008 . As this decision is transparently taken by the software, when the missing data are non-random, this means, in practice, that the analysis is performed without a certain subpopulation. For example, in the database with animals leaving on a certain ecosystem, all registers describing a mammal will be ignored in the analysis, just because they have a missing in the form of the beak variable. The first consequence is that the resulting description of life in the ecosystem obtained is implicitly ignoring all things happening with mammals. As this decision is transparently taken by the software, the risks to assume that results describe well the complete ecosystem is very high, with potential dramatic consequences. So, using the default missing data treatment provided by the preprocessing tool software directly impacts on the scope of the analysis results, and restricts the conclusions to the smaller part of the population really analyzed by the software. In some cases, particularly when a big number of variables are considered, this produces software errors: it may happen that none of the data rows contains values in none of the remaining variables, or that some of them become constant, and then the software will intend to analyze an empty dataset or one with null variance. Also, this might produce loss of significant information (imagine for example, excluding from the analysis data row that lacks one out of 100 variables). Also, some systems assume substitution by the variable global mean. This permits to take advantage of the whole data rows, and it has the property of keeping the variable means invariant after imputation. However, this reduces the variance of the variable, having consequences in any inference operation and distorting the relationship with other variables, with effects in coefficients like covariances, and biases in inducted models. Thus, leaving the missing data treatment to the software requires a good knowledge of what is the software doing, and evaluating if it is correct for the particular case analyzed. Otherwise, this kind of automatic treatment can seriously damage the analysis in a transparent way. A better option is imputation. Imputation  is a complex process for converting missing data into useful data using estimation techniques. For qualitative variables, sometimes it is useful to use a specific modality to encode missings, like Un-kownValue. This is treated as an ordinary modality by the software pre-processing tool and the variable can be entered in the models. Impact of the qualitative missing in modeling could be then better evaluated, and decisions at the data mining step made accordingly. Structural missing data: the experts might provide rules to manually impute them, i.e. substitute form of the beak of all mammals by lackOfBeak or 0 legs for fishes. For temporal data, interpolation might be a good option. When data with different granularities are represented in a data matrix, the slower (or higher level) variables (measured less frequently) will produce lots of false missing data. They can be interpolated between useful data in case of temporal data, or properly propagated to the lower abstraction levels in case of spatial data, by taking advantage of some implicit assumptions, like constant behavior inside the interval (whether spatial or temporal), this being the reason for saving measurements at certain intervals (or granularity levels). As an example BIB007 , consider the levels of BOD, measured in a WWTP, once a week, while pH in the bioreactor is measured daily. Since BOD evolves very slowly and require a laboratory test, they are measured less frequently. If the dataset contains one daily register, this situation is producing 6 weekly missings for BOD when, in fact, it can be assumed that it behaves constantly from week to week. Monotonous interpolation or constant imputation will be used according to the knowledge on the variables provided by the expert. For example, in water resources management, different levels of aggregation regarding a water consumption model cannot be disconnected between them and from the global information available on the resources BIB005 BIB011 . This coexistence of different granularities in both time and space, in geospatial phenomena, characteristic in ESs, is a reason why learning global models for environmental phenomena is so difficult BIB002 . In a most precise approach, model-based imputation builds a predictive model for each variable to be imputed. Many different predictive models has been used to this purpose, like regression BIB009 , local least square imputation  , Support Vector Machines BIB003 . In  a decision tree has been used to impute the type of cooking habits of a household, and an artificial neural net to impute the PM10 emissions of their traditional stoves. However, model-based imputation is criticized by many authors for being time consuming BIB012 . A more sophisticated proposal consists in performing the missing imputation by preserving the relationships among variables in the entire dataset. This is the case of Maximum Likelihood Estimation (MLE method) and related methods  BIB004 , which performs well, with good theoretical properties, provided that a sound statistical model is available for the target problem, which is not always the case, especially in ESs. Also, building a MLE from scratch at every single application is often difficult and too time-consuming. The EM algorithm is also well founded, and involves bootstrapping to find standard errors for imputation, but requires high expertise to be properly used. Multiple imputation BIB001 BIB010  ] uses a distributional model for every variable and estimates several complete imputed data matrices by repeatedly making random trials from the reference distributional model. Then, all the imputed data matrices are mined and a consensus model is built, combining the results obtained for each of the matrices. It requires realistic statistical assumptions for the variables, only specialized software performs it, and it is no trivial to get the consensus final model. In BIB013  surveys of missing imputation data can be found, some of them with a very practical approach BIB006 . The most important aspect is to avoid false assumptions when considering imputation methods, which may have a significant effect on the results. All the methods have pros and cons, and the choice must be made with care. The most sophisticated a method is, the more precise are the imputations provided, but the more expertise is required to apply properly, the most time consuming is. Considering that missing imputation is a specific part of the pre-processing step, which in turn is only the first step of the KDD process, excessive time consuming methods are not the best option in real applications, where often answers are required in a very limited time. In the following, some intermediate solutions from the context of data mining that try to provide acceptable imputation in short time are mentioned.