A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Factorial methods and related <s> Introduction. Matrix Algebra. Characterizing and Displaying Multivariate Data. The Multivariate Normal Distribution. Tests on One or Two Mean Vectors. Multivariate Analysis of Variance. Tests on Covariance Matrices. Discriminant Analysis: Description of Group Separation. Classification Analysis: Allocation of Observations to Groups. Multivariate Regression. Canonical Correlation. Principal Component Analysis. Factor Analysis. Cluster Analysis. Graphical Procedures. Tables. Answers and Hints to Problems. Data Sets and SAS Files. References. Index. <s> BIB001 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Factorial methods and related <s> High-dimensional data can be converted to low-dimensional codes by training a multilayer neural network with a small central layer to reconstruct high-dimensional input vectors. Gradient descent can be used for fine-tuning the weights in such “autoencoder” networks, but this works well only if the initial weights are close to a good solution. We describe an effective way of initializing the weights that allows deep autoencoder networks to learn low-dimensional codes that work much better than principal components analysis as a tool to reduce the dimensionality of data. <s> BIB002 </s> A survey on pre-processing techniques: Relevant issues in the context of environmental data mining <s> Factorial methods and related <s> Introduction Regression Modeling Classification and Discrimination Dimension Reduction Clustering Linear Regression Models Relationship between Two Variables Relationships Involving Multiple Variables Regularization Nonlinear Regression Models Modeling Phenomena Modeling by Basis Functions Basis Expansions Regularization Logistic Regression Models Risk Prediction Models Multiple Risk Factor Models Nonlinear Logistic Regression Models Model Evaluation and Selection Criteria Based on Prediction Errors Information Criteria Bayesian Model Evaluation Criterion Discriminant Analysis Fisher's Linear Discriminant Analysis Classification Based on Mahalanobis Distance Variable Selection Canonical Discriminant Analysis Bayesian Classification Bayes' Theorem Classification with Gaussian Distributions Logistic Regression for Classification Support Vector Machines Separating Hyperplane Linearly Nonseparable Case From Linear to Nonlinear Principal Component Analysis Principal Components Image Compression and Decompression Singular Value Decomposition Kernel Principal Component Analysis Clustering Hierarchical Clustering Nonhierarchical Clustering Mixture Models for Clustering Appendix A: Bootstrap Methods Appendix B: Lagrange Multipliers Appendix C: EM Algorithm Bibliography Index <s> BIB003
Reducing the original set of variables of a dataset to a smaller set of equivalent variables is an interest topic of Multivariate Statistics from the beginning of XXth century  . The main goal of factorial methods is to substitute the original list of variables by a subset of factors that keep the same information as the original dataset. The approach is radically different from the one followed in feature selection. Here, an algebraic metaphor (classical in statistics), in which each row of the data matrix is associated with a K-dimensional point in a vector space (K being the number of variables), is used to find subspaces of smaller dimension where the projection of the original data cloud conserves the adjacency relationships among both variables and individuals. These methods are mainly based in matrix rotations and diagonalizations and several techniques are available according to the kind of information contained in the dataset. The Principal Components Analysis (PCA)  is one of the best known techniques of this group and it is suitable for datasets with only numerical variables. Each principal component is a linear combination of the original variables, and the aim is to work with a reduced set of these, such that the loss of information is not relevant. Thus, PCA is suitable for synthesizing an original set of numerical variables into a small number of fictitious variables conserving as much information as possible from the original dataset. Equivalent techniques are available for qualitative data, like simple or multiple correspondence analysis  . A general formulation including all factorial techniques as particular cases is the canonical analysis, in which other more general methods are embraced, like non-linear multivariate analysis BIB003 and methods based on oblique rotations BIB001 . In the context of finding a relevant set of descriptors from images, non-linear generalizations of PCA based on artificial neural networks and deep learning methods are used BIB002 . In these methods, the information retained in each of the factors is quantified and the number of significant factors to be retained can be evaluated. However, it has to be taken into account that, in most cases, interpretation of the new variables (or factors) may not be clear. If the factors are later used to build data mining models with them, implications over understandability of the final results might appear. An appropriate alternative for the case when the interpretation of the significant factors is difficult is to identify the subset of original variables having higher contribution to that factors, and use this subset of original variables in the data mining. This is an intermediate alternative producing a subset of original variables bigger than the set of significant factors, but keeping the data mining in the original space, thus maintaining the interpretability of the obtained models.