Generating Synthetic Missing Data: A Review by Missing Mechanism <s> A. UNIVARIATE MCAR IMPLEMENTATIONS <s> Increasing the awareness of how incomplete data affects learning and classification accuracy has led to increasing numbers of missing data techniques. This article investigates the robustness and accuracy of seven popular techniques for tolerating incomplete training and test data for different patterns of missing data—different proportions and mechanisms of missing data on resulting tree-based models. The seven missing data techniques were compared by artificially simulating different proportions, patterns, and mechanisms of missing data using 21 complete datasets (i.e., with no missing values) obtained from the University of California, Irvine repository of machine-learning databases (Blake and Merz, 1998). A four-way repeated measures design was employed to analyze the data. The simulation results suggest important differences. All methods have their strengths and weaknesses. However, listwise deletion is substantially inferior to the other six techniques, while multiple imputation, that utilizes the expectation maximization algorithm, represents a superior approach to handling incomplete data. Decision tree single imputation and surrogate variables splitting are more severely impacted by missing values distributed among all attributes compared to when they are only on a single attribute. Otherwise, the imputation—versus model-based imputation procedures gave—reasonably good results although some discrepancies remained. Different techniques for addressing missing values when using decision trees can give substantially diverse results, and must be carefully considered to protect against biases and spurious findings. Multiple imputation should always be used, especially if the data contain many missing values. If few values are missing, any of the missing data techniques might be considered. The choice of technique should be guided by the proportion, pattern, and mechanisms of missing data, especially the latter two. However, the use of older techniques like listwise deletion and mean or mode single imputation is no longer justifiable given the accessibility and ease of use of more advanced techniques, such as multiple imputation and supervised learning imputation. <s> BIB001 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> A. UNIVARIATE MCAR IMPLEMENTATIONS <s> Pattern classification has been successfully applied in many problem domains, such as biometric recognition, document classification or medical diagnosis. Missing or unknown data are a common drawback that pattern recognition techniques need to deal with when solving real-life classification tasks. Machine learning approaches and methods imported from statistical learning theory have been most intensively studied and used in this subject. The aim of this work is to analyze the missing data problem in pattern classification tasks, and to summarize and compare some of the well-known methods used for handling missing values. <s> BIB002 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> A. UNIVARIATE MCAR IMPLEMENTATIONS <s> In Random Forests [2] several trees are constructed from bootstrapor subsamples of the original data. Random Forests have become very popular, e.g., in the fields of genetics and bioinformatics, because they can deal with high-dimensional problems including complex interaction effects. Conditional Inference Forests [8] provide an implementation of Random Forests with unbiased variable selection. Like the original Random Forests, they employ surrogate variables to handle missing values in the predictor variables. In this paper we report the results of an extensive simulation study covering both classification and regression problems under a variety of scenarios, including different missing value generating processes as well as different correlation structures between the variables. Moreover, a high dimensional setting with a high number of noise variables was considered in each case. The results compare the performance of Conditional Inference Forests with surrogate variables to that of knn imputation prior to fitting. The results show that while in some settings one or the other approach is slightly superior, there is no overall difference in the performance of Conditional Inference Forests with surrogate variables and with prior knn-imputation. <s> BIB003 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> A. UNIVARIATE MCAR IMPLEMENTATIONS <s> Datasets with missing values are frequent in real-world classification problems. It seems obvious that imputation of missing values can be considered as a series of secondary tasks, while classification is the main purpose of any machine dealing with these datasets. Consequently, Multi-Task Learning (MTL) schemes offer an interesting alternative approach to solve missing data problems. In this paper, we propose an MTL-based method for training and operating a modified Multi-Layer Perceptron (MLP) architecture to work in incomplete data contexts. The proposed approach achieves a balance between both classification and imputation by exploiting the advantages of MTL. Extensive experimental comparisons with well-known imputation algorithms show that this approach provides excellent results. The method is never worse than the traditional algorithms - an important robustness property - and, also, it clearly outperforms them in several problems. <s> BIB004 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> A. UNIVARIATE MCAR IMPLEMENTATIONS <s> A novel algorithm based on random forests with surrogate splits is proposed to address the classification problem of incomplete data without imputation.The algorithm allows each tree to cast a vote even the voting process is interrupted by missing attributes.Experimental results on various acknowledged datasets show that the proposed method is robust and efficient. Random forests (RF) is known as an efficient algorithm in classification, however it depends on the integrity of datasets. Conventional methods in dealing with missing values usually employ estimation and imputation approaches whose efficiency is tied to the assumptions of data features. Recently, algorithm of surrogate decisions in RF was developed and this paper proposes a random forests algorithm with modified surrogate splits (Adjusted Weight Voting Random Forest, AWVRF) which is able to address the incomplete data without imputation.Differing from the present surrogate method, in AWVRF algorithm, when the primary splitting attribute and the surrogate attributes of an internal node are all missing, the undergoing instance is allowed to exit at the current node with a vote. Then the weight of the vote is adjusted by the strength of the involved attributes and the final decision is made by weighted voting. AWVRF does not comprise imputation step, thus it is independent of data features.AWVRF is compared with the methods of mean imputation, LeoFill, knnimpute, BPCAfill and conventional RF with surrogate decisions (surrRF) using 50 times repeated 5-fold cross validation on 10 acknowledged datasets. In a total of 22 experiment settings, the method of AWVRF harvests the highest accuracy in 14 settings and the largest AUC in 7 settings, exhibiting its superiority over other methods. Compared with surrRF, AWVRF is significantly more efficient and remain good discrimination of prediction. Experimental results show that the present AWVRF algorithm can successfully handle the classification task for incomplete data. <s> BIB005
The MCAR univa implementation of Twala BIB001 considers that the feature to be missing, x miss , should be the one most correlated with the class labels t. Furthermore, Twala et al. considered the definition of MR as the percentage of missing values over the entire dataset, as explained in Section II. To respect the overall MR specified, the individual percentage of missing values in the chosen feature must be adjusted: for an overall percentage of MR% (over the entire dataset), an individual feature must have p × MR% of missing values, with p being the number of features in X. To determine which elements should be missing in x miss , a Bernoulli distribution is used. The Bernoulli distribution is a discrete distribution that has outcome 1 with probability prob and outcome 0 with probability 1 − prob, as shown in  . The missing elements of x miss are chosen by performing n Bernoulli trials with probability of success prob, with n being the number of patterns in the dataset and prob being the expected MR. Thus being, each pattern is associated with a probability of success (probability of being missing) equal to MR (Figure 2a) . A different MCAR univa implementation was proposed by Rieger et al. BIB003 and Xia et al. BIB005 , where random locations of x miss are chosen (using a random number generator) and their values are deleted (Figure 2b) . Finally, García-Laencina et al. BIB002 , BIB004 consider a MCAR univa implementation where x miss is either chosen randomly or according to its relevance for classification. In this implementation, the ''relevance'' of a feature is determined by the Normalized Mutual Information (NMI) between such feature and the classification target BIB004 . The missing values are randomly introduced in the feature of interest, x miss , and the missing rate is specified for that feature only (MR% of missing values in that feature alone, not in the entire dataset).