Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. UNIVARIATE MAR IMPLEMENTATIONS <s> Increasing the awareness of how incomplete data affects learning and classification accuracy has led to increasing numbers of missing data techniques. This article investigates the robustness and accuracy of seven popular techniques for tolerating incomplete training and test data for different patterns of missing data—different proportions and mechanisms of missing data on resulting tree-based models. The seven missing data techniques were compared by artificially simulating different proportions, patterns, and mechanisms of missing data using 21 complete datasets (i.e., with no missing values) obtained from the University of California, Irvine repository of machine-learning databases (Blake and Merz, 1998). A four-way repeated measures design was employed to analyze the data. The simulation results suggest important differences. All methods have their strengths and weaknesses. However, listwise deletion is substantially inferior to the other six techniques, while multiple imputation, that utilizes the expectation maximization algorithm, represents a superior approach to handling incomplete data. Decision tree single imputation and surrogate variables splitting are more severely impacted by missing values distributed among all attributes compared to when they are only on a single attribute. Otherwise, the imputation—versus model-based imputation procedures gave—reasonably good results although some discrepancies remained. Different techniques for addressing missing values when using decision trees can give substantially diverse results, and must be carefully considered to protect against biases and spurious findings. Multiple imputation should always be used, especially if the data contain many missing values. If few values are missing, any of the missing data techniques might be considered. The choice of technique should be guided by the proportion, pattern, and mechanisms of missing data, especially the latter two. However, the use of older techniques like listwise deletion and mean or mode single imputation is no longer justifiable given the accessibility and ease of use of more advanced techniques, such as multiple imputation and supervised learning imputation. <s> BIB001 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. UNIVARIATE MAR IMPLEMENTATIONS <s> Pattern classification has been successfully applied in many problem domains, such as biometric recognition, document classification or medical diagnosis. Missing or unknown data are a common drawback that pattern recognition techniques need to deal with when solving real-life classification tasks. Machine learning approaches and methods imported from statistical learning theory have been most intensively studied and used in this subject. The aim of this work is to analyze the missing data problem in pattern classification tasks, and to summarize and compare some of the well-known methods used for handling missing values. <s> BIB002 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. UNIVARIATE MAR IMPLEMENTATIONS <s> In Random Forests [2] several trees are constructed from bootstrapor subsamples of the original data. Random Forests have become very popular, e.g., in the fields of genetics and bioinformatics, because they can deal with high-dimensional problems including complex interaction effects. Conditional Inference Forests [8] provide an implementation of Random Forests with unbiased variable selection. Like the original Random Forests, they employ surrogate variables to handle missing values in the predictor variables. In this paper we report the results of an extensive simulation study covering both classification and regression problems under a variety of scenarios, including different missing value generating processes as well as different correlation structures between the variables. Moreover, a high dimensional setting with a high number of noise variables was considered in each case. The results compare the performance of Conditional Inference Forests with surrogate variables to that of knn imputation prior to fitting. The results show that while in some settings one or the other approach is slightly superior, there is no overall difference in the performance of Conditional Inference Forests with surrogate variables and with prior knn-imputation. <s> BIB003 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. UNIVARIATE MAR IMPLEMENTATIONS <s> Breast Cancer is the most common type of cancer in women worldwide. In spite of this fact, there are insufficient studies that, using data mining techniques, are capable of helping medical doctors in their daily practice. <s> BIB004 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. UNIVARIATE MAR IMPLEMENTATIONS <s> A novel algorithm based on random forests with surrogate splits is proposed to address the classification problem of incomplete data without imputation.The algorithm allows each tree to cast a vote even the voting process is interrupted by missing attributes.Experimental results on various acknowledged datasets show that the proposed method is robust and efficient. Random forests (RF) is known as an efficient algorithm in classification, however it depends on the integrity of datasets. Conventional methods in dealing with missing values usually employ estimation and imputation approaches whose efficiency is tied to the assumptions of data features. Recently, algorithm of surrogate decisions in RF was developed and this paper proposes a random forests algorithm with modified surrogate splits (Adjusted Weight Voting Random Forest, AWVRF) which is able to address the incomplete data without imputation.Differing from the present surrogate method, in AWVRF algorithm, when the primary splitting attribute and the surrogate attributes of an internal node are all missing, the undergoing instance is allowed to exit at the current node with a vote. Then the weight of the vote is adjusted by the strength of the involved attributes and the final decision is made by weighted voting. AWVRF does not comprise imputation step, thus it is independent of data features.AWVRF is compared with the methods of mean imputation, LeoFill, knnimpute, BPCAfill and conventional RF with surrogate decisions (surrRF) using 50 times repeated 5-fold cross validation on 10 acknowledged datasets. In a total of 22 experiment settings, the method of AWVRF harvests the highest accuracy in 14 settings and the largest AUC in 7 settings, exhibiting its superiority over other methods. Compared with surrRF, AWVRF is significantly more efficient and remain good discrimination of prediction. Experimental results show that the present AWVRF algorithm can successfully handle the classification task for incomplete data. <s> BIB005 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. UNIVARIATE MAR IMPLEMENTATIONS <s> Abstract When applying data-mining techniques to real-world data, we often find ourselves facing observations that have no value recorded for some attributes. This can be caused by several phenomena, such as a machine’s incapability to record certain characteristics or a person refusing to answer a question in a poll. Depending on that motivation, values gone missing may follow one kind of pattern or another, or describe no regularity at all. One approach to palliate the effect of missing data on machine learning tasks is to replace the missing observations. Imputation algorithms attempt to calculate a value for a missing gap, using information associated with it, i.e., the attribute and/or other values in the same observation. While several imputation methods have been proposed in the literature, few works have addressed the question of the relationship between the type of missing data, the choice of the imputation method, and the effectiveness of classification algorithms that used the imputed data. In this paper we address the relationship among these three factors. By constructing a benchmark of hundreds of databases containing different types of missing data, and applying several imputation methods and classification algorithms, we empirically show that an interaction between imputation methods and supervised classification can be deduced. Besides, differences in terms of classification performance for the same imputation method in different missing data patterns have been found. This points to the convenience of considering the combined choice of the imputation method and the classifier algorithm according to the missing data type. <s> BIB006
Regarding MAR univa, five different implementations are reviewed -MAR1 univa , MAR2 univa , MAR3 univa , MAR4 univa and MAR5 univa -following the research works of Twala BIB001 , Rieger et al. BIB003 , and Xia et al. BIB005 . All MAR implementations make use of an observed, determining feature, x d or x obs (also referred to as a causative feature in some works BIB006 ), which defines the missing locations in x miss . An example is given in Figure 3 , where the missing positions in x miss are influenced by the corresponding values of x obs . MAR1 univa refers to the research work of Twala BIB001 , and similar to the MCAR univa implementation, the feature most correlated with the class labels is chosen as x miss . Then, among the remaining features, the one most correlated with x miss is chosen to be the determining feature x obs . As explained for MCAR1 univa , the individual feature x miss must have p × MR% of missing values, since in the implementations suggested by Twala et al. the MR is defined for the entire dataset. After the pair of correlated features {x miss , x obs } is found, the locations where x miss will be missing are then defined according to the values of x obs . Let us define a variable k that represents the necessary MR adjustment, k = p × MR. The value of k% will define the percentile of x obs that must be found in order to produce the missing values in x miss : values of x miss lower than the k% percentile of x obs are set to be missing. In other words, the percentile of k% returns the cut-off value for which k% of x obs are lower than that cut-off. As an example, consider an overall MR of 45% and the pair of features {x 1 , x 2 }, where x obs is x 1 and x miss is x 2 . The missing locations in x 2 will be determined by the p × MR% = 90% percentile of x 1 . Imagine that the 90% percentile of x 1 is 3.4: values of x 2 where the corresponding values x 1 are lower than 3.4 will be set to missing values. Thus being, x 2 will have a total of 90% of missing values, resulting in an overall (0+90)/2 = 45% MR, as specified. Figure 3 shows a pictorial example of MAR1 univa where the light green positions represent the lowest values of x obs , where x miss is missing. Rieger et al. BIB003 propose implementations MAR2 univa to MAR5 univa . MAR2 univa is based on the ranks of x obs (r obs ): the probability of an element x i,miss to be missing is computed by dividing the rank of x i,miss in the determining feature x obs by the sum of all ranks of x obs BIB002 . This is also the implementation proposed by Xia et al. BIB005 . The patterns to have missing values in x miss are then sampled according to their resulting probability P(x i,miss ). The choice of x miss and x obs is arbitrary and can either be random or specified by the researcher. Furthermore, the definition of MR is not described in the original paper and one might consider a MR for the entire dataset or for each feature individually. In MAR3 univa , the patterns are divided into two groups according to the median of the determining feature x obs , so that the probability of missingness is different among groups according to (7) (nG BIB004 and nG 2 are the number of patterns in Group 1 and Group 2, respectively). Again, the patterns are sampled according to the established probability of missingness  . In MAR4 univa , the locations of x miss that will be missing are chosen according to the positions where x obs assumes its highest values (Figure 4a ). MAR5 univa considers both the highest and lowest values of x obs : given the necessary number of elements to have missing values for the specified MR, call it N , MAR5 univa sets N /2 elements to have missing values according to the highest values of x obs , and N /2 according to the lowest (Figure 4b ).