Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. MULTIVARIATE MAR IMPLEMENTATIONS <s> Increasing the awareness of how incomplete data affects learning and classification accuracy has led to increasing numbers of missing data techniques. This article investigates the robustness and accuracy of seven popular techniques for tolerating incomplete training and test data for different patterns of missing data—different proportions and mechanisms of missing data on resulting tree-based models. The seven missing data techniques were compared by artificially simulating different proportions, patterns, and mechanisms of missing data using 21 complete datasets (i.e., with no missing values) obtained from the University of California, Irvine repository of machine-learning databases (Blake and Merz, 1998). A four-way repeated measures design was employed to analyze the data. The simulation results suggest important differences. All methods have their strengths and weaknesses. However, listwise deletion is substantially inferior to the other six techniques, while multiple imputation, that utilizes the expectation maximization algorithm, represents a superior approach to handling incomplete data. Decision tree single imputation and surrogate variables splitting are more severely impacted by missing values distributed among all attributes compared to when they are only on a single attribute. Otherwise, the imputation—versus model-based imputation procedures gave—reasonably good results although some discrepancies remained. Different techniques for addressing missing values when using decision trees can give substantially diverse results, and must be carefully considered to protect against biases and spurious findings. Multiple imputation should always be used, especially if the data contain many missing values. If few values are missing, any of the missing data techniques might be considered. The choice of technique should be guided by the proportion, pattern, and mechanisms of missing data, especially the latter two. However, the use of older techniques like listwise deletion and mean or mode single imputation is no longer justifiable given the accessibility and ease of use of more advanced techniques, such as multiple imputation and supervised learning imputation. <s> BIB001 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. MULTIVARIATE MAR IMPLEMENTATIONS <s> This review presents a practical summary of the missing data literature, including a sketch of missing data theory and descriptions of normalmodel multiple imputation (MI) and maximum likelihood methods. Practical missing data analysis issues are discussed, most notably the inclusion of auxiliary variables for improving power and reducing bias. Solutions are given for missing data challenges such as handling longitudinal, categorical, and clustered data with normal-model MI; including interactions in the missing data model; and handling large numbers of variables. The discussion of attrition and nonignorable missingness emphasizes the need for longitudinal diagnostics and for reducing the uncertainty about the missing data mechanism under attrition. Strategies suggested for reducing attrition bias include using auxiliary variables, collecting follow-up data on a sample of those initially missing, and collecting data on intent to drop out. Suggestions are given for moving forward with research on missing data and attrition. <s> BIB002 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. MULTIVARIATE MAR IMPLEMENTATIONS <s> Missing data imputation is an important research topic in data mining. The impact of noise is seldom considered in previous works while real-world data often contain much noise. In this paper, we systematically investigate the impact of noise on imputation methods and propose a new imputation approach by introducing the mechanism of Group Method of Data Handling (GMDH) to deal with incomplete data with noise. The performance of four commonly used imputation methods is compared with ours, called RIBG (robust imputation based on GMDH), on nine benchmark datasets. The experimental result demonstrates that noise has a great impact on the effectiveness of imputation techniques and our method RIBG is more robust to noise than the other four imputation methods used as benchmark. <s> BIB003 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. MULTIVARIATE MAR IMPLEMENTATIONS <s> Treatment of missing data has become increasingly significant in scientific research and engineering applications. The classic imputation strategy based on the K nearest neighbours (KNN) has been widely used to solve the plague problem. However, former studies do not give much attention to feature relevance, which has a significant impact on the selection of nearest neighbours. As a result, biased results may appear in similarity measurements. In this paper, we propose a novel method to impute missing data, named feature weighted grey KNN (FWGKNN) imputation algorithm. This approach employs mutual information (MI) to measure feature relevance. We present an experimental evaluation for five UCI datasets in three missingness mechanisms with various missing rates. Experimental results show that feature relevance has a non-ignorable influence on missing data estimation based on grey theory, and our method is considered superior to the other four estimation strategies. Moreover, the classification bias can be significantly reduced by using our approach in classification tasks. <s> BIB004 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. MULTIVARIATE MAR IMPLEMENTATIONS <s> Abstract When applying data-mining techniques to real-world data, we often find ourselves facing observations that have no value recorded for some attributes. This can be caused by several phenomena, such as a machine’s incapability to record certain characteristics or a person refusing to answer a question in a poll. Depending on that motivation, values gone missing may follow one kind of pattern or another, or describe no regularity at all. One approach to palliate the effect of missing data on machine learning tasks is to replace the missing observations. Imputation algorithms attempt to calculate a value for a missing gap, using information associated with it, i.e., the attribute and/or other values in the same observation. While several imputation methods have been proposed in the literature, few works have addressed the question of the relationship between the type of missing data, the choice of the imputation method, and the effectiveness of classification algorithms that used the imputed data. In this paper we address the relationship among these three factors. By constructing a benchmark of hundreds of databases containing different types of missing data, and applying several imputation methods and classification algorithms, we empirically show that an interaction between imputation methods and supervised classification can be deduced. Besides, differences in terms of classification performance for the same imputation method in different missing data patterns have been found. This points to the convenience of considering the combined choice of the imputation method and the classifier algorithm according to the missing data type. <s> BIB005 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> B. MULTIVARIATE MAR IMPLEMENTATIONS <s> In fact, raw data in the real world is dirty. Each large data repository contains various types of anomalous values that influence the result of the analysis, since in data mining, good models usually need good data, databases in the world are not always clean and includes noise, incomplete data, duplicate records, inconsistent data and missing values. Missing data is a common drawback in many real-world data sets. In this paper, we proposed an algorithm depending on improving (MIGEC) algorithm in the way of imputation for dealing missing values. We implement grey relational analysis (GRA) on attribute values instead of instance values, and the missing data were initially imputed by mean imputation and then estimated by our proposed algorithm (PA) used as a complete value for imputing next missing value.We compare our proposed algorithm with several other algorithms such as MMS, HDI, KNNMI, FCMOCS, CRI, CMI, NIIA and MIGEC under different missing mechanisms. Experimental results demonstrate that the proposed algorithm has less RMSE values than other algorithms under all missingness mechanisms. <s> BIB006
As stated at the beginning of Section V, there are two main approaches in what concerts MAR unifo implementations: • Consider a determining feature x obs that will determine the missing pattern of the remaining features (p − 1 features or a subset of n x miss features), which is the approach proposed by Garciarena and Santana BIB005 ; • Consider several pairs of features {x obs , x miss }: for each pair, there is a determining feature x obs that defines the missing pattern of its corresponding x miss , which is the approach of Twala BIB001 , Ali and Omer BIB006 , Zhu et al. BIB003 , and Pan et al. BIB004 . We start by the simplest MAR unifo approach, the one proposed by Garciarena and Santana BIB005 , which we designate by MAR1 unifo . MAR1 unifo considers the desired MR percentage and number of features n x miss losing their values and starts by randomly choosing the determining feature x obs and the missing features x miss . Then, similarly to MAR1 univa , elements of the x miss features corresponding to lower values of x obs are deleted (Figure 9a ). Due to the freedom of choosing a given number of n x miss , the missing rates that are possible to generate are restricted by the number of existing features p and chosen features n x miss , as will be further explained in Section VI. We follow to the MAR unifo implementation by Twala BIB001 , MAR2 unifo . As a natural extension of MAR1 univa , MAR2 unifo considers the creation of several correlated pairs where, for each pair, the most correlated feature is chosen as x miss ( Figure 9b ). As an example, for X = {x 1 , x 2 , · · · , x 8 }, we could define the pairs {x 1 , x 2 }, {x 3 , x 4 }, {x 5 , x 6 } and {x 7 , x 8 }, assuming that x 1 is highly correlated with x 2 , x 3 with x 4 and so forth. Although Twala et al. BIB001 do not specify the procedure for an odd number of features (say, 9 features in the previous example), we assume the creation of triples, where the remaining feature (in the example, x 9 ) is added to the pair that includes its most correlated feature. Following the example, assuming that the feature most correlated with x 9 is x 3 , then the triple {x 3 , x 4 , x 9 } is created instead of {x 3 , x 4 }. VOLUME 7, 2019 FIGURE 9. Schemes describing missing data patterns of (a) MAR1 unifo and (b) MAR2 unifo . After the pairs are created, the feature of each pair most correlated with the class labels t is selected to have its values missing; for triples, the two most correlated features with the class labels are chosen. The desired MR is defined for the entire dataset, but since only one feature will be missing in each pair (or two features in case of triples), the MR must be adjusted for the individual x miss features (9). For an overall MR% k = 2 × MR% for pairs k = 1.5 × MR% for triples BIB002 The positions where each feature x miss will be missing are defined according to the values of x obs : for each pair/triple, the k% percentile of x obs is determined. Then, values of x obs lower than the k% percentile are set missing. Similarly to MAR1 univa , the k% percentile of x obs returns the cut-off value for which k% of x obs are lower than that cut-off. As an example, consider an overall MR of 45% and 5 features already paired: {x 1 , x 2 } and {x 3 , x 4 , x 5 }, where x 2 , x 4 and x 5 are the most correlated with the class labels t. The missing positions in x 2 will be determined by the 2 × MR = 90% percentile of x 1 and the missing positions in x 4 and x 5 will be determined by the 1.5 × MR = 67.5% percentile of x 3 . Imagine that the 90% percentile of x 1 is 3.4: values of x 2 where the corresponding values of x 1 are lower than 3.4 will be set missing and x 2 individually will have 90% of missing values. The same is performed for x 4 and x 5 . Thus, x 1 and x 3 will be complete, x 2 will have 90% of missing values and x 4 and x 5 will have 67.5% of missing values each, resulting in an overall (0 + 90 + 0 + 67.5 + 67.5)/5 = 45% missing rate. Ali and Omer BIB006 propose a similar approach to the above, and we will refer to their approach as MAR3 unifo . In MAR3 unifo , the dataset X is first decomposed into pairs/triples of correlated features, and one feature in each pair/triple conditions the missing pattern of the remaining. However, authors do not elaborate on the choice of which features should be missing and which should be observed; therefore, we assume that the choice may be performed randomly. For each pair/triple, one feature is randomly chosen to be the determining feature x obs and the remaining are therefore the missing features, x miss . Another difference of this approach in comparison to MAR2 unifo is that it considers the median of each x obs to define the missing pattern of x miss . Given a pair of features {x obs , x miss }, the median of x obs is determined and two groups are defined: one that contains the positions of x obs whose values are lower than (or equal to) its median and the other containing the positions whose values are higher than its median. Then, one of those groups is randomly selected and will define the missingness of x miss in the following way: given a missing rate MR%, 4× MR% (or 3× MR% for triples) of missing positions are randomly chosen from the group, and the corresponding positions in x miss are set missing. The MAR unifo approach by Zhu et al. BIB003 and Pan et al. BIB004 (MAR4 unifo ) handles features according to their type. If x obs is continuous or ordinal, the median of x obs is determined and two groups are created, as in the approach by Ali and Omer BIB006 : one where the values of x obs are lower or equal to the median and other where values of x obs are higher than the median. Otherwise, if x obs is nominal, the existing categories are assigned to two groups of equal size. According to the original paper BIB003 , this assignment is performed by randomly dividing the categories of x obs into two parts, although this does not guarantee that two equally-sized groups are formed, as further detailed in Section VI. After creating the groups, one is randomly chosen and their corresponding values in x miss are set missing with 4 × MR or 3 × MR, for triples.