Generating Synthetic Missing Data: A Review by Missing Mechanism <s> This review focuses on <s> Pattern classification has been successfully applied in many problem domains, such as biometric recognition, document classification or medical diagnosis. Missing or unknown data are a common drawback that pattern recognition techniques need to deal with when solving real-life classification tasks. Machine learning approaches and methods imported from statistical learning theory have been most intensively studied and used in this subject. The aim of this work is to analyze the missing data problem in pattern classification tasks, and to summarize and compare some of the well-known methods used for handling missing values. <s> BIB001 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> This review focuses on <s> This review presents a practical summary of the missing data literature, including a sketch of missing data theory and descriptions of normalmodel multiple imputation (MI) and maximum likelihood methods. Practical missing data analysis issues are discussed, most notably the inclusion of auxiliary variables for improving power and reducing bias. Solutions are given for missing data challenges such as handling longitudinal, categorical, and clustered data with normal-model MI; including interactions in the missing data model; and handling large numbers of variables. The discussion of attrition and nonignorable missingness emphasizes the need for longitudinal diagnostics and for reducing the uncertainty about the missing data mechanism under attrition. Strategies suggested for reducing attrition bias include using auxiliary variables, collecting follow-up data on a sample of those initially missing, and collecting data on intent to drop out. Suggestions are given for moving forward with research on missing data and attrition. <s> BIB002 </s> Generating Synthetic Missing Data: A Review by Missing Mechanism <s> This review focuses on <s> Datasets with missing values are frequent in real-world classification problems. It seems obvious that imputation of missing values can be considered as a series of secondary tasks, while classification is the main purpose of any machine dealing with these datasets. Consequently, Multi-Task Learning (MTL) schemes offer an interesting alternative approach to solve missing data problems. In this paper, we propose an MTL-based method for training and operating a modified Multi-Layer Perceptron (MLP) architecture to work in incomplete data contexts. The proposed approach achieves a balance between both classification and imputation by exploiting the advantages of MTL. Extensive experimental comparisons with well-known imputation algorithms show that this approach provides excellent results. The method is never worse than the traditional algorithms - an important robustness property - and, also, it clearly outperforms them in several problems. <s> BIB003
Step 2 -Missing Data Generationby discussing the existing approaches found in the literature. Over the years, a great effort has been done in what concerns the comparison of different approaches to handle MD (deletion, imputation, model-based approaches) BIB001 ,  , BIB002 ), with a special emphasis on the evaluation of new machine learning methods for imputation (Steps 3 and 4) BIB003 . However, the process of missing data generation strongly conditions the validity of the conclusions derived from the following steps. If the MD generation approach is ill-defined, some hitches may arise during the experimental setup (e.g. the desired missing rate may not be achieved for some scenarios, the mechanisms under which data should be missing may be broken). Thus, the established missing data setup may deviate from what was intended by the researcher, causing the derived conclusions to be biased or invalid. In sum, although the evaluation of different methods to synthetically generate MD remains an understudied topic, it is of crucial importance since they define the working ground for the missing data experiments. The goal of this paper is to illustrate several approaches to missing data generation, thoroughly analyse their practical details and discuss their application in realworld contexts from a theoretical and empirical perspective. To the extent of the authors' knowledge, there is no systematic research on the assessment and evaluation of missing data generation approaches, which constitutes the novelty of this work. The contributions of this research are as follows: • Providing a thorough analysis of the practical details of each approach and uncovering some issues that may arise during their application; • Discussing the limitations and restrictions of each approach (e.g. maximum possible MR that they are able to generate); • Explaining the MR assumptions of each approach (i.e., whether MR is defined for the entire dataset or for a single feature) and presenting the necessary MR adjustments accordingly; • Suggesting some modifications to the original approaches and elaborating on some implementation details left undiscussed in the original papers. Considering the contributions given above, this review could prove instrumental for researchers from the Machine Learning field as well as for researchers farther from this field. Researchers familiarized with the missing data topic may learn from an extensive analysis on missing data generation algorithms (their benefits, flaws and limitations) while researchers farther from this topic encounter a complete review where the key concepts on missing data theory, as well as several approaches to missing data generation, are well described and illustrated recurring to schemas and practical examples. The structure of the paper is as follows: Section II starts by introducing some important notation that will be used throughout the paper, whereas Section III formally describes and illustrates the existing missing data mechanisms. Then, in Sections IV and V, we review several univariate and multivariate implementations for missing data generation that are generic and applicable in several domains, and thoroughly analyse and compare them (by missing mechanism and configuration) in Section VI. Section VII discusses some domain-specific missing data generation approaches, tailored to the peculiarities of a given context, while Section VIII summarizes the key issues one might face when performing experiments using the reviewed generic approaches and discusses the advantages/disadvantages of domain-specific approaches. Finally, Section IX concludes the paper and outlines some potential directions for future research.