PROMISSING: Pruning Missing Values in Neural Networks
While data are the primary fuel for machine learning models, they often
suffer from missing values, especially when collected in real-world scenarios.
However, many off-the-shelf machine learning models, including artificial
neural network models, are unable to handle these missing values directly.
Therefore, extra data preprocessing and curation steps, such as data
imputation, are inevitable before learning and prediction processes. In this
study, we propose a simple and intuitive yet effective method for pruning
missing values (PROMISSING) during learning and inference steps in neural
networks. In this method, there is no need to remove or impute the missing
values; instead, the missing values are treated as a new source of information
(representing what we do not know). Our experiments on simulated data, several
classification and regression benchmarks, and a multi-modal clinical dataset
show that PROMISSING results in similar prediction performance compared to
various imputation techniques. In addition, our experiments show models trained
using PROMISSING techniques are becoming less decisive in their predictions
when facing incomplete samples with many unknowns. This finding hopefully
advances machine learning models from being pure predicting machines to more
realistic thinkers that can also say "I do not know" when facing incomplete
sources of information.