The training of a deep learning model starts with a domain-specific model architecture. The final model instance that is selected, and its performance, are determined by a series of data-dependent design choices, including model initialization, data used for training, validation, and testing, optimization of loss function, and evaluation metrics. Each of these design choices impacts the generalization power of a trained model. The development of several recent deep learning-based approaches—notably transfer learning[1]}, self-supervised representation learning[2]}, and meta-learning[3]}[4]}—has been motivated by the OOD challenge. However, each of these methods focuses on only one aspect in the training pipeline of a deep model. Causal learning and mechanism-based modeling could be a more effective solution to the OOD problem [5]}, but at present these approaches can be applied only on modest scales because of data scarcity and limited domain knowledge. Solving large-scale OOD problems in biomedicine, via machine learning, would benefit from a systematic framework for integrative, beginning-to-end model development, training, and testing.
