Pre-training Transformers for Molecular Property Prediction Using Reaction Prediction
Molecular property prediction is essential in chemistry, especially for drug
discovery applications. However, available molecular property data is often
limited, encouraging the transfer of information from related data. Transfer
learning has had a tremendous impact in fields like Computer Vision and Natural
Language Processing signaling for its potential in molecular property
prediction. We present a pre-training procedure for molecular representation
learning using reaction data and use it to pre-train a SMILES Transformer. We
fine-tune and evaluate the pre-trained model on 12 molecular property
prediction tasks from MoleculeNet within physical chemistry, biophysics, and
physiology and show a statistically significant positive effect on 5 of the 12
tasks compared to a non-pre-trained baseline model.