@inproceedings{
chen2024beyond,
title={Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization},
author={Dingshuo Chen and Zhixun Li and Yuyan Ni and Guibin Zhang and Ding Wang and Qiang Liu and Shu Wu and Jeffrey Xu Yu and Liang Wang},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=GJ0qIevGjD}
}
With the emergence of various molecular tasks and massive datasets, how to perform efficient training has become an urgent yet under-explored issue in the area. Data pruning (DP), as an oft-stated approach to saving training burdens, filters out less influential samples to form a coreset for training. However, the increasing reliance on pretrained models for molecular tasks renders traditional in-domain DP methods incompatible. Therefore, we propose a **Mol**ecular data **P**runing framework for **e**nhanced **G**eneralization (**MolPeg**), which focuses on the source-free data pruning scenario, where data pruning is applied with pretrained models. By maintaining two models with different updating paces during training, we introduce a novel scoring function to measure the informativeness of samples based on the loss discrepancy. As a plug-and-play framework, MolPeg realizes the perception of both source and target domain and consistently outperforms existing DP methods across four downstream tasks. Remarkably, it can surpass the performance obtained from full-dataset training, even when pruning up to 60-70% of the data on HIV and PCBA dataset. Our work suggests that the discovery of effective data-pruning metrics could provide a viable path to both enhanced efficiency and superior generalization in transfer learning.
Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization
Dingshuo Chen Zhixun Li Yuyan Ni Guibin Zhang Ding Wang Qiang Liu Shu Wu Jeffrey Xu Yu Liang Wang