Many challenging prediction problems, from molecular optimization to program synthesis, involve creating complex structured objects as outputs. However, available training data may not be sufficient for a generative model to learn all possible complex transformations. By leveraging the idea that evaluation is easier than generation, we show how a simple, broadly applicable, iterative target augmentation scheme can be surprisingly effective in guiding the training and use of such models. Our scheme views the generative model as a prior distribution, and employs a separately trained filter as the likelihood. In each augmentation step, we filter the model's outputs to obtain additional prediction targets for the next training epoch. Our method is applicable in the supervised as well as semi-supervised settings. We demonstrate that our approach yields significant gains over strong baselines both in molecular optimization and program synthesis. In particular, our augmented model outperforms the previous state-of-the-art in molecular optimization by over 10% in absolute gain. Deep architectures are becoming increasingly adept at generating complex objects such as images, text, molecules, or programs. Many useful generation problems can be seen as translation tasks, where the goal is to take a source (precursor) object such as a molecule and turn it into a target satisfying given design characteristics. Indeed, molecular optimization of this kind is a key step in drug development, though the adoption of automated tools remains limited due to accuracy concerns. We propose here a simple, broadly applicable meta-algorithm to improve translation quality. Translation is a challenging task for many reasons. Objects are complex and the available training data pairs do not fully exemplify the intricate ways in which valid targets can be created from the precursors. Moreover, precursors provided at test time may differ substantially from those available during training -a scenario common in drug development. While data augmentation and semisupervised methods have been used to address some of these challenges, the focus has been on either simple prediction tasks (e.g., classification) or augmenting data primarily on the source side. We show, in contrast, that iteratively augmenting translation targets significantly improves performance on complex generation tasks in which each precursor corresponds to multiple possible outputs. Our iterative target augmentation approach builds on the idea that it is easier to evaluate candidate objects than to generate them. Thus a learned predictor of target object quality (a filter) can be used to effectively guide the generation process. To this end, we construct an external filter and apply it to the complex generative model's sampled translations of training set precursors. Candidate translations that pass the filter criteria become part of the training data for the next training epoch. The translation model is therefore iteratively guided to generate candidates that pass the filter. The generative model can be viewed as an adaptively tuned prior distribution over complex objects, with the filter as the likelihood. For this reason, it is helpful to apply the filter at test time as well, or to use the approach transductively 1 to adapt the generation process to novel test cases. The approach is reminiscent of self-training or reranking approaches employed with some success for parsing (McClosky et al., 2006; Charniak et al., 2016) . However, in our case, it is the candidate generator that is complex while the filter is relatively simple and remains fixed during the iterative process. We demonstrate that our meta-algorithm is quite effective and consistent in its ability to improve translation quality in the supervised setting. On a program synthesis task (Bunel et al., 2018) , under the same neural architecture, our augmented model outperforms their MLE baseline by 8% and their RL model by 3% in top-1 generalization accuracy (in absolute measure). On molecular optimization (Jin et al., 2019a) , their sequence to sequence translation baseline, when combined with our target data augmentation, achieves a new state-of-the-art result and outperforms their graph based approach by over 10% in success rate. Their graph based methods are also improved by iterative target augmentation with more than 10% absolute gain. The results reflect the difficulty of generation in comparison to evaluation; indeed, the gains persist even if the filter quality is reduced somewhat. Source side augmentation with unlabeled precursors (the semi-supervised setting) can further improve results, but only when combined with the filter in the target data augmentation framework. We provide ablation experiments to empirically highlight the effect of our method and also offer some theoretical insights for why it is effective. In this work, we have presented an iterative target augmentation framework for generation tasks with multiple possible outputs. Our approach is theoretically motivated, and we demonstrate strong empirical results on both the molecular optimization and program synthesis tasks, significantly outperforming baseline models on each task. Moreover, we find that iterative target augmentation is complementary to architectural improvements, and that its effect can be quite robust to the quality of the external filter. Finally, in principle our approach is applicable to other domains as well.
We improve generative models by proposing a meta-algorithm that filters new training data from the model's outputs.
{"src": "scitldr-AIC_train", "paper_id": "rylztAEYvr"}