@inproceedings{
larsen2023improving,
title={Improving Precision in Language Models Learning from Invalid Samples},
author={Niels Larsen and Giorgio Giannone and Ole Winther and Kai Blin},
booktitle={NeurIPS 2023 Generative AI and Biology (GenBio) Workshop},
year={2023},
url={https://openreview.net/forum?id=73wnK2BvWg}
}
Language Models are powerful generative tools capable of learning intricate patterns from vast amounts of unstructured data. 
Nevertheless, in domains that demand precision, such as science and engineering, the primary objective is to obtain an exact and accurate answer. Precision takes precedence in these contexts. In specialized tasks like chemical compound generation, the emphasis is on output accuracy rather than response diversity. Traditional self-refinement methods are ineffective for such domain-specific input/output pairs, unlike general language tasks. In this study, we introduce invalid2valid, a powerful and general post-processing mechanism that can significantly enhance precision in language models for input/output tasks spanning different domains and specialized applications.
Improving Precision in Language Models Learning from Invalid Samples
Niels Larsen Giorgio Giannone Ole Winther Kai Blin