In contrast in this work, we aim to explore the effect of real bug fixes obtained from open source repositories on the training of learning-based bug localization and repair methods.
For this, we employ a novel dataset of 33k real-world bug fixes obtained from public Python projects.
Since this dataset is still comparably small to the datasets typically used for training localization and repair models [1]}, [2]} (which oftentimes contain millionth of artificial bugs), we propose RealiT (pronounced “reality”), a novel training scheme for learning to Repair and localize with Transformers.
RealiT  is designed to combine the strengths of both training on mutants and real bug fixes by first pre-training on a high number of mutants and then fine-tuning on a smaller set of real bug fixes. This design allows us to not only evaluate the impact of real bug fixes and mutants together on the training process, but also individually (by skipping either pre-training or fine-tuning phase).
