Pretrained Models for Code Edits.
Prior work already explored applying pretrained models, despite not
well-suited, on editing tasks. [1]} used
PLBART for code bug fixing, which we compared to in
our work. [2]} pretrained a model on 67K
Java repositories mined from GitHub and fine-tuned for bug fixing task.
Codex [3]} showed promising performance on editing tasks by specifying the existing code as a prompt and providing an edit instruction to the model.
[4]} and [5]} both proposed
a transformer-based encoder-decoder model pretrained on large code reviewer
specific data for code review related tasks including
code change quality estimation, review comment generation and
code refinement.
As we showed in this work, the combination of an edit-based language
model and a standard language model can achieve better performance
than using the standard language model alone.
