Alternatively, edits can be encoded into vector representations (or
embeddings). [1]} studied learning edit
embeddings for natural language generation in a prototype-then-edit
style. [2]} studied learning code edits as
embeddings and then applying them to natural language insertion and
code bug fixing. [3]} developed a
retrieve-and-edit framework for text-to-code generation, where the
edits are learned as parameters of a seq2seq model.
Similarly, [4]} proposed a retrieve-and-edit framework for code summarization task where the model first learns an edit vector and then generate the revised summary conditioned on it.
Although learning edits as embeddings can be
effective for individual tasks, it is not suitable to be used in the
pretraining fine-tuning paradigm, because there is a large domain gap
between the edit embeddings learned on different tasks. More over,
edit embeddings are less explainable compared to the explicit edit
representations we use.
