For instance, inspired by BART, [1]} have developed PLBART, which is a large pretrained language model that has been fine-tuned for a number of code understanding (e.g., code summarization) and generation (e.g., code translation) tasks. Similarly, inspired by T5, [2]} built a larger model CodeT5, which is pretrained on eight programming languages together
with their natural language comments collected from open-source
repositories.
Specially, it is pretrained to incorporate information from identifiers in the code.
CodeT5 has shown promising results in code-related generation
tasks such as code summarization, code generation and code-related
understanding tasks such as clone detection and defect detection.
