StructCoder: Structure-Aware Transformer for Code Generation
There has been a recent surge of interest in automating software engineering
tasks using deep learning. This work addresses the problem of code generation
where the goal is to generate target code given source code in a different
language or a natural language description. Most of the state-of-the-art deep
learning models for code generation use training strategies that are primarily
designed for natural language. However, understanding and generating code
requires a more rigorous comprehension of the code syntax and semantics. With
this motivation, we develop an encoder-decoder Transformer model where both the
encoder and decoder are trained to recognize the syntax and data flow in the
source and target codes, respectively. We not only make the encoder
structure-aware by leveraging the source code's syntax tree and data flow
graph, but we also ensure that our decoder preserves the syntax and data flow
of the target code by introducing two auxiliary tasks: AST (Abstract Syntax
Tree) paths prediction and data flow prediction. To the best of our knowledge,
this is the first work to introduce a structure-aware Transformer decoder to
enhance the quality of generated code by modeling target syntax and data flow.
The proposed StructCoder model achieves state-of-the-art performance on code
translation and text-to-code generation tasks in the CodeXGLUE benchmark.