Generative models forsource code are an interesting structured prediction problem, requiring to reason about both hard syntactic and semantic constraints as well as about natural, likely programs. We present a novel model for this problem that uses a graph to represent the intermediate state of the generated output. Our model generates code by interleaving grammar-driven expansion steps with graph augmentation and neural message passing steps. An experimental evaluation shows that our new model can generate semantically meaningful expressions, outperforming a range of strong baselines. Learning to understand and generate programs is an important building block for procedural artificial intelligence and more intelligent software engineering tools. It is also an interesting task in the research of structured prediction methods: while imbued with formal semantics and strict syntactic rules, natural source code carries aspects of natural languages, since it acts as a means of communicating intent among developers. Early works in the area have shown that approaches from natural language processing can be applied successfully to source code BID11 , whereas the programming languages community has had successes in focusing exclusively on formal semantics. More recently, methods handling both modalities (i.e., the formal and natural language aspects) have shown successes on important software engineering tasks BID22 BID4 BID1 and semantic parsing (Yin & Neubig, 2017; BID20 ).However , current generative models of source code mostly focus on only one of these modalities at a time. For example , program synthesis tools based on enumeration and deduction BID24 BID19 BID8 BID7 are successful at generating programs that satisfy some (usually incomplete) formal specification but are often obviously wrong on manual inspection, as they cannot distinguish unlikely from likely, "natural" programs. On the other hand, learned code models have succeeded in generating realistic-looking programs BID17 BID5 BID18 BID20 Yin & Neubig, 2017) . However, these programs often fail to be semantically relevant, for example because variables are not used consistently.In this work, we try to overcome these challenges for generative code models and present a general method for generative models that can incorporate structured information that is deterministically available at generation time. We focus our attention on generating source code and follow the ideas of program graphs BID1 ) that have been shown to learn semantically meaningful representations of (pre-existing) programs. To achieve this, we lift grammar-based tree decoder models into the graph setting, where the diverse relationships between various elements of the generated code can be modeled. For this, the syntax tree under generation is augmented with additional edges denoting known relationships (e.g., last use of variables). We then interleave the steps of the generative procedure with neural message passing BID9 to compute more precise representations of the intermediate states of the program generation. This is fundamentally different from sequential generative models of graphs BID14 BID23 , which aim to generate all edges and nodes, whereas our graphs are deterministic augmentations of generated trees.To summarize, we present a) a general graph-based generative procedure for highly structured objects, incorporating rich structural information; b) ExprGen, a new code generation task focused on (a, u) ← insertChild(a, ) We presented a generative code model that leverages known semantics of partially generated programs to direct the generative procedure. The key idea is to augment partial programs to obtain a graph, and then use graph neural networks to compute a precise representation for the partial program. This representation then helps to better guide the remainder of the generative procedure. We have shown that this approach can be used to generate small but semantically interesting expressions from very imprecise context information. The presented model could be useful in program repair scenarios (where repair proposals need to be scored, based on their context) or in the code review setting (where it could highlight very unlikely expressions). We also believe that similar models could have applications in related domains, such as semantic parsing, neural program synthesis and text generation.
Representing programs as graphs including semantics helps when generating programs
{"src": "scitldr-AIC_train", "paper_id": "Bke4KsA5FX"}