We design several distinct components for the structural CNN, suited to program generation: (1) We first adopt an idea of tree-based convolution that applies sliding windows on the AST structures [1]}. Then we design another CNN module to the pre-order traversal of nodes in the partial AST. These two types of CNNs capture neighboring information not only in the sequence but also in the tree structure.
(2) To enhance “autoregressiveness,” we apply another CNN module to the ancestors of the node to be generated, and thus the network is aware of where to generate at a certain step.
(3) We design a dedicated attentive pooling mechanism that aggregates CNN features interacting with different neural modules. In particular, we find it useful to consider the scope names (e.g., function and method names) during code generation, and use such information as the controller of several attentive pooling layers.
