TOKEN PAIRING TO IMPROVE NEURAL PROGRAM SYNTHESIS MODELS
In neural program synthesis (NPS) a network is trained to output or aid in the output of code that satisfies a given program specification. In our work, we make modifications upon the simple sequence to sequence (Seq2Seq) LSTM model. Extending the most successful techniques from previous works, we guide a beam search with an encoder decoder scheme augmented with attention mechanisms and a specialized syntax layer. But one of the withstanding difficulties of NPS is the implicit tree structure of programs, which makes it inherently more difficult for linearly structured models. To address this, we experiment with a novel technique we call token pairing. Our model is trained and evaluated on AlgoLisp, a dataset of English description to code programming problems paired with example solutions and test cases on which to evaluate programs. We also create a new interpreter for AlgoLisp that fixes the bugs present in the builtin executor. In the end, our model achieves 99.24% accuracy at evaluation, which greatly improves on the previous state of the art of 95.80% while using fewer of parameters.