The abundance of open-source code, coupled with the success of recent advances in deep learning for natural language processing, has given rise to a promising new application of machine learning to source code. In this work, we explore the use of a Siamese recurrent neural network model on Python source code to create vectors which capture the semantics of code. We evaluate the quality of embeddings by identifying which problem from a programming competition the code solves. Our model significantly outperforms a bag-of-tokens embedding, providing promising results for improving code embeddings that can be used in future software engineering tasks.
The work of @cite_4 encodes nodes of an AST using a weighted mixture of left and right weight matrices and children of that node. A tree-based convolutional neural network (CNN) is then applied against the tree to encode the AST. The only way semantic equivalents are learned in this model are by recognizing that certain nodes have the same children. This assumption is not necessarily correct, and as such, may not fully capture the semantic meaning of code.
@cite_4
1512847993
Deep neural networks have made significant breakthroughs in many fields of artificial intelligence. However, it has not been applied in the field of programming language processing. In this paper, we propose the tree-based convolutional neural network (TBCNN) to model programming languages, which contain rich and explicit tree structural information. In our model, program vector representations are learned by the "coding" pretraining criterion based on abstract syntax trees (ASTs); the convolutional layer explicitly captures neighboring features on the tree; with the "binary continuous tree" and "3-way pooling," our model can deal with ASTs of different shapes and sizes.We evaluate the program vector representations empirically, showing that the coding criterion successfully captures underlying features of AST nodes, and that program vector representations significantly speed up supervised learning. We also compare TBCNN to baseline methods; our model achieves better accuracy in the task of program classification. To our best knowledge, this paper is the first to analyze programs with deep neural networks; we extend the scope of deep learning to the field of programming language processing. The experimental results validate its feasibility; they also show a promising future of this new research area.