The abundance of open-source code, coupled with the success of recent advances in deep learning for natural language processing, has given rise to a promising new application of machine learning to source code. In this work, we explore the use of a Siamese recurrent neural network model on Python source code to create vectors which capture the semantics of code. We evaluate the quality of embeddings by identifying which problem from a programming competition the code solves. Our model significantly outperforms a bag-of-tokens embedding, providing promising results for improving code embeddings that can be used in future software engineering tasks.
introduce CODEnn @cite_7 , which creates vector representations of Java source code by jointly embedding code with a natural language description of the method. Their architecture uses recurrent neural networks (RNN) on sequences of API calls and on tokens from the method name. It then fuses this with the output of a multi-layer perceptron which takes inputs from the non-API tokens in the code. By jointly embedding the code with natural language, the learned vectors are tailored to summarize code at a human-level description, which may not always be accurate (given that code often evolves independently from comments), and is limited by the ability of natural langauges to describe specifications. Additionally, natural languages (and consequently, code comments) are context-sensitive, so the comment may be missing crucial information about the semantics of the code.
@cite_7
2794601162
To implement a program functionality, developers can reuse previously written code snippets by searching through a large-scale codebase. Over the years, many code search tools have been proposed to help developers. The existing approaches often treat source code as textual documents and utilize information retrieval models to retrieve relevant code snippets that match a given query. These approaches mainly rely on the textual similarity between source code and natural language query. They lack a deep understanding of the semantics of queries and source code. In this paper, we propose a novel deep neural network named CODEnn (Code-Description Embedding Neural Network). Instead of matching text similarity, CODEnn jointly embeds code snippets and natural language descriptions into a high-dimensional vector space, in such a way that code snippet and its corresponding description have similar vectors. Using the unified vector representation, code snippets related to a natural language query can be retrieved according to their vectors. Semantically related words can also be recognized and irrelevant noisy keywords in queries can be handled. As a proof-of-concept application, we implement a code search tool named D eep CS using the proposed CODEnn model. We empirically evaluate D eep CS on a large scale codebase collected from GitHub. The experimental results show that our approach can effectively retrieve relevant code snippets and outperforms previous techniques.