Dynamic Programming Languages are quite popular because they increase the programmer's productivity. However, the absence of types in the source code makes the program written in these languages difficult to understand and virtual machines that execute these programs cannot produced optimized code. To overcome this challenge, we develop a technique to predict types of all identifiers including variables, and function return types. We propose the first implementation of @math order Inside Outside Recursive Neural Networks with two variants (i) Child-Sum Tree-LSTMs and (ii) N-ary RNNs that can handle large number of tree branching. We predict the types of all the identifiers given the Abstract Syntax Tree by performing just two passes over the tree, bottom-up and top-down, keeping both the content and context representation for all the nodes of the tree. This allows these representations to interact by combining different paths from the parent, siblings and children which is crucial for predicting types. Our best model achieves 44.33 across 21 classes and top-3 accuracy of 71.5 on our gathered Python data set from popular Python benchmarks.
The architecture and algorithm introduced by Le and Zuidema @cite_9 allows information to flow not only bottom-up, as in traditional recursive neural network, but top-down as well. Every node in the hierarchical structure is associated with two vectors: (i) inside representation to represent content under the node, and (ii) outside representation to represent context (see Figure ). The inside vectors are computed in the bottom-up (post-order traversal) manner combining the inside representation of the children, thus capturing the content of the sub-tree. The outside vectors are computed in the top-down (pre-order traversal) manner combining the representation of the parent's outside (context) and the siblings' inside (content), thus capturing the rest of the tree (see Figure ). This approach is very critical when we want to classify leaf nodes and the information may come from any combination of paths in the tree.
@cite_9
2251628756
We propose the first implementation of an infinite-order generative dependency model. The model is based on a new recursive neural network architecture, the Inside-Outside Recursive Neural Network. This architecture allows information to flow not only bottom-up, as in traditional recursive neural networks, but also topdown. This is achieved by computing content as well as context representations for any constituent, and letting these representations interact. Experimental results on the English section of the Universal Dependency Treebank show that the infinite-order model achieves a perplexity seven times lower than the traditional third-order model using counting, and tends to choose more accurate parses in k-best lists. In addition, reranking with this model achieves state-of-the-art unlabelled attachment scores and unlabelled exact match scores.