We propose a deep learning-based approach to the problem of premise selection: selecting mathematical statements relevant for proving a given conjecture. We represent a higher-order logic formula as a graph that is invariant to variable renaming but still fully preserves syntactic and semantic information. We then embed the graph into a vector via a novel embedding method that preserves the information of edge ordering. Our approach achieves state-of-the-art results on the HolStep dataset, improving the classification accuracy from 83 to 90.3 .
@cite_4 were the first to apply deep networks to proof guidance. They experimented with both sequential representations and tree representations (recursive neural networks @cite_28 @cite_12 ). Note that their tree representations are simply the parse trees, which, unlike our graphs, are not invariant to variable renaming and do not capture how quantifiers bind variables. @cite_45 used GRU networks to guide the exploration of partial proof trees, with formulas represented as sequences of tokens.
@cite_28 @cite_45 @cite_4 @cite_12
98255950 2617995259 1753823461 1512847993
We propose a new fast purely discriminative algorithm for natural language parsing, based on a “deep” recurrent convolutional graph transformer network (GTN). Assuming a decomposition of a parse tree into a stack of “levels”, the network predicts a level of the tree taking into account predictions of previous levels. Using only few basic text features which leverage word representations from Collobert and Weston (2008), we show similar performance (in F1 score) to existing pure discriminative parsers and existing “benchmark” parsers (like Collins parser, probabilistic context-free grammars based), with a huge speed advantage. We introduce neural networks for end-to-end differentiable proving of queries to knowledge bases by operating on dense vector representations of symbols. These neural networks are constructed recursively by taking inspiration from the backward chaining algorithm as used in Prolog. Specifically, we replace symbolic unification with a differentiable computation on vector representations of symbols using a radial basis function kernel, thereby combining symbolic reasoning with learning subsymbolic vector representations. By using gradient descent, the resulting neural network can be trained to infer facts from a given incomplete knowledge base. It learns to (i) place representations of similar symbols in close proximity in a vector space, (ii) make use of such similarities to prove queries, (iii) induce logical rules, and (iv) use provided and induced logical rules for multi-hop reasoning. We demonstrate that this architecture outperforms ComplEx, a state-of-the-art neural link prediction model, on three out of four benchmark knowledge bases while at the same time inducing interpretable function-free first-order logic rules. Recently, deep architectures, such as recurrent and recursive neural networks have been successfully applied to various natural language processing tasks. Inspired by bidirectional recurrent neural networks which use representations that summarize the past and future around an instance, we propose a novel architecture that aims to capture the structural information around an input, and use it to label instances. We apply our method to the task of opinion expression extraction, where we employ the binary parse tree of a sentence as the structure, and word vector representations as the initial representation of a single token. We conduct preliminary experiments to investigate its performance and compare it to the sequential approach. Deep neural networks have made significant breakthroughs in many fields of artificial intelligence. However, it has not been applied in the field of programming language processing. In this paper, we propose the tree-based convolutional neural network (TBCNN) to model programming languages, which contain rich and explicit tree structural information. In our model, program vector representations are learned by the "coding" pretraining criterion based on abstract syntax trees (ASTs); the convolutional layer explicitly captures neighboring features on the tree; with the "binary continuous tree" and "3-way pooling," our model can deal with ASTs of different shapes and sizes.We evaluate the program vector representations empirically, showing that the coding criterion successfully captures underlying features of AST nodes, and that program vector representations significantly speed up supervised learning. We also compare TBCNN to baseline methods; our model achieves better accuracy in the task of program classification. To our best knowledge, this paper is the first to analyze programs with deep neural networks; we extend the scope of deep learning to the field of programming language processing. The experimental results validate its feasibility; they also show a promising future of this new research area.