We introduce a new paradigm of learning for reasoning, understanding, and prediction, as well as the scaffolding network to implement this paradigm. The scaffolding network embodies an incremental learning approach that is formulated as a teacher-student network architecture to teach machines how to understand text and do reasoning. The key to our computational scaffolding approach is the interactions between the teacher and the student through sequential questioning. The student observes each sentence in the text incrementally, and it uses an attention-based neural net to discover and register the key information in relation to its current memory. Meanwhile, the teacher asks questions about the observed text, and the student network gets rewarded by correctly answering these questions. The entire network is updated continually using reinforcement learning. Our experimental results on synthetic and real datasets show that the scaffolding network not only outperforms state-of-the-art methods but also learns to do reasoning in a scalable way even with little human generated input.
Another inspiration of this work is from the incremental learning framework in which the input text is received in sequences of sentences and encoded separately. While @cite_15 uses an Markov Decision Process (MDP) framework to learn when to stop reading to answer a focused question, @cite_20 builds an incremental classifier for a trivia game to decide whether additional features are needed. The closest to our work is the Recurrent Entity Networks @cite_4 , in which the authors try to learn an internal state representation of each sentence in a sequential order and store in memory by parallel recurrent units with tied weights using gating functions. Although our network's structure has similarities, they use supervised objective while we view the task as an MDP @cite_18 , in which our agent choses its actions to maximize its rewards towards learning to encode text by answering teacher's questions.
@cite_18 @cite_15 @cite_4 @cite_20
1789842862 2403702038 2951976932
This paper addresses cost sensitive classi cation in the setting where there are costs for measuring each attribute as well as costs for misclassi cation errors We show how to formulate this as a Markov Decision Pro cess in which the transition model is learned from the training data Speci cally we as sume a set of training examples in which all attributes and the true class have been measured We describe a learning algorithm based on the AO heuristic search procedure that searches for the classi cation policy with minimum expected cost We provide an ad missible heuristic for AO that substantially reduces the number of nodes that need to be expanded particularly when attribute mea surement costs are high To further prune the search space we introduce a statistical prun ing heuristic based on the principle that if the values of two policies are statistically in distinguishable on the training data then we can prune one of the policies from the AO search space Experiments with realis tic and synthetic data demonstrate that these heuristics can substantially reduce the mem ory needed for AO search without signi cantly a ecting the quality of the learned pol icy Hence these heuristics expand the range of cost sensitive learning problems for which AO is feasible Traditional dialog systems used in goal-oriented applications require a lot of domain-specific handcrafting, which hinders scaling up to new domains. End-to-end dialog systems, in which all components are trained from the dialogs themselves, escape this limitation. But the encouraging success recently obtained in chit-chat dialog may not carry over to goal-oriented settings. This paper proposes a testbed to break down the strengths and shortcomings of end-to-end dialog systems in goal-oriented applications. Set in the context of restaurant reservation, our tasks require manipulating sentences and symbols, so as to properly conduct conversations, issue API calls and use the outputs of such calls. We show that an end-to-end dialog system based on Memory Networks can reach promising, yet imperfect, performance and learn to perform non-trivial operations. We confirm those results by comparing our system to a hand-crafted slot-filling baseline on data from the second Dialog State Tracking Challenge (, 2014a). We show similar result patterns on data extracted from an online concierge service. We introduce a new model, the Recurrent Entity Network (EntNet). It is equipped with a dynamic long-term memory which allows it to maintain and update a representation of the state of the world as it receives new data. For language understanding tasks, it can reason on-the-fly as it reads text, not just when it is required to answer a question or respond as is the case for a Memory Network (, 2015). Like a Neural Turing Machine or Differentiable Neural Computer (, 2014; 2016) it maintains a fixed size memory and can learn to perform location and content-based read and write operations. However, unlike those models it has a simple parallel architecture in which several memory locations can be updated simultaneously. The EntNet sets a new state-of-the-art on the bAbI tasks, and is the first method to solve all the tasks in the 10k training examples setting. We also demonstrate that it can solve a reasoning task which requires a large number of supporting facts, which other methods are not able to solve, and can generalize past its training horizon. It can also be practically used on large scale datasets such as Children's Book Test, where it obtains competitive performance, reading the story in a single pass.