Most tasks in natural language processing can be cast into question answering (QA) problems over language input. We introduce the dynamic memory network (DMN), a neural network architecture which processes input sequences and questions, forms episodic memories, and generates relevant answers. Questions trigger an iterative attention process which allows the model to condition its attention on the inputs and the result of previous iterations. These results are then reasoned over in a hierarchical recurrent sequence model to generate answers. The DMN can be trained end-to-end and obtains state-of-the-art results on several types of tasks and datasets: question answering (Facebook's bAbI dataset), text classification for sentiment analysis (Stanford Sentiment Treebank) and sequence modeling for part-of-speech tagging (WSJ-PTB). The training for these different tasks relies exclusively on trained word vector representations and input-question-answer triplets.
The episodic memory in humans stores specific experiences in their spatial and temporal context. For instance, it might contain the first memory somebody has of flying a hang glider. Eichenbaum and Cohen have argued that episodic memories represent a form of relationship (i.e., relations between spatial, sensory and temporal information) and that the hippocampus is responsible for general relational learning @cite_23 . Interestingly, it also appears that the hippocampus is active during transitive inference @cite_12 , and disruption of the hippocampus impairs this ability @cite_31 .
@cite_31 @cite_12 @cite_23
1970006557 2009928698 2160623437 2143853187
Finding one's way around an environment and remembering the events that occur within it are crucial cognitive abilities that have been linked to the hippocampus and medial temporal lobes. Our review of neuropsychological, behavioral, and neuroimaging studies of human hippocampal involvement in spatial memory concentrates on three important concepts in this field: spatial frameworks, dimensionality, and orientation and self-motion. We also compare variation in hippocampal structure and function across and within species. We discuss how its spatial role relates to its accepted role in episodic memory. Five related studies use virtual reality to examine these two types of memory in ecologically valid situations. While processing of spatial scenes involves the parahippocampus, the right hippocampus appears particularly involved in memory for locations within an environment, with the left hippocampus more involved in context-dependent episodic or autobiographical memory. Human declarative memory involves a systematic organization of information that supports generalizations and inferences from acquired knowledge. This kind of memory depends on the hippocampal region in humans, but the extent to which animals also have declarative memory, and whether inferential expression of memory depends on the hippocampus in animals, remains a major challenge in cognitive neuroscience. To examine these issues, we used a test of transitive inference pioneered by Piaget to assess capacities for systematic organization of knowledge and logical inference in children. In our adaptation of the test, rats were trained on a set of four overlapping odor discrimination problems that could be encoded either separately or as a single representation of orderly relations among the odor stimuli. Normal rats learned the problems and demonstrated the relational memory organization through appropriate transitive inferences about items not presented together during training. By contrast, after disconnection of the hippocampus from either its cortical or subcortical pathway, rats succeeded in acquiring the separate discrimination problems but did not demonstrate transitive inference, indicating that they had failed to develop or could not inferentially express the orderly organization of the stimulus elements. These findings strongly support the view that the hippocampus mediates a general declarative memory capacity in animals, as it does in humans. Episodic memory is widely conceived as a fundamentally constructive, rather than reproductive, process that is prone to various kinds of errors and illusions. With a view towards examining the functions served by a constructive episodic memory system, we consider recent neuropsychological and neuroimaging studies indicating that some types of memory distortions reflect the operation of adaptive processes. An important function of a constructive episodic memory is to allow individuals to simulate or imagine future episodes, happenings and scenarios. Since the future is not an exact repetition of the past, simulation of future episodes requires a system that can draw on the past in a manner that flexibly extracts and recombines elements of previous experiences. Consistent with this constructive episodic simulation hypothesis, we consider cognitive, neuropsychological and neuroimaging evidence showing that there is considerable overlap in the psychological and neural processes involved in remembering the past and imagining the future. SUMMARY The acquisition of new memories for places and events requires synaptic plasticity in the hippocampus,andplasticitydependsontemporalcoordination among neurons. Spatial activity in the hippocampus is relatively disorganizedduring the initial exploration of a novel environment, however, and it is unclear howneuralactivityduringtheinitialstagesoflearning drives synaptic plasticity. Here we show that pairs of CA1 cells that represent overlapping novel locations are initially more coactive and more precisely coordinatedthan arecells representing overlappingfamiliar locations. This increased coordination occurrs specifically during brief, high-frequency events (HFEs) in the local field potential that are similar to ripples and is not associated with better coordination of place-specific neural activity outside of HFEs. As novel locations become more familiar, correlations betweencellpairsdecrease.Thus,hippocampalneural activity during learning has a unique structure that iswellsuitedtoinducesynapticplasticityandtoallow for rapid storage of new memories.