The interplay between hippocampus and prefrontal cortex (PFC) is fundamental to spatial cognition. Complementing hippocampal place coding, prefrontal representations provide more abstract and hierarchically organized memories suitable for decision making. We model a prefrontal network mediating distributed information processing for spatial learning and action planning. Specific connectivity and synaptic adaptation principles shape the recurrent dynamics of the network arranged in cortical minicolumns. We show how the PFC columnar organization is suitable for learning sparse topological-metrical representations from redundant hippocampal inputs. The recurrent nature of the network supports multilevel spatial processing, allowing structural features of the environment to be encoded. An activation diffusion mechanism spreads the neural activity through the column population leading to trajectory planning. The model provides a functional framework for interpreting the activity of PFC neurons recorded during navigation tasks. We illustrate the link from single unit activity to behavioral responses. The results suggest plausible neural mechanisms subserving the cognitive “insight” capability originally attributed to rodents by Tolman & Honzik. Our time course analysis of neural responses shows how the interaction between hippocampus and PFC can yield the encoding of manifold information pertinent to spatial planning, including prospective coding and distance-to-goal correlates.
Spatial cognition requires long-term neural representations of the spatiotemporal properties of the environment [1]. These representations are encoded in terms of multimodal descriptions of the animal-environment interaction during active exploration. Exploiting these contextual representations (e. g. through reward-based learning) can produce goal-oriented behavior under different environmental conditions and across subsequent visits to the environment. The complexity of the learned neural representations has to be adapted to the complexity of the spatial task and, consequently, to the flexibility of the navigation strategies used to solve it [2], [3]. Spatial navigation planning —defined here as the ability to mentally evaluate alternative sequences of actions to infer optimal trajectories to a goal— is among the most flexible navigation strategies [3]. It can enable animals to solve hidden-goal tasks even in the presence of dynamically blocked pathways (e. g. detour navigation tasks, [4]). Experimental and theoretical works have identified three main types of representations suitable for spatial navigation planning, namely route-based, topological, and metrical maps [2], [3], [5]–[7]. Route-based representations encode sequences of place-action-place associations independently from each other, which does not guarantee optimal goal-oriented behavior (e. g. in terms of capability of either finding the shortest pathway or solving detour tasks). Topological maps merge routes into a common goal-independent representation that can be understood as a graph whose nodes and edges encode spatial locations and their connectivity relations, respectively [2]. Topological maps provide compact representations that can generate coarse spatial codes suitable to support navigation planning in complex environments. Metrics-based maps go beyond pure topology in the sense they embed the metrical relations between environmental places and/or cues —i. e. their distances and angles— within an allocentric (i. e. world centered) reference frame [5]. Here, we model a spatial memory system that primarily learns topological maps. In addition, the resultant representation also encodes directional-related information, allowing some geometrical regularities of the environment to be captured. The encoding of metric information favors the computation of novel pathways (e. g. shortcuts) even through unvisited regions of the environment. In contrast to the qualitative but operational space code provided by topological maps, metrical representations form more precise descriptions of the environment that are available only at specific locations until the environment has been extensively explored [5]. However, purely metric representations are prone to errors affecting distance and angle estimations (e. g. path integration [8]). Behavioral and neurophysiological data suggest the coexistence of multiple memory systems that, by being instrumental in the encoding of routes, topological maps and metrical information, cooperate to subserve goal-oriented navigation planning [9]. An important question is how these representations can be encoded by neural populations within the brain. Similar to other high-level functions, spatial cognition involves parallel information processing mediated by a network of brain structures that interact to promote effective spatial behavior [3], [9]–[11]. An extensive body of experimental work has investigated the neural bases of spatial cognition, and a significant amount of evidence points towards a prominent role of the hippocampal formation [12]. This limbic region has been thought to mediate spatial learning functions ever since location-selective neurons —namely hippocampal place cells [1], and entorhinal grid cells [13]— and orientation-selective neurons —namely head-direction cells [14]— were observed by means of electrophysiological recordings from freely moving rats. Yet, the role of the hippocampal formation in goal representation and reward-dependent navigation planning remains unclear [15]. On the one hand, the hippocampus has been proposed to encode topological-like representations suitable for action sequence learning [16] (see [15] for a review of models). This hypothesis mainly relies on the recurrent dynamics generated by the CA3 collaterals of the hippocampus [17]. On the other hand, the hippocampal space code is likely to be highly redundant and distributed [18], which does not seem adequate for learning compact topological representations of high-dimensional spatial contexts. Also, the experimental evidence for high-level spatial representations mediated by a network of neocortical areas (e. g. the posterior parietal cortex [19] and the prefrontal cortex [20]) suggests the existence of an extra-hippocampal action planning system shared among multiple brain regions [21], [22]. The model presented here relies on the hypothesis of a distributed spatial cognition system in which the hippocampal formation would contribute to navigation planning by conveying redundant spatial representations to higher associative areas, and a cortical network would elaborate more compact representations of the spatial context —accounting for motivation-dependent memories, action cost/risk constraints, and temporal sequences of goal-directed behavioral responses [23]. Among the cortical areas involved in map building and action planning, the prefrontal cortex (PFC) is likely to play a central role, as suggested by anatomical PFC lesion studies showing impaired navigation planning in rats [24], [25] and neuroimaging studies [26], [27]. Also, the anatomo-functional properties of the PFC seem appropriate to encode multimodal contextual memories that are not merely based on spatial correlates. The PFC receives direct projections from sub-cortical structures (e. g. the hippocampus [28], the thalamus [29], the amygdala [30] and the ventral tegmental area [31]), and indirect connections from the basal ganglia through the basal ganglia - thalamocortical loops [32]. These projections convey multidimensional information onto the PFC, including (but not limited to) emotional and motivational inputs [33], reward-dependent modulation [34], and action-related signals [32]. The PFC seems then well suited to (i) process manifold spatial information [35], (ii) encode the motivational values associated to spatiotemporal events [15], and (iii) perform supra-modal decision making [36], [37]. Also, the PFC may be involved in integrating events in the temporal domain at multiple time scales [38]. Indeed, its recurrent dynamics, regulated by the modulatory action of dopaminergic afferents, may maintain patterns of activity over long time scales [39]. Finally, the PFC is likely to be critical to detecting cross-temporal contingencies, which is relevant to the temporal organization of behavioral responses, and to the encoding of retrospective and prospective memories [38]. This article presents a neurocomputational model of the PFC columnar organization [40] and focuses on its possible role in spatial navigation planning. The cortical column model generates compact topological maps from afferent redundant spatial representations encoded by the hippocampal place cell activity patterns as modeled by Sheynikhovich et al. [41]. The model exploits the multimodal coding property offered by the possibility to refine the cortical architecture by adding a sublevel to the column, i. e. the minicolumn. It also exploits the recurrent nature of the columnar organization to learn multilevel topological maps accounting for structural regularities of the environment (such as maze alleys and arms). It shows how specific connectivity principles regulated by unsupervised Hebbian mechanisms for synaptic adaptation can mediate the learning of topological neural representations in the PFC. Then, the model uses the underlying topological maps to plan goal-directed pathways through a neural implementation of a simple breadth-first graph search mechanism called activation diffusion or spreading activation [42]–[44]. The activation diffusion process is based on the propagation of a reward-dependent signal from the goal state through the entire topological network. This propagation process enables the system to generate action sequences (i. e. trajectories) from the current position towards the goal. We show how the modeled anatomo-functional interaction between the hippocampal formation and the prefrontal cortex can enable simulated rats to learn detour navigation tasks such as Tolman & Honzik' s task [4]. The model presented here aims at shedding some light on the link between single-cell activity and behavioral responses. We perform a set of statistical and information theoretical analyses to characterize the encoding properties of hippocampal and PFC neuronal activity —in terms of both main correlates (e. g. location, distance-to-goal, and prospective coding) and functional time course changes. We interpret and validate the results of these analyses against available experimental data (e. g. extracellular electrophysiological recordings of PFC units).
Cortical maps consist of local circuits —i. e. the cortical columns [40]— that share common features in sensory, motor and associative areas, and thus reflect the modular nature of cortical organization and function [45]. Cortical columns can be divided in six main layers including: layer I, which mostly contains axons and dendrites; layers II-III, called supragranular layers, which are specialized in cortico-cortical connections to both adjacent and distant cortical zones; layer IV, which receives sensory inputs from sub-cortical structures (mainly the thalamus) or from columns of cortical areas involved in earlier stages of sensory processing; and layers V–VI, called infragranular layers, which send outputs to sub-cortical brain areas (e. g. to the striatum and the thalamus) regulating the ascending information flow through feedback connections. According to the cytoarchitectonic properties of the rat medial PFC [32], no layer IV is considered in the model of cortical column described henceforth. Neuroanatomical findings (see [45] for a review; see [46], [47] for anatomical data on rat PFC) suggest that columns can be further divided into several minicolumns, each of which consists of a population of interconnected neurons [48]. Thus, a column can be seen as an ensemble of interrelated minicolumns receiving inputs from cortical and sub-cortical areas. It processes these afferent signals and projects the responses both within and outside the cortical network. This twofold columnar organization has been suggested to subserve efficient computation and information processing [45], [49]. Several models have been proposed to study the cortical columnar architecture, from early theories on cortical organization [50]–[52] to recent computational approaches (e. g. the blue brain project [53]). These models either provide a detailed description of the intrinsic organization of the column in relation to cytological properties and cell differentiation or focus on purely functional aspects of columnar operations. The approach presented here attempts to relate the columnar organization to decision making and behavioral responses using a highly simplified neural architecture which does not account for cell diversity and biophysical properties of PFC neurons. Fig. 1A shows an overview of the model architecture based on this notion of cortical column organization. As aforementioned, the underlying hypothesis is that the PFC network may mediate a sparsification of the hippocampal place () representation to encode topological maps and subserve goal-directed action planning. The model exploits the anatomical excitatory projections from hippocampus to PFC [28] to convey the redundant state-space representation to the columnar PFC network, where a sparse state-action code is learned. Within a column, each minicolumn becomes selective to a specific state-action pair, with actions representing allocentric motion directions to perform transitions between two states. Each column is thus composed by a population of minicolumns that represent all the state-action pairs experienced by the animal at a location. This architecture is consistent with data showing that minicolumns inside a column have similar selectivity properties [54] and that some PFC units encode purely cue information while others respond to cue-response associations [55]. The model employs the excitatory collaterals between minicolumns [45], [56] to learn multilevel topological representations. Egocentric self-motion information (provided by proprioceptive inputs) biases the selectivity properties of a subpopulation of columns to capture morphological regularities of the environment. Unsupervised learning also modulates the recurrent projections between minicolumns to form forward and reverse associations between states. During planning, the spreading of a reward signal from the column selective for the goal through the entire network mediates the retrieval of goal-directed pathways. Then, a local competition between minicolumns allows the most appropriate goal-directed action to be inferred. The following sections provide a functional description of the model columnar structure, connectivity and input-output functional properties. A more comprehensive account –including equations, parameter settings and explanatory figures– can be found in Supplementary Text S1. We demonstrate the ability of the model to learn topological representations and plan goal-oriented trajectories by considering a navigation task: the Tolman & Honzik' s detour task. The behavioral responses of simulated rats are constraint by intersecting alleys, which, in contrast to open field mazes, generate clear decision points and permit dynamic blocking of goal-directed pathways.
We first examined the behavioral responses of simulated animals solving the 1∶1 version of Tolman & Honzik' s task (see Sec. sec: tolmantask and Fig. 2 for details on the experimental apparatus and protocol). The qualitative and quantitative results shown on Figs. 3A and B, respectively, demonstrate that the model reproduced the behavioral observations originally reported by Tolman & Honzik [4]. During the first 12 training trials (Day 1) the simulated animals learned the topology of the maze and planned their navigation trajectories in the absence of blocks A and B. Similar to Tolman & Honzik' s findings, the model selected the shortest pathway P1 significantly more than alternative paths P2 and P3 (ANOVA, ; Figs. 3A, B left column). During the following 156 training trials (Days 2–14), a block at location A forced the animals to update their topological maps dynamically, and plan a detour to the goal. The results reported by Tolman & Honzik provided strong evidence for a preference for the shortest detour path P2. Consistently, we observed a significantly larger number of transits through P2 compared to P3 (ANOVA, ; Figs. 3A, B central column). The simulated protocol included 7 probe trials (Day 15) during which the block A was removed whereas a block at location B was added. This manipulation aimed at testing the “insight” working hypothesis: after a first run through the shortest path P1 and after having encountered the unexpected block B, will animals try P2 (wrong behavior) or will they go directly through P3 (correct behavior)? In agreement with Tolman & Honzik' s findings, simulated animals behaved as predicted by the insight hypothesis, i. e. they tended to select the longer but effective P3 significantly more often than P2 (ANOVA, ; see Figs. 3A, B, right column). The patterns of path selection during this task is explained by the ability of the model to choose shortest paths. When a block is added into the environment, the goal propagation signal is also blocked at the level of the column network, and hence the simulated animals choose the shortest unblocked pathways. We then tested the robustness of the above behavioral results with respect to the size of the environment. We considered a 4∶1 scaled version of Tolman & Honzik' s maze and we compared the performances of simulated animals with intact populations (“control” group) against those of simulated animals lacking the cortical population (“no ” group). The latter group did not have the multilevel encoding property provided by the – recurrent dynamics (see Sec. Recurrent cortical processing for multilevel topological mapping). Fig. 3C compares the average path selection responses of the two simulated groups across the different phases of the protocol. During Day 1 (i. e. no blocks in the maze) both groups selected the shortest path P1 significantly more often (ANOVA, ; Fig. 3C left). However, the action selection policy of subjects without began to suffer from mistakes due to the enlarged environment, as suggested by lower median value corresponding to P1. During Days 2–14 (with block A), the group without did not succeed in solving the detour task, because no significant preference was observed between P2 (shortest pathway) and P3 (ANOVA, ; Fig. 3C center). By contrast, control animals coped with the larger environmental size successfully (i. e. P2 was selected significantly more often than P3, ANOVA,). During the probe trials of Day 15 (with a block at B but not at A), the group without was impaired in discriminating between P2 and P3 (ANOVA, ; Fig. 3C right), whereas control subjects behaved accordingly to the insight hypothesis (i. e. they selected the longer but effective P3 significantly more than P2; ANOVA,). The better performances of control subjects were due to the fact that back-propagating the goal signal through the cortical network benefited from the higher-level representation encoded by the population and from the - interaction during planning (see Supplementary Text S1 Sec. Exploiting the topological representation for navigation planning, Fig. S2). Indeed, an intact population allowed the goal signal to decay with a slower rate compared to, due to the smaller number of intermediate columns in (i. e. planning could benefit from a more compact topological representation). Henceforth we demonstrate how the modeled neural processes can be interpreted as elements of a functional network mediating spatial learning and decision making. We show that the neural activity patterns of all types of neurons in the cortical model are biologically plausible in the light of PFC electrophysiological data [20], [35], [59]–[66]. We studied to what extent the neural populations of the model (i. e. , , , and neurons) could be quantitatively segregated on the basis of a set of statistical measures. We then compared the results to those obtained by applying the same clustering analysis to a population of neurons recorded from the medial PFC of navigating rats (see Materials and Methods Sec. Statistical analysis of neural activities). We first gathered all non-silent simulated neurons recorded during the 4∶1 version of Tolman & Honzik' s task. All types of units (i. e. , , , ,) were pulled together in a data set. We characterized each neuron' s discharge by measuring its mean firing rate, standard deviation, skewness, lifetime kurtosis, spatial information per spike and spatial mutual information (see Supplementary Text S2). Then, we performed a principal component analysis (PCA) on the multidimensional space containing the values provided by these measures per each neuron (see Figs. S4 A, C for details). Fig. 9A shows the resulting data distribution in the space defined by the first three principal components. Interestingly, model neurons with different functional roles tended to occupy distinct regions of the PCA space. For instance, neurons, whose function in the model is to propagate goal information and code for the distance to the goal, were located within the same portion of the PCA space (blue and cyan crosses and circles). All neurons, which primarily code for spatial locations, were also clustered within the PCA space (red crosses). Neurons (and also), responsible for forward signal propagation and local decision making, respectively, were aggregated within the same region (gray and black crosses, and black circles). Finally, neurons, mainly involved in high-level mapping and navigation planning, were also separated from other units in PCA space (gray and red circles). Figs. 9B, C, D display the mean values, averaged over each population of the model, of three statistical measures (out of six) used to perform the PCA. These diagrams can help understanding the data point distribution of Fig. 9A. When considering the mean spatial information per spike (Fig. 9B), at least three groups could be observed: neurons whose activity had nearly no spatial correlate (), neurons conveying intermediate amounts of spatial information (and), and neurons with larger spatial information values (). The mean firing rate parameter (Fig. 9C) allowed two distinct groups to be clearly identified: one with low average firing (neurons), and one with high firing rates (neurons). Together with Fig. 9A, this diagram can help understanding why neurons, which had almost no spatial correlate and very high firing rates compared to other populations of the model, were well segregated within the same region of the PCA space (Fig. 9A, blue and cyan crosses and circles). Finally, when comparing the mean skewness values of all neural populations (Fig. 9D), neurons and were pulled apart, according to their distribution in the PCA space (Fig. 9A, gray and black crosses, and black circles). As a control analysis, we extended the data set used for the PCA by adding a population of neurons with random Poisson activities. As shown in supplementary Figs. S5 A–B, the population of Poisson neurons (light green data points) was well separated from all model neurons within the space defined by the first three principal components, suggesting that the variability of model discharge properties could not be merely explained by a random Poisson-like process. We then applied an unsupervised clustering algorithm (k-means clustering method with) to partition the distribution of data points of Fig. 9A, based on the discharge characteristics of model neurons. This blind clustering analysis (i. e. without any a priori knowledge on neural populations) allowed us to identify three main groups (Fig. 10A). The first cluster (blue data points) corresponded to non-spatial, reward-related neuronal activities (i. e. neurons). The second cluster (green points) represented location-selective activity (mainly from neurons, but also including some neurons). The third cluster (red data points) corresponded to location-selective activity of neurons in the cortical network (i. e. mainly). See supplementary Fig. S6 for details on the composition of the three identified clusters. We performed the same series of analyses on a dataset of medial PFC neurons recorded from navigating rats (see Materials and Methods, Sec. Statistical analysis of neural activities). We characterized every recorded activity according to the same set of statistical measures used for model neurons (i. e. mean firing rate, standard deviation, skewness, lifetime kurtosis, spatial information per spike and spatial mutual information, see Supplementary Text S2). Then, we applied a PCA on the resulting high dimensional space containing, per each neuron, the resulting values of these measures (see Figs. S4 B, D for details). Finally, we used the same unsupervised k-mean clustering algorithm to partition the data distribution in the space defined by the first three principal components. As for simulated data, the clustering method identified three main classes (Fig. 10B; with red, green, and blue data points corresponding to three subsets of electrophysiologically recorded activities in the PFC). We then compared model and experimental clusters (Figs. 10C, D, E) in order to investigate whether real and simulated data points belonging to the same clusters shared some discharge characteristics. In terms of mean spatial information (Fig. 10C), we found similar non-homogeneous distributions between model and real clusters. Both red clusters encoded the largest spatial information content. Recall that the model red cluster mainly contained activities from location-selective neurons (as quantified in supplementary Fig. S6 B). When looking at mean firing rates averaged over each cluster (Fig. 10D), we found that both real and simulated activities within the blue clusters had significantly larger frequencies than others. The model blue cluster was mainly composed by neurons propagating reward-related information. Finally, when comparing the mean absolute values of the skewness of receptive fields (Fig. 10E), we found both model and experimental populations with asymmetric fields (i. e. non-zero skewness). Model-wise, the red and green clusters (containing neurons, Fig. S6 B) had the largest mean skewness. Similarly, experimental red and green subpopulations had larger skewness values than the blue population. As a control analysis, we computed the three mentioned measures (i. e. information per spike, mean firing rate and skewness of the receptive field) for two populations of neurons with random Uniform and Poisson activities. As shown in supplementary Fig. S7, unlike model data, the two populations of random neurons could not explain the experimental data in terms of information content and skewness of the receptive field. Taken together, these results indicated that, within the data set of experimental PFC recordings, subpopulations of neurons existed with distinct discharge properties, and that these subpopulations might be related to distinct functional groups predicted by the model.
Our model is based upon three main assumptions. First, the model relies on the columnar organization of the cortex. Although this concept is supported by many experimental studies [45], [48], no clear general function for columns has emerged to explain their role in cortical processing [69]. In addition, Rakic [70] stressed that the size, cell composition, synaptic organization, expression of signaling molecules, and function of various types of columns are dramatically different across the cortex, so that the general concept of column should be employed carefully. In our model, we call “column” a local micro-circuit composed by neurons processing common spatial information, and we propose that this columnar organization may be a substrate suitable to implement a topological representation of the environment. Second, our planning model relies on an activation diffusion mechanism. At the neural level, this suggests that strong propagation of action potentials should occur in the neocortex. This is not a strong assumption, since several studies have demonstrated such phenomena as propagating waves of activity in the brain [71], [72]. For example, Rubino et al. [73] suggested that oscillations propagate as waves across the surface of the motor cortex, carrying relevant information during movement preparation and execution. Third, the multiscale representation is based on a putative signal. There are several potential candidates for its implementation in the brain. One of these candidates is habit learning involving the striatum [74], [75]. Indeed, if for instance the rat always turns left at a particular location it may acquire a corresponding habit. The neural activity corresponding to this stimulus-response association may serve as the signal. In this case, the time scale of learning in the population should correspond to the time scale of habit acquisition (potentially many trials, see e. g. [75]). Topological map learning and path planning have been extensively studied in biomimetic models (see [6] for a general review; see also [76] for theoretical discussions on hierarchical cognitive maps). These models aimed at either providing more efficient path planning algorithms or, like our work, establishing relations between anatomical substrates, electrophysiology and behavior. In particular, several studies took inspiration from the anatomical organization of the cortex and used the activation diffusion mechanism to implement planning. Burnod [49] proposed one of the first models of the cortical column architecture, called “cortical automaton”. He also described a “call tree” process that can be seen as a neuromimetic implementation of the activation diffusion principle. Some subsequent studies employed the cortical automaton concept [77], [78], while others used either connectionist architectures [16], [79]–[84] or Markov decision processes [85]. Our approach is similar to that of Hasselmo [44], who also addressed goal-directed behavior by modeling the PFC columnar structure. Both Hasselmo' s and our model architectures employ minicolumns as basic computational units to represent locations and actions, to propagate reward-dependent signals, and mediate decision making. Yet, the two models differ in the encoding principles underlying the learned representations, which generate, consequently, distinct behavioral responses. The connectivity layout of Hasselmo' s model allows state-response-state chains to be encoded, with each minicolumn representing either a state or an action. In our model, a state and its related actions are jointly encoded by a set of minicolumns within a column. Similar to Koene and Hasselmo [44], [86], we compared the discharge of simulated PFC units against experimental recordings exhibiting place-, action- and reward-related correlates. As explained henceforth, we focused further on the functional relationship between the hippocampus and the PFC in encoding complementary aspects of spatial memory, with a quantitative approach based on the analysis of statistical properties and information content of the neural codes. We also put the emphasis on the time course analysis of neural responses mediating place coding vs. decision making. The successful performance of our model in large environments relies on its ability to build a multiscale environment representation. This is in line with the proposal by McNamara et al. [87] who have suggested that humans can solve complex spatial problems by building a hierarchical cognitive map, including multiple representations of the same environment at different spatial scales. Moreover, animals may be able to chunk available information and build hierarchical representations to facilitate learning [88]–[92]. Recently, multiscale spatial representations have been identified at the neural level. For example in the entorhinal cortex, Hafting et al. [13] have shown that grid cells have spatial fields forming grids with different spacing and place field sizes. Kjelstrup et al. [93] have provided neural recordings of place cell activities in a large maze, supporting the multiscale coding property in the hippocampus. These entorhinal and hippocampal multiscale representations are likely to encode spatial contextual information at variable resolution. Complementing this code, we suggest that multiscale representations related with space, action and reward should also be found in neocortical areas such as the prefrontal cortex, commonly associated with high-level cognitive processes. Moreover, there are several works suggesting a role of the PFC in the learning of hierarchical representations. For example, Botvinick [94] reviewed how the hierarchical reinforcement learning framework [95] could explain the mechanism by which the PFC aggregates actions into reusable subroutines or skills. The multiscale property is applied there for actions instead of states as in our approach. From a biological point of view, recent studies directly pointed out the role of the PFC for hierarchical representations, with a possible anatomical mapping of the hierarchical levels along the rostro-caudal axis of the PFC [96]. In spite of a possible complementary role for the PFC and the hippocampus in multiscale space coding, our work focuses on different roles of the PFC and the hippocampus in the planning process. Namely, we propose that the hippocampus is more involved in the representation of location [1] and, possibly, route learning [97], [98], while the PFC is responsible for topological representations and action selection. From a more general perspective, a route could be seen as an example of navigation from a point to another, an episode. In contrast, the more integrated topological representation would be more similar to a set of navigation rules. This hypothesis is in accordance with data showing that the hippocampus would be involved in instance-based episodic memory, whereas the PFC would be linked to rule learning from examples [99]–[101]. Our model is consistent with recent studies suggesting a role for the PFC in prospective memory [102], [103]. Goto and Grace [102] showed that, depending on the dopamine receptors activation, PFC either incorporates retrospective information processed by the hippocampus or processes its own information to effect preparation of future actions. This is in accordance with our model which includes hippocampal information to localize itself in the environment, and then propagates reward signal to generate a goal-directed sequence of action. Moreover, Mushiake et al. [104] showed that activity in the PFC reflects multiple steps of future events in action plans. They suggested that animals may be engaged in planning sequences in a retrograde order (starting from the goal to the first motion), in conjunction with a sequence planning with an anterograde order. At the cognitive level, the activation diffusion planning process provides a capacity of mental simulation of action selection: the back-propagated goal signal allows possible navigation trajectories to be identified, whereas the forward-propagated path signal actually simulates the execution of the selected action sequence. Schacter et al. [103] recently reviewed theories on simulation of future events and neural structures associated with this cognitive ability. They showed that the same core network, which plays a role in remembering, is also implied in mental simulation. This network involves prefrontal as well as medial temporal regions including the hippocampus, which is also involved in prospective and retrospective memory coding ([105], [106]; see also [107], [108] for theoretical works modeling the role of this core network in memory retrieval and mental imagery). Our simulation results on the Tolman and Honzik detour task show that the behavior of the model is consistent with an “insight” demonstrated by rats in this task. The insight, as defined by Tolman and Honzik, is the ability to conceive that two paths have a common section, and so when a passage through the common section is blocked, both of these paths are necessarily blocked and a third, alternative pathway, should be chosen. The realization that a common section exists leads to two conclusions. First, animals do not act exclusively according to stimulus-response associations, but use some kind of mental representation of the environment [109]. For example, in the conditions of the detour task (Fig. 2A), the rats chose path 3 without actually testing path 2 during probe trials and so they did not have a chance to form the correct stimulus-response associations to solve the task. In order to choose the correct path 3, rats had to mentally replay path 2 and realize that it was blocked, suggesting the existence of a spatial representation. Second, a representation of the environment in terms of routes is not sufficient to solve the task. Indeed, if after training animals store separate representations of routes via paths 1–3, then the fact that route 1 is blocked should not lead to the conclusion that route 2 is also blocked. In summary, the results of this experiment suggest the existence of a topological graph-like representation in which common points (nodes) and common sections (edges) are identified. The model presented here proposes a plausible way of how such a representation can be built (see below). In terms of the model, the insight capability in the detour task is mediated by the propagation of the goal signal through the nodes of the spatial graph, in which the common section of paths 1 and 2 is blocked. The other important question addressed by the present study is whether the requirements of the proposed model are consistent with the neural activities observed in the PFC. We show that all types of neurons that are required by the model, have actually been observed in the PFC. Namely, (i) the state-encoding neurons in the model correspond to spatially selective prefrontal neurons with different receptive field sizes (Fig. 4D, see also [20]); (ii) the distance-to-goal, or value, neurons correspond to the PFC neurons with constant discharge rate (Fig. 6B), giving rise to the prediction that neurons with higher (constant) discharge rates can code for locations closed to reward; (iii) the prospective-coding neurons in the model correspond to PFC neurons with the firing rate that increases when the animal moves toward the goal (Figs. 8B, D, see also [62], [65]); and, finally, (iv) neurons and, which together encode state-action values, show activity patterns similar to strategy-switching neurons observed by Rich and Shapiro [37]. Indeed, the authors reported that in their task (i. e. strategy switching in a plus-maze) during the periods before and after reward contingency change, different subsets of PFC neurons were highly active. This is exactly what was observed in our model. For example, neurons and that were more active than neurons and before the contingency change (Figs. 7B, C at 4 s) became relatively less active after the change (Figs. 7B, C at 5 s). The model provided a vantage point to interpret PFC electrophysiological data in terms of quantitative clustering of population activity. On the basis of a set of statistical measures, we performed a principal component analysis on both simulated and real data sets of PFC recordings. This study gave rise to comparative results based on the identification of clusters of characteristic discharge properties. We could put forth some hypotheses about the functional meaning of the observed clusters —in terms of their role in spatial localization and planning, reward coding, and prospective memory. For instance, model neurons mediating planning in large scale mazes (i. e. belonging to the cortical population of the model) could be segregated from other simulated units (red cluster in Fig. 10). A corresponding cluster was found when analyzing real recordings, corroborating the hypothesis of the presence of neurons with similar discharge properties in the PFC. We also identified another cluster of real PFC activities which contained both pyramidal cell and interneuron responses (and, respectively). This cluster corresponded to goal propagating neurons of the model (blue cluster in Fig. 10), leading to the prediction that interneurons may contribute to decision making by participating to the propagation of information relevant to the next decision to be taken. Interestingly, in their study of spatial navigation, Benchenane et al. [60] showed that the inhibitory action of PFC interneurons onto pyramidal cells is enhanced during periods of high coherence in theta oscillations between hippocampus and PFC occurring at decision points. In this model, the simulated hippocampal population does not account for the full range of place cell firing properties that have been extensively studied during the past decades. Particularly, the dynamics of the model hippocampal population in terms of learning, extrafield firings and rhythmic discharges are not reproduced. Experimental data show that the introduction or the removal of a barrier in the environment may induce learning-related changes in the hippocampal population (remapping). For example, previously silent cells may discharge and previously active cells may be silent when the animal visits the modified environment [110], [111]. In addition, complementing their location selectivity, hippocampal place cells may have extrafield firings, and neural ensembles in the hippocampus may transiently encode paths forward of the animal [112]. Finally, it has been shown that hippocampal place cell discharges are modulated by theta oscillations (e. g. phase precession phenomena, [113]) and that the hippocampus and the PFC seem to synchronize at behaviorally relevant places in a maze, such as decision points [114]. Although the scope of the presented model is targeted to address the PFC firing patterns, these experimental data suggest that improving our hippocampal place cell model is relevant to provide plausible predictions about the interactions between the hippocampus and the PFC during decision making in spatial navigation tasks. The second limitation of the model is related to the issue of goal representation in the PFC. The model makes decisions based on an appetitive motivational signal only (i. e. the reward at the goal site). Clearly, there are other variables apart from the reward size that influence the planning process. For example, there is evidence that physical efforts required to reach the goal or delay in reward delivery influence PFC-dependent behavioral decisions [115]. Moreover, the model can merely deal with a single goal at present and can not estimate relative values of different goals [63]. In order to address these limitations, the activation diffusion mechanism in the model can be extended to propagate several motivational signals, the intensities of which are proportional to their subjective values. In this case, a goal-related effort or delayed reward can be modeled by adjusting the relative values of motivation signals at different locations in the maze. We limited our study to a structured maze (i. e. Tolman & Honzik' s maze [4]) to focus on the adaptive response to dynamic blocking of goal-directed pathways, a required property to validate detour-like navigation behavior. Furthermore, Tolman & Honzik' s maze provided us with the possibility to investigate the neural dynamics of the modeled network at clear decision points –i. e. at the intersections between corridors. Several models have addressed spatial navigation in open-field environments based on place-triggered-response strategies (i. e. locale navigation), in which hippocampal place cell activity is associated to the best local action leading to the goal (e. g. [57], [116], [117]). In fact, two components are relevant to avoid the combinatorial explosion of the space in open-fields: (i) the reliability of the spatial code in terms of minimum hidden-state probability, to avoid, for instance, that a same place cell population can code for different locations –a problem often arising from sensory-aliasing phenomena in purely topological maps; (ii) the use of a discrete action space, meaning that a finite set of actions are available at each state (location). Our hippocampal-PFC model satisfies these requirements. We already used a highly simplified version of the model presented in this paper to solve open-field navigation problems (e. g. Morris water maze [11]). Note that, however, Dollé et al. (2010) focused on navigation strategy switching and did not model the PFC columnar organization and the (possibly) involved computational processes (e. g. multiscale coding) to drive planning behavior [11]. In open-field environments with no obstacles our model predicts -like units with uniform activity across the whole space –as a result of a uniform signal reflecting equal probability of turning at each location. Adding borders or barriers would result in the “recruitment” of new units preferentially active on either one side or the other of the barriers. In more structured environments such as interconnected arenas (e. g. [15]), the model predicts separate units for each space. To our knowledge, there is no direct experimental evidence in favor or against the existence of such PFC units. Another interesting direction of future work is to study the encoding of task-related information in the PFC during sleep. Although it is likely that information is transferred during task learning, memory consolidation during sleep also appears to play a central role [59]. In particular, sharp wave-ripple complexes in the hippocampus seem prominent for transferring labile memories from the hippocampus to the neocortex for long-term storage [118]. A key issue for modeling approaches is to understand computational properties of this learning mechanism.
We study spatial cognition, a high-level brain function based upon the ability to elaborate mental representations of the environment supporting goal-oriented navigation. Spatial cognition involves parallel information processing across a distributed network of interrelated brain regions. Depending on the complexity of the spatial navigation task, different neural circuits may be primarily involved, corresponding to different behavioral strategies. Navigation planning, one of the most flexible strategies, is based on the ability to prospectively evaluate alternative sequences of actions in order to infer optimal trajectories to a goal. The hippocampal formation and the prefrontal cortex are two neural substrates likely involved in navigation planning. We adopt a computational modeling approach to show how the interactions between these two brain areas may lead to learning of topological representations suitable to mediate action planning. Our model suggests plausible neural mechanisms subserving the cognitive spatial capabilities attributed to rodents. We provide a functional framework for interpreting the activity of prefrontal and hippocampal neurons recorded during navigation tasks. Akin to integrative neuroscience approaches, we illustrate the link from single unit activity to behavioral responses while solving spatial learning tasks.
circuit models
cognitive neuroscience
cognition
computational neuroscience
decision making
neural networks
biology
computational biology
neuroscience
learning and memory
Spatial Learning and Action Planning in a Prefrontal Cortical Network
                    Model