We present a model for the self-organized formation of place cells, head-direction cells, and spatial-view cells in the hippocampal formation based on unsupervised learning on quasi-natural visual stimuli. The model comprises a hierarchy of Slow Feature Analysis (SFA) nodes, which were recently shown to reproduce many properties of complex cells in the early visual system [1]. The system extracts a distributed grid-like representation of position and orientation, which is transcoded into a localized place-field, head-direction, or view representation, by sparse coding. The type of cells that develops depends solely on the relevant input statistics, i. e. , the movement pattern of the simulated animal. The numerical simulations are complemented by a mathematical analysis that allows us to accurately predict the output of the top SFA layer.
The brain needs to extract behaviorally relevant information from sensory inputs in order to successfully interact with the environment. Position and head direction of an animal in the space surrounding it is part of this relevant information. Neural representations of a rodent' s spatial position—termed place cells—were found more than 35 years ago in hippocampal areas CA1 and CA3 [2], correlates of head orientation—termed head-direction cells—were found twenty years later [3], and recently nonlocalized representations—termed grid cells—were found in entorhinal cortex (EC) of rats [4]. Primates possibly also have place cells, certainly head-direction cells, and also spatial-view cells that do not encode the animal' s own (idiothetic) position but fire whenever the animal views a certain part of the environment [5–8]. Grid cells in primates have not yet been reported. All of these cells selectively encode some aspects of position and/or orientation of the animal, while being invariant to others. Head-direction cells are strongly selective for the direction of the animal' s head and largely invariant to its position [9]. They typically have a single peak of activity with a Gaussian or triangular shape and a tuning width of roughly 60° to 150° [10], depending on brain area. In contrast, most place cells recorded in open fields are invariant to head direction while being selective for the animal' s position. Interestingly, the degree of orientation–invariance depends on the behavioral task of the animal and possibly on the structure of the environment. In linear track environments and for repeated linear paths in open environment most place cells are orientation-specific [11]. Grid cells in EC also exhibit conjunctive representations of position and orientation [12]. Spatial-view cells in primates show very different firing properties. These cells are neither position invariant nor orientation invariant but fire when a certain part of the environment is in the animal' s field of view (FOV), resembling head-direction cells for the case of an infinitely distant view. Figure 1 illustrates the difference between grid cells, place cells, head-direction cells, and spatial-view cells. Throughout this paper, oriospatial cells will be used as a superordinate term for place cells, grid cells, head-direction cells, and spatial-view cells. While the precise role of these oriospatial cells is still discussed, they probably form the neural basis for the ability of an animal to self-localize and navigate [13]. Stimuli available to oriospatial cells can be classified as either idiothetic, including motor feedback, proprioception, and vestibular input, or as allothetic, which includes all information from sensors about the external environment, e. g. , vision or olfaction. While place cells are influenced by several modalities, they seem to be driven primarily by visual input (e. g. , [14]), but since their firing properties remain stable in the absence of external sensory cues for several minutes, idiothetic stimuli must play a major role for place-cell firing as well [15]. Using idiothetic information for navigation, which is referred to as path integration (or dead reckoning), inherently accumulates errors over longer timescales, which can only be corrected by allothetic information. For the head-direction cells it is commonly assumed that idiothetic input from the vestibular system is dominant (e. g. , [9]), but like place cells they need external sensory stimuli to correct for drift. We introduce here a model for the self-organized formation of hippocampal place cells, head-direction cells, and spatial-view cells based on unsupervised learning on quasi-natural visual stimuli. Our model has no form of memory and receives raw high-dimensional visual input. The former means that our model cannot perform path integration, the latter means that positional information has to be extracted from complex images. While such a model can certainly not be a complete model of oriospatial cells, it can show how far a memoryless purely sensory-driven system can model oriospatial cells. The learning rule of the model is based on the concept of slowness or temporal stability, which is motivated by the observation that raw sensory signals (like individual pixel values of a camera) typically vary much more quickly than some behaviorally relevant features of the animal or its environment, such as the animal' s position in space. By extracting slowly varying features from the sensory input, one can hope to obtain a useful representation of the environment. This slowness principle forms the basis for a variety of learning rules (e. g. , [16–18]). The implementation used here is Slow Feature Analysis (SFA) as introduced by Wiskott [19,20]. For a given set of time-dependent training data, in our case video sequences, we are looking for a nonlinear scalar function from a given function space that generates the slowest possible output signal y (t) when applied to the training data. The slowness of the signal is measured in terms of its Δ-value, which is given by the mean square of the signal' s temporal derivative (see the section Slow Feature Analysis). As small Δ-values correspond to slowly varying signals, the objective is to find the function that minimizes the Δ-value. To avoid the trivial constant solution, the signal is required to have unit variance and zero mean. Furthermore, we can find a second function that optimizes the objective under the additional constraint that its output signal is uncorrelated to the first, a third function, whose output is uncorrelated to the first two signals, and so on. In this manner we generate a sequence of functions with increasing Δ-value that extracts slowly varying features from the training data. More details on the approach as well as its mathematical formalization can be found in the section Slow Feature Analysis. It is important, however, to stress that SFA is not related to low-pass filtering, as the apparent paradox of slowly varying but instantaneously extracted output signals is a frequent source of misunderstandings. Low-pass filtering is a trivial way to generate slowly varying, but most often completely uninformative, outputs. Such signals cannot be instantaneous, as by definition they are generated by averaging over the past. In contrast, the representations our model finds depend on the temporal structure of sensory data during the training phase of the model, but once they are established they are instantaneous, i. e. , a single “snapshot”of sensory stimuli is sufficient to generate the model output (e. g. , a model place cell response). SFA has been successfully applied as a model for the self-organized formation of complex cell receptive fields in primary visual cortex [1]. Here, we embed this approach in a biologically inspired hierarchical network of visual processing of a simulated rat where each layer learns the slowest features from the previous layer by SFA (see the section Experimental Methods). We find that the output of the highest layer performing SFA forms a distributed oriospatial representation. In a subsequent linear step, the model applies a mechanism for sparse coding resulting in localized oriospatial codes. The same model in the same environment can reproduce the firing characteristics of place cells, head-direction cells, and spatial-view cells, depending solely on the movement statistics of the simulated rat. For roughly uncorrelated head direction and body movement, the system learns head-direction cells or place cells depending on the relative speed of head rotation and body movement. If the movement statistics is altered such that spots in the room are fixated for a while during simulated locomotion, the model learns spatial-view cell characteristics. Any computation in the brain is useless unless it leads to a change of behavior of the animal. We assume a phenomenological approach and model rat and primate oriospatial cells without asking the question what behavioral purpose these oriospatial cells serve. The last linear step of sparsification might seem irrelevant in this context; however, sparse codes have a number of advantages for subsequent processing steps that include easier decoding, energy efficiency, and, notably in the context of hippocampus, increased efficiency of memory storage in recurrent networks such as CA3 [21]. We introduce a mathematical framework in the section Theoretical Methods that analytically explains the results of the SFA output. The mathematically less inclined reader may consider skipping this section. Both analytical and computer simulation results are presented in the Results section. We conclude that a purely sensory-driven model can capture the key properties of several major cell types associated with spatial coding, namely place cells, head-direction cells, spatial-view cells, and to some extent grid cells.
SFA solves the following learning task: given a multidimensional input signal we want to find instantaneous scalar input–output functions that generate output signals that vary as slowly as possible but still carry significant information. To ensure the latter, we require the output signals to be uncorrelated and to have unit variance. In mathematical terms, this can be stated as follows. Optimization problem. Given a function space ℱ and an I-dimensional input signal x (t), find a set of J real-valued input–output functions gj (x) ∈ ℱ such that the output signals yj (t): = gj (x (t) ) under the constraints with 〈·〉t and indicating temporal averaging and the derivative of y, respectively. Equation 1 introduces the Δ-value, which is a measure of the temporal slowness of the signal yj (t). It is given by the mean square of the signal' s temporal derivative, so small Δ-values indicate slowly varying signals. The constraints (2) and (3) avoid the trivial constant solution and constraint (4) ensures that different functions gj code for different aspects of the input. It is important to note that although the objective is slowness, the functions gj are instantaneous functions of the input, so that slowness cannot be enforced by low-pass filtering. Slow output signals can only be obtained if the input signal contains slowly varying features that can be extracted instantaneously by the functions gj. In the computationally relevant case where ℱ is finite-dimensional, the solution to the optimization problem can be found by means of SFA [1,20]. This algorithm, which is based on an eigenvector approach, is guaranteed to find the global optimum. Biologically more plausible learning rules for the optimization problem, both for graded response and spiking units, exist [22,23]. If the function space is infinite-dimensional, the problem requires variational calculus and will in general be difficult to solve. In the section The modified optimization problem, we demonstrate that the optimization problem for the high-dimensional visual input, as faced by the hierarchical model, can be reformulated for the low-dimensional configural input of position and orientation. In this case, the variational calculus approach becomes tractable and allows us to make analytical predictions for the behavior of the full model. The outcome of an unsupervised learning rule, such as SFA, is crucially determined by the statistics of the training data. As we want to show that oriospatial cells can be learned from raw sensory stimuli, we approximate the retinal stimuli of a rat by video sequences generated in a virtual-reality environment. The input statistics of the training data are thus jointly determined by the structure of the virtual-reality environment and the movement pattern of the simulated rat. As this video data is very high-dimensional, nonlinear SFA in a single step is computationally unfeasible. To overcome this problem, the model is organized as a hierarchy of SFA nodes in analogy to the hierarchy of the brain' s visual system (see Figure 2C). Considering the complexity of the computational model presented in the last section, one might expect that it would be impossible to make any analytical statement about the model' s behavior. However, in this section we introduce a mathematical framework that actually allows us to make detailed predictions depending on the movement statistics of the simulated rat. The theoretically less inclined reader should feel free to skip all sections marked by a * without loss of the general understanding of our model and the results.
One of the most common environments for place-cell experiments is an open-field apparatus of rectangular or circular shape. Here, the most typical experimental paradigm is to throw food pellets randomly into the apparatus at regular intervals, leading to a random search behavior of the rat. For this case, the rat' s oriospatial configuration space comprises the full three-dimensional manifold of position and orientation. In this section, we present results from experiments with simulated rat trajectories at either high or low relative rotational speeds leading to undirected place cells or position invariant head-direction cell-type results, respectively. In a linear track, the rat' s movement is essentially restricted to two degrees of freedom, a spatial one and an orientational one. In experimental measurements, the orientational dimension is often collapsed into a binary variable indicating only the direction of movement. In the linear track, these two dimensions are thus experimentally much easier to sample smoothly than the full three-dimensional parameter space of the open field. Although most of the parameters in our model (i. e. , all the weights in the SFA and ICA steps) are learned in an unsupervised manner, a number of parameters were chosen by hand. These parameters include the input picture size, receptive field sizes, receptive field positions, and overlaps in all layers, the room shape, and textures, the expansion function space, number of layers, choice of sparsification algorithm, movement pattern, FOV, and number of training steps. We cannot explore the entire parameter space here and show instead that the model performance is very robust with respect to most of these parameters. The fact that the simulation results presented are very similar to the analytical solutions also indicates that the results presented are generic and not a mere artifact of a specific parameter set. The most interesting parameters are discussed in the following.
According to Redish' s classification, our model is a local view model, for it “only depends on the local view to explain place- cell firing” [41]. Models of this class usually extract a number of features from sensory inputs in order to obtain a lower-dimensional representation that still carries information about spatial position in the environment but is invariant to everything else. Pure local view models do not comprise a path integration system and thus cannot fully explain oriospatial firing properties, e. g. , in darkness. Pure path integration systems without external sensory input on the other hand accumulate errors, and hence a sensory coding mechanism, as proposed here, is necessary to complement any such model. Therefore, many models combine local view and path integration mechanisms [41,42], but here we focus only on local view models. The model by Wyss et al. [43] is based on similar principles as our model. It applies a learning rule based on temporal stability to natural stimuli, some of which are obtained from a robot. The resulting spatial representations are localized, resembling hippocampal place fields. The learning rule involves local memory, and no explicit sparsification method is applied. The fact that the resulting representations are localized is somewhat surprising, since by itself temporal stability does not lead to localized representations [27]. This article does not investigate the influence of movement statistics on the learned representations. The model by Sharp [44] assumes abstract sensory inputs and acquires a place code by CL, resulting in units that code for views with similar input features. Thus, this model is similar to our model' s last layer performing sparsification. Similarly to our results, the degree of head-direction invariance depends on the movement statistics. Unlike our results, however, this is not due to the temporal structure of input views but to the relative density with which orientation or position are sampled. The work by Fuhs et al. [45] uses realistic natural stimuli obtained by a robot and extracts “blobs” of uniform intensity with rectangular or oval shape from these images. Radial basis functions are tuned to blob parameters at specific views, and a CL scheme on these yields place cell-like representations. Our model agrees with their conclusion that rodents need no explicit object recognition in order to extract spatial information from natural visual stimuli. The model by Brunel and Trullier [46] investigates the head-direction dependency of simulated place fields using abstract local views as inputs. A recurrent network learns with an unsupervised Hebbian rule to associate local views with each other, so that their intrinsically directional place cells can become head-direction–invariant for maze positions with many rotations. The article also conjectures that movement patterns determine head-direction dependence of place cells, which is consistent with our results. The results by de Araujo et al. [47] suggest that the size of the rat' s FOV is important for the distinction between spatial-view cells and place cells. With a large FOV (as for rats), the animal can see most landmarks from all orientations, while an animal with a small FOV (like a monkey) can only see a subset of all landmarks at each timepoint. We find no dependence of our results on the FOV size for values between 60° and 320° as long as the environment is rich enough (e. g. , diverse textures, not a single cue card). Instead, our results suggest that differences in the movement statistics play a key role for establishing this difference. To our knowledge, no prior model allows the learning of place cells, head-direction cells, and spatial-view cells with the same learning rule. Furthermore there are only a few models that allow clear theoretical predictions, learn oriospatial cells from (quasi) natural stimuli, and are based on a learning rule that is also known to model early visual processing well. Our simulated visual stimuli come from a virtual reality environment which is completely static during the training of the virtual rat. In this case, the slowest features are position, orientation, or view direction, as shown before. However, the assumption that the environment remains unchanged during oriospatial cell learning certainly does not hold for the real world. A more realistic environment will include other changing variables such as lighting direction, pitch and roll of the head, etc. The impact of these variables on the model representations depends on the timescale on which the variables change. For instance, the additional white noise in all SFA layers of the model is ignored since it varies much more quickly than position and orientation, but the direction of sunlight might become the slowest feature. Generally, the SFA solutions will depend on any variable whose timescale is equal to or slower than the position and orientation of the animal. After the sparse coding step, representations become not only localized in position and/or head direction but in the other variables as well. This behavior is not consistent with the definition of an ideal place or head-direction cell. However, many experiments show correlations of place-cell firing with nonspatial variables as well [41]. One particularly interesting instance of such a variable is “room identity. ” If a rat experiences multiple environments, usually transitions between these will be seldom, i. e. , the rat will more often turn and traverse a single room rather than switch rooms. In this case, room identity is encoded by the SFA outputs (unpublished data). For n rooms at most (n – 1) decorrelated SFA outputs can code for the room identity. The following outputs will then code for a joint representation of space and room identity. After sparse coding, many output units will fire in one room only (the less sparse ones in few rooms), and possibly in a completely unrelated fashion to their spatial firing patterns in another room. This behavior is consistent with the “remapping” phenomenon in place cells (e. g. , [48]). A great amount of work has been done investigating the impact of environmental manipulations on oriospatial cell firing in known rooms, e. g. , shifts and rotations of landmarks relative to each other [41]. How would our model behave after such changes to the learned environment? Such transformations effectively lead to visual input stimuli outside the set of all possible views in the training environment. In this case, we expect the system' s performance to deteriorate unless a new representation is learned, but more work is necessary to investigate this question. Our approach predicts increasing slowness (i. e. , decreasing Δ-values of firing rates) in the processing hierarchy between retina and hippocampus. Additionally, place cell and head-direction cell output should be significantly sparser than their inputs. Our main prediction is that changing movement statistics directly influences the invariance properties of oriospatial cells. For instance, an experiment in a linear track where the rat more often turns on mid-track should yield less head-direction–dependent place cells. Our model is not limited to processing visual stimuli, as presented here, but can integrate other modalities as well. The integration of olfactory cues, for example, might lead to even more accurate representations and possibly to an independence of the model of visual stimuli (simulated darkness). Experimentally, the joint positional and orientational dependence of oriospatial cells is hard to measure due to the size of the three-dimensional parameter space, and even more so if the development over time is to be measured. Furthermore, precise data on movement trajectories is rare in the existing literature on oriospatial cells. Accordingly, little data is available to verify or falsify our prediction of how the brain' s oriospatial codes depend on the movement statistics. As an alternative to determining the movement statistics in behavioral tasks, some work has been done on passive movement of rats, where the movement statistics is completely controlled by the experimenter (e. g. [49]), but these results might not be representative for voluntary motion [50]. Markus et al. find directional place fields in the center of a plus maze, although more rotations occur in the center of the maze than in the arms [11]. This could be a contradiction to our model, although the relative speed (which was not measured in [11]) not the frequency determines head-direction invariance in our model. Overall, the dependence of oriospatial cells on the animal' s movement statistics as proposed here remains to be tested experimentally. We conclude that a purely sensory-driven unsupervised system can reproduce many properties of oriospatial cells in the rodent brain, including place cells, head-direction cells, spatial-view cells, and to some extent even grid cells. These different cell types can be modeled with the same system, and the output characteristics depend solely on the movement statistics of the virtual rat. Furthermore, we showed that the integration of vestibular acceleration information can be used to learn place cells and head-direction cells with the same movement statistics and thus at the same time.
Rats excel at navigating through complex environments. In order to find their way, they need to answer two basic questions. Where am I? In which direction am I heading? As the brain has no direct access to information about its position in space, it has to rely on sensory signals—from eyes and ears for example—to answer these questions. Information about its position and orientation is typically present in the information it gathers from its senses, but unfortunately it is encoded in a way that is not obvious to decode. Three major types of cells in the brain whose firing directly reflects spatial information are place, head-direction, and view cells. Place cells, for example, fire when the animal is at a particular location independent of the direction the animal is looking in. In this study, we present a self-organizational model that develops these three representation types by learning on naturalistic videos mimicking the visual input of a rat. Although the model works on complex visual stimuli, a rigorous mathematical description of the system is given as well.
primates
neuroscience
rattus (rat)
computational biology
Slowness and Sparseness Lead to Place, Head-Direction, and Spatial-View Cells