article: A long-standing problem is how memories can be stored for very long times despite the volatility of the underlying neural substrate, most notably the high turnover of dendritic spines and synapses. To address this problem, here we are using a generic and simple probabilistic model for the creation and removal of synapses. We show that information can be stored for several months when utilizing the intrinsic dynamics of multi-synapse connections. In such systems, single synapses can still show high turnover, which enables fast learning of new information, but this will not perturb prior stored information (slow forgetting), which is represented by the compound state of the connections. The model matches the time course of recent experimental spine data during learning and memory in mice supporting the assumption of multi-synapse connections as the basis for long-term storage.
  Learning and memory in neuronal networks is commonly attributed to changes in the connectivity, i. e. the number of synapses between neurons and their transmission efficacy [1–3], which are then consolidated to form long lasting memories [2,4–6]. The majority of the excitatory cortical synapses resides on dendritic spines [7]. Thus, very likely they contribute much to these phenomena. One central problem here is the apparent contradiction between the facts that memories are often stable over very long times, while the underlying neural substrate—most notably the synaptic spine structure—undergoes constant changes. Experiments reveal that even under control conditions, more than 5% of the spines are exchanged every day [8–13]. This turnover increases during learning (e. g, motor learning [11,12]), sensory deprivation [13] or environmental enrichment [12]. Intriguingly, although learning triggers increased turnover, repeated learning of the same task—even after long intervals—does not. The missing need for restructuring during the repeated training has been attributed to structural changes that had persisted from the previous training, and this, in turn, has been interpreted as a long-lasting memory trace [11–13]. This trace can be acquired during a few days of training, but maintained for several months, which indicates different time scales for learning and forgetting. However, the majority of the spines formed during learning vanishes a few days after training and only very small proportions may persist for the survival duration of the memory trace. Thus, it seems more reasonable that the collective properties of the pool of synapses, rather then any momentarily existing connectivity configuration, have to account for learning and memory formation. Several such properties have been found, such as the abundance of two- or three-neuron microcircuits [14,15] or the characteristic distribution of the number of synapses between two neurons, which shows a large peak at zero synapses and another smaller peak at multiple synapses [16–20]. Theoretical results suggest that such bimodal distributions emerge from the collective dynamics of multiple synapses [21]. One possibility to obtain this collective dynamics is to establish a positive feedback between the number of synapses and synapse stability. For this, however, each synapse has to sense the number of synapses from the same presynaptic neuron. It has been shown theoretically that the postsynaptic activity, which can be accessed by each synapse, provides sufficient information [22,23]. However, only a complex non-linear interaction between structural and synaptic plasticity, i. e. between the number of synapses, the postsynaptic activity, the synaptic weights, and synapse stability, gives rise to the observed bimodal distributions [22,23]. For this, synaptic plasticity rules must be sensitive to either correlations of pre- and postsynaptic firing [22] or to the postsynaptic firing rates [23]. A systematic variation of the neural activity in such systems reveals that the bimodal distribution emerges from a hysteresis in the creation and deletion of synapses [23]. The existence of such a hysteresis could be used to store information [23–29]. Considering these results, here, we abstract this collective dynamics by using a theoretical model in which the stability of synapses directly depends on the actual number of synapses. We demonstrate that storing information in the collective dynamics of multiple synapses can resolve the contradiction between memory duration and structural/synaptic volatility. Furthermore, we show that especially bimodal synapse distributions correspond to a bistability in the number of synapses which enables prolonged information storage. These bistable systems allow fluctuations of the actual number of synapses within certain limits which keeps the structure flexible and leads to the intriguing additional effect that learning is orders of magnitude faster than forgetting. Recent experimental data on the dynamics of spines [11] can, therefore, naturally be explained by this model.
  First we simulated two groups of 500 compound-connections with deletion probabilities which enforce the development into the working-point-distribution. Hence, this corresponds to medium activities at all connections. However, we initialized both groups differently: the synapse distribution of the first group is from the beginning in the wp-distribution (Fig 2A), while in the second group all compound-connections initially consist of seven realized synapses (peak at S = 7; Fig 2B). The first group demonstrates the validity of the model setup as the system indeed remains stationary in the bimodal wp-distribution (Fig 2A). The development of the second group (Fig 2B) reveals an interesting feature of the model dynamics: first, the compound-connections tend to delete synapses and, thus, the average number of synapses reduces from the initial value (S = 7) to the mean of the upper peak in the wp-distribution (μ = 5). In parallel the variance increases, thus, different compound-connections consist of different numbers of synapses. Later, although the probability distribution of the whole group of connections remains rather constant, the number of synapses of each single compound-connection varies frequently (see purple line for one example). Furthermore, even if during development the total number of realized synapses is the same at two different time points (e. g. , S (t1) = S (t2) = 5), the location of these realized synapses could be different (Fig 2B, insets). However, the connection remains in the upper peak of the wp-distribution and fluctuates around the mean μ. Finally, only after a long period (note the logarithmic time scale) many compound-connections delete all their synapses and a second peak at zero synapses emerges in the probability distribution. Even then—after reaching the wp-stationary distribution—the number of synapses at any given compound-connection fluctuates as can be seen, for instance, in the figure when the pink curve jumps between zero and six realized synapses. Such fluctuations within the stationary distribution (also in Fig 2A) represent the experimentally measured turnover of dendritic spines hosting a synapse [9]. Based on these results, we hypothesize that for long time scales the bistable dynamics of the number of synapses and the resulting probability distribution stores information rather than the existence of single synapses. This will be investigated next. In the second step, we investigate the capacity of the system to store information. For this, we evaluate how much information the number of synapses S (t) at time point t provides about the initial number of synapses S (0) using mutual information. Mutual information is an information theoretic measure, which determines how much of the variability of one quantity can be predicted by another one. Here, we calculate the mutual information between S (0) and S (t) (Fig 3) for different initial probability distributions (Fig 3A1–3A3) and stimulation-conditions resulting in different stationary distributions (Fig 3B4,3C4 and 3D4). Note, although we use a population of connections to evaluate mutual information, this information is stored by each connection on its own as connections have no means of interacting in the model. The use of a population is necessary to estimate the time evolution of the distribution p[S] which describes the number of synapses on a single connection as a random variable. To apply mutual information also the initial condition is treated as a random variable which follows a given probability distribution, although each distribution has only one initial condition and populations with different initial conditions are separately tracked (see Models and Methods). Here, we used three different initial conditions for the starting distributions: (1) the wp-distribution (Fig 3A1); (2) a two peaked distribution with probabilities p[S = 0] = 1 − C and p[S = 7] = C (Fig 3A2); as well as (3) a distribution which is uniform in each peak (Fig 3A3) where the summed probabilities are 1 − C for the lower peak (S ∈ 0,1, 2) and C for the upper peak (S > 3). Using these three initial conditions, the three stimulation-conditions from above (see Fig 1B) were investigated. Each condition yields after convergence a different stationary distribution (see Models and Methods for more details): (i) low induces a single peaked Poisson distribution (Fig 3B), (ii) wp yields the working-point-distribution (Fig 3C), and (iii) high leads to a Gaussian distribution (Fig 3D). For low and high the mutual information and, therefore, the information about the initial distribution, decays after a short period to zero (107 to 108 time steps; Fig 3B and 3D). This period corresponds to the time scale of the synapse-formation probability rate b = 10−8/time step, which is the intrinsic time scale of a compound-connection. Thus, information cannot be stored longer than the average duration of a single synapse turnover. For the wp-condition, this decay also occurs after the same period (Fig 3C). However, in contrast to all other cases, information does not decay to zero but converges for a longer time to a constant non-zero value. After another long period, mutual information starts again to decay until it finally reaches zero and all information about the initial number of synapses is ‘forgotten’. The period between the first and the second decay provides a plateau indicative of prolonged information storage which will be further investigated next. Up to now, we have analyzed how information decays in a system of compound-connections with stochastic synapses. Hence, we had investigated storage time and information loss. On the other hand, so far, it is unclear how this information can be encoded into the connectivity of such systems. This can be achieved through learning as discussed next. First, before learning, the system has reached its wp-stationary distribution (Fig 5A, left) corresponding to the steady state under resting state activity. Then, learning is triggered where a changed stimulation would elicit higher activities at some connections and lower activity at others, e. g. due to increased excitation and inhibition. This is modeled by applying either high- or low-stimulation, or—as a control case—the system just stays in the wp-condition (Fig 5A, center). Here, the learning phase is always long enough such that the new stationary distribution is reached (Fig 5B, green color code). Afterwards, learning stops and the system reverts again to the wp-condition (retention phase; Fig 5A, right). The two different “learning” stimulations (low, high) induce development of the probability distribution p[S (t) ] to their characteristic distribution (green phase in Fig 5B1 and 5B3), while for the wp-condition—as expected—the wp-distribution remains roughly constant (Fig 5B2). Remarkably, in the high- or low-stimulation case the system reaches the stationary distribution during the learning phase about 2-3 orders of magnitude earlier than during the retention phase (red phase). We quantify this effect by assessing how much information the number of synapses S (t) at time t shares with the stimulation case (low, high, or wp) applied to the system. Thus, we performed 1500 random draws of these three different cases from a uniform distribution with P (low) = P (wp) = P (high) = 1/3, where each drawn stimulation case was used to drive one compound-connection. By this, the stimulation-conditions can be considered as a random variable and the mutual information between S (t) and the stimulus-condition can be calculated. Note, the actual choice of the distribution for P (low), P (wp) and P (high) does not matter and results remain qualitatively the same. During learning, mutual information grows until the number of synapses S (t) represents with a high confidence the stimulation-condition applied to the system (green line in Fig 5C). By comparing this learning curve with the retention curve (red line), we see that the learning curve has reached its maximum before the retention curve starts decaying. In other words, information is indeed preserved several orders of magnitude longer than the duration the system needs to store it. Similar to the decay time scale (Fig 4C) the difference in time scales between learning and forgetting should also depend on the shape of the finally reached stationary distribution. To quantify this, we determined the time scales τ0. 50 at which the mutual information between the number of synapses and the stimulation-conditions reaches 50% of its maximal value in the learning and in the retention phase. The difference between these time scales ΔT can be determined by the ratio between τ0. 50 (retention) and τ0. 50 (learning) (Fig 5D). As observed for the decay time scale τ0. 95 (Fig 4C), the ratio increases if the two peaks of the bimodal stationary distribution are narrower (small λ and σ) and further apart (larger μ). Also in this case the weighting C between the peaks has only a minor influence. The difference in time scales of learning and forgetting can even be observed for learning with single peaked distributions which are piecewise uniform, i. e. for the broadest or sharpest possible peaks (see S1 Text). This again highlights that the presented results generalise to a much broader class of distributions. In conclusion, the interactions between multiple synapses in a compound-connection lead to a fast encoding of information, which is then preserved for longer periods. In the final step we will compare these dynamics with data from in-vivo experiments [11]. A similar difference of time scales between learning and forgetting is also observed in experiments investigating the dynamics of dendritic spines; the morphological correlates of excitatory synapses in cortex. For example a few days of training, sensory enrichment, or deprivation can influence the spine turnover several months later [11–13]. As the dynamics of the synapses in the model are comparable to spine dynamics in experiments, next we will reproduce the experimental data from the motor learning tasks of Xu and co-workers [11]. In these experiments, animals were selectively trained in an early (postnatal month one) and/or a late motor training phase (month four). Some animals were only trained during the late training phase (late-only paradigm; Fig 6B), some were trained with the same task during early and late training phase (retraining paradigm; Fig 6C) and some did not receive any training at all (control paradigm; Fig 6A). For all groups the number of created and removed spines had been tracked. When comparing changes during the early training phase, motor training increases spine creation as well as removal compared to the control paradigm (Fig 6D, red bars). During the late training phase (Fig 6E) a similar increase in spine turnover emerges in the late-only paradigm (compare red bars in Fig 6E right panels with Fig 6D right panels). However, animals in the retraining paradigm, which had already experienced training before, show in this case spine creation and removal at control levels (compare red bars in Fig 6E left with center panels), which indicates that the early training has left a long-lasting trace in the connectivity [11–13]. In our model these experimental paradigms can be reproduced. For this, again, we take the wp-condition as the basic distribution and emulate the process of motor learning by exposing very small sub-populations of compound-connections to high- or low-stimulation-conditions. For the control case, connections are left in the wp-condition during the whole experiment (Fig 6A). This case was used to adjust the time scale of the model. For this, we considered the complete temporal characteristic of the experimental control case for spine creation during early training phase (red bars in Fig 6D, top-left) and compared it to the model dynamics to match the time scales (see Models and Methods for details). For the retraining paradigm (Fig 6B) we start the system in the wp-condition for 30 days. Then, for the next 16 days, we define the early phase by dividing the population into 4. 35% of compound-connections receiving low- and 0. 65% receiving high-stimulation. The rest does not receive a learning signal and stays in the wp-condition. After this, the learning signal is withdrawn and the system and all connections are in the wp-condition for another 74 days (similar to Fig 5A). Finally, we perform a second (late) phase of learning for 8 days with the same changes applied to the same sub-populations as during the early phase. Note, the rather small changes of only 5% of the compound-connections applied here correspond to the sparse changes observed in experiments [11,12]. For the late-only paradigm (Fig 6C) we maintain the wp-condition for 120 days and only then perform, for 8 days, the same subdivision of the population as above (4. 35% low, 95% wp, and 0. 65% high). From these simulations, we determined the realized synapses before and during training phases and calculated the number of created and removed synapses for each of the three different paradigms. The most interesting case concerns the late phase (Fig 6E), where the retraining paradigm had experimentally led to only a control-level rate for creating and removing synapses, indicating that a long-lasting trace might exist in the connectivity. If we compare the numbers of created and removed synapses during the late training phase in our model (blue bars, Fig 6E) with the experiments, we find that the model reproduces the data almost perfectly. As in the experiments, the synapse turnover during the late training phase in the retraining paradigm is comparable to the control levels, whereas the turnover is increased, when there is no early training. This indicates that also in our model early training leaves a structural trace in the connections (like in Fig 5), which persists until the late training. During early training (Fig 6D), model and experiment also match but experimental results show larger creation and removal rates. This is due to the fact that animals show an over-expression of dendritic spines during their early life [9,32] and accordingly also a stronger removal due to homoeostatic processes. This leads to a non-stationary trend in early development, which is not covered by the model in its current state. To explain the early training phase data completely, we would have to combine the here presented approach with homoeostatic structural plasticity [33,34], which is beyond the scope of this study. Thus, apart from such non-stationary effects, our model provides a simple explanation for long-lasting traces in the connectivity. Remarkably, these traces do not rely on the persistence of single synapses (different from [12]) or on different time scales of plastic changes during learning and forgetting but are generated by the natural dynamics of such compound-connection systems.
  Here we have demonstrated that storing information on an permanently changing substrate, like dendritic spines, can be achieved by the collective dynamics of spines (or synapses). If the dynamics renders the experimentally observed bimodal connectivity distribution, prolonged information storage about the state of the connections in the network is possible. Other (unimodal) stationary distributions do not exhibit this information storage property. Our results instead suggest that transitions to unimodal distributions might happen during learning and will lead to the desired property that learning is much faster than forgetting in such systems. Importantly these dynamics are consistent with the long-term dynamics of dendritic spines in memory related experiments as shown at the end. The problem of storing information on variable substrates is inherent to all neural structures. For example, proteins and molecules, which form the substrate for storing information at any single synapse, are sometimes completely exchanged in rather short times, while the synapse’s long-lasting properties persist [35,36]. Similar to the here presented approach, this is resolved by the collective dynamics of the involved compounds, i. e. , the dynamics of their concentrations and reaction rates. Long-term information storage is achieved if they show a bistable or hysteretic characteristic [24–29]. Thus, such systems do not rely on the existence of any single protein, analogous to the synapses in the compound-connections in our model. Such a dynamic is, however, only possible when the single entities are homogeneous, i. e. have the same properties. For multiple synapses between two neurons this might not be the case. Here, the location of synapses on the dendritic tree might influence their impact on the generation of action potentials due to dendritic processing [37] or even change their plastic behaviour [38]. In this case, it would not only matter how many synapses are realised but also which ones. Interestingly, recent experiments suggest that the spine volumes, which are related to synapse stability [39,40], are very similar for synapses between the same axon and dendrite [41], supporting the homogeneity assumption of our model. However, in general, we expect that qualitatively the here shown prolongation of storage time by the collective dynamics of multiple synapses, is unaffected by such synapse differences, as long as they also implement the experimentally measured bimodal distributions. Prolongation of storage time is, however, accompanied by a decrease in the amount of stored information. Previous estimates of the information, which can be encoded by the existence of a single synapse, predict it to be around 2-3 bits [3]. In our model, this information dissolves during the first decay phase (see Fig 3). The maximum information, which can be stored for longer periods by the collective dynamics of all potential synaptic locations at one connection is here 1 bit—connected or unconnected. This value further decreases, when these two states are not equally probable. However, encoding of information by the collective dynamics of the multiple synapses of compound-connections rescues information from fluctuations of the number of synapses. This has at least three advantages: First, information storage does not rely on the persistence of single synapses. Memories are, therefore, robust against synapse loss. Second, the lifetimes of memories will be much longer than estimated from the lifetime of a synapse (as, e. g. , in [12,13]). Third, connections can keep the same degree of plasticity (by the formation probability rate b) during learning and storage phase. This enables the system to learn very quickly when a learning signal is applied. The latter seems to contradict the suggestion that there are distinct learning- and memory-spines [12,40]. However, in the here proposed model, synaptic plasticity leads to very different dynamics for synapses in the lower as compared to the upper peak. Synapses of compound-connections in the lower peak will repeatedly appear and disappear, thus, exhibiting small synaptic weights. It is known that small weights are susceptible to large changes induced by synaptic plasticity [42]. As a consequence, the present model inherently contains a population of synapses which resembles learning spines. On the other hand, synapses of compound-connections in the upper peak create a rather stable connection relating to memory spines. This yields a second population of spines with long life times, which have been proposed to encode long-lasting memory traces. This picture is supported by the fact that the long-term dynamics of dendritic spines observed in experiments indeed match the dynamics of the here proposed compound-connections (Fig 6). Only at short time scales model and experiment correspond less well and the time course of spine creation and removal during training is faster in model than in experiments. This might be due to an increased formation rate of new filopodia or spines during periods of high activity [43,44], because this leads to an activity-dependent synapse-formation rate. In our study, we had restricted ourselves to a constant formation rate to demonstrate the effect of the bimodal wp-distribution on the time scale of information storage without having to account for interfering non-stationary effects at the same time scale. Analysis of such coupled systems, in possible future work, is very difficult and needs to build on the here presented results. Strikingly, as all time scales reported here are proportional to the formation probability b, an increase of b during the stimulation or learning phase would yield to an even faster learning and a larger difference in the time scales of learning and forgetting. In summary, the here presented model is to our knowledge the first that shows how information can be acquired and stored on the variable substrate of continuously changing spines (or synapses). To achieve this, encoding exploits the existence of multiple synapses between two neurons and their mutual interactions. This principle may lend itself helping to understand the dynamics of long-lasting memories.
  In this study we model the connection between two neurons by a number of potential synapses N and the number of realized synapses S. To mimic the turnover dynamics of dendritic spines observed in experiments, synapses are created and removed randomly. Hereby, new synapses are formed with a constant probability b at each of the potential synapses which are not realized and existing synapses are removed with a probability dℑ (S) which depends on the number of synapses and the current stimulation (see Fig 1A). The number of synapses is, thus, a random variable, which follows the dynamics of a Markov chain with the different numbers of synapses as states. The time development of the distribution of the number of synapses p[S] is then characterised by the transition matrix between the states which is given by b and dℑ (S). As long as all dℑ (S) > 0, this matrix is non-negative implying that the Markov chain is irreducible. As connections can also stay in the same state, the Markov chain is also aperiodic. For a Markov chain with these properties, the Frobenius-Perron theorem guarantees that there is a stationary probability distribution p[S], to which an ensemble of such connections will converge [22,23]. Our previous investigations of the interaction of activity and synaptic plasticity with a similar structural plasticity rule [23] showed that this stationary distribution can be controlled by applying specific stimulations to the pre- or postsynaptic neuron. In this study, we simplify these stimulations to three different stimulation conditions: a high, a low and a working-point condition. For each of those conditions, we choose a stationary distribution pℑ[S] with ℑ ∈ {high, low, wp} inspired by our previous results (see Fig 1B): The working-point distribution will take a bimodal shape like the distributions of the number of synapses which were experimentally found in cortex [16–20]. The high and the low stimulation stationary distributions follow the shape of the upper and the lower peak of these distributions respectively: p h i g h [ S ] = 1 N · exp (- (S - μ) 2 σ 2) with N = ∑ S = 0 N exp (- (S - μ) 2 σ 2) (1) p l o w [ S ] = 1 M λ S S! with M = ∑ S = 0 N λ S S! (2) with μ and σ as the location and width of the upper peak and λ adjusting the steepness of the decay in the lower peak. The working-point distribution is then given by a mixture: p w p [ S ] = (1 - C) · p l o w [ S ] + C · p h i g h [ S ] (3) weighted by the connection probability C (see Fig 1D). To construct connections which have the given stationary distributions, we have to determine suitable deletion probabilities dℑ (S). For this, we use an approximation of the stationary distribution in the described system: When we only allow for creating or removing one synapse at a time (first step approximation [22,23]), the transition matrix of the system is (1 - N b N b 0 0. . . . . . 0 d [ 1 ] 1 - d [ 1 ] - (N - 1) b (N - 1) b 0. . . . . . 0 0 2 d [ 2 ] 1 - 2 d [ 2 ] - (N - 2) b (N - 2) b. . . . . . 0. . . . . . . . . . . . . . . . . . . . . 0 0 0 0. . . N d [ N ] 1 - N d [ N ]). In equilibrium, the probability flow between states S − 1 and S must be equal (detailed balance) and the stationary distribution pℑ[S] fulfils p I [ S ] p I [ S - 1 ] = (N - S + 1) b S d I [ S ]. Thus, we choose the deletion probabilities as d I [ S ] = (N - S + 1) S p I [ S - 1 ] p I [ S ] b (4) for each stimulation condition. As, by construction, pℑ[S] > 0, this also yields dℑ[S] > 0. As described above, this ensures the convergence to the stationary distributions from any initial condition. To quantify shared information between two random variables and their probability distributions, the information theoretic concept of mutual information can be used. Here, we use it for two different analyses:
summary: It is widely believed that information is stored in the connectivity, i. e. the synapses of neural networks. Yet, the morphological correlates of excitatory synapses, the dendritic spines, have been found to undergo a remarkable turnover on daily basis. This poses the question, how information can be retained on such a variable substrate. In this study, using connections with multiple synapses, we show that connections which follow the experimentally measured bimodal distribution in the number of synapses can store information orders of magnitude longer than the lifetime of a single synapse. This is a consequence of the underlying bistable collective dynamic of multiple synapses: Single synapses can appear and disappear without disturbing the memory as a whole. Furthermore, increasing or decreasing neural activity changes the distribution of the number of synapses of multi-synaptic connections such that only one of the peaks remains. This leads to a desirable property: information about these altered activities can be stored much faster than it is forgotten. Remarkably, the resulting model dynamics match recent experimental data investigating the long-term effect of learning on the dynamics of dendritic spines.
title: Formation and Maintenance of Robust Long-Term Information Storage in the Presence of Synaptic Turnover