Neural populations respond to the repeated presentations of a sensory stimulus with correlated variability. These correlations have been studied in detail, with respect to their mechanistic origin, as well as their influence on stimulus discrimination and on the performance of population codes. A number of theoretical studies have endeavored to link network architecture to the nature of the correlations in neural activity. Here, we contribute to this effort: in models of circuits of stochastic neurons, we elucidate the implications of various network architectures—recurrent connections, shared feed-forward projections, and shared gain fluctuations—on the stimulus dependence in correlations. Specifically, we derive mathematical relations that specify the dependence of population-averaged covariances on firing rates, for different network architectures. In turn, these relations can be used to analyze data on population activity. We examine recordings from neural populations in mouse auditory cortex. We find that a recurrent network model with random effective connections captures the observed statistics. Furthermore, using our circuit model, we investigate the relation between network parameters, correlations, and how well different stimuli can be discriminated from one another based on the population activity. As such, our approach allows us to relate properties of the neural circuit to information processing.
In the search for clues about the function of neural circuits, it has become customary to rely upon recordings of the activity of large populations of neurons. These measurements exhibit the concerted responses of neurons in different conditions, such as presentations of different stimuli. With the stimulus-dependent, high-dimensional statistics of neural responses in hand, one can ask two questions: How are these statistics generated in the neural population? What purpose, if any, do they serve? While the first question is mechanistic and the second is functional, the two are intimately linked. We address these questions by relating observed population activity to the output of different circuit models. In particular, we examine the correlated response variability that arises from diverse circuit architecture, and, in turn, its effect on stimulus representation. Along the mechanistic line of research, a number of network models have been proposed to explain the relation between anatomical and physiological parameters, on the one hand, and the noise correlations, on the other hand. The way in which correlations emerge, and may be suppressed, in recurrent networks was analyzed in detail with simplified biophysical models [1–4], as well as more abstracted models using binary neurons [5] or Poisson neurons [6]. As an alternative to recurrent connections, shared external input (e. g. , from top-down afferents) or shared gain fluctuations can also be at the origin of correlations in neural populations. This and similar mechanisms were exploited in recent studies [7,8], in particular to account for observations in both visual cortex [9–12] and retina [13,14]. In modeling stochastic neural activity, models of Poisson neurons or generalizations thereof have proved eminently useful. These have been employed to investigate experimentally measured statistics in the context of both feed-forward [15] and recurrent networks [16], as well as in the context of models with shared gain fluctuations [9,13]. Poisson models can capture a wide range of statistics; their parameters can be viewed as an abstraction from the biophysical properties of neurons. An important motivation to understand the origin of the statistics of neural activity is the latter’s connection with the representation of sensory information in neural populations (for a recent review, see Ref. [17]). A much studied example is that of the influence of recurrent connections on the properties of the mean response of neurons, or tuning curves [18–20]. Beyond the study of first-order statistics, a great deal of work has been devoted to the investigation of the relation between correlated variability in population response and the accuracy of stimulus representation. Several classic studies have pointed to the fact that noise correlations can be harmful to information coding [21–23], while a number of more recent studies indicate that noise correlation can also have a beneficial impact on coding, depending on its fine structure [23–26]. For example, the effect of noise correlation can depend appreciable on its stimulus dependence [13,14] or on the physiological heterogeneity present in the population [27–29]. The way in which noise correlations arise from circuit properties has been examined in simulated networks [18] and analyzed theoretically in the framework of Poisson neurons [30,31]. Recent work has also investigated the limits on the representation of information imposed by properties of neural circuits, via the structure of correlations these induce [32,33]. Here, we aim at relating possible mechanistic origins of correlation to the statistics of neural population activity, on the one hand, and at relating circuit properties to the representation of information, on the other hand. Rather than starting from a specific circuit model or relying on simulations of a certain network architecture, we want to be able to infer circuit properties of fairly general models for large and interconnected populations from observed activity. We achieve these aims by establishing mathematical expressions that relate circuit properties to population activity statistics; in particular, to variances and covariances in the population. These expressions connect the various parts of our investigation. We obtain these expressions in the framework of the Hawkes model [34], made up of coupled Poisson neurons. We examine three types of circuit architectures, namely, recurrent networks, feed-forward networks, and networks with shared gain fluctuations. Instead of focusing on specific realizations of these networks, however, we consider ensembles of such networks, defined through distributions of random connections. This allows us to identify generic signatures of circuit architectures in the correlation structures they induce, which can be used to distinguish them based on recorded activity. We then analyze experimental data, namely, populations recordings in mouse auditory cortex [35], in the light of our model results, and we show that the structure of noise correlations in auditory cortex is in agreement with predictions from a recurrent network. Finally, we turn to the relation between circuit architecture and information coding. Using the derived mathematical expressions, we analyze the effect of network parameters on the representation of information, in the case of different circuit architectures.
The data set was first reported on in Ref. [35] which discussed the structure of the average population responses as a function of stimulus. The data consists of the activity of neural populations (46-99 neurons) recorded with calcium imaging in the auditory cortex of mice. Animals were isoflurane anesthetized (1%). Signals were obtained from neurons labeled with the synthetic calcium indicator OGB1. Fluorescence was measured at 30 Hz sampling rate, and firing rates were inferred from temporal deconvolution of the fluorescence signal. Up to 6 neural populations in each of 14 animals were recorded. The data points we use are the average firing rates over a window of 250 ms after presentation of each of 65 different sound stimuli, each for 15 trials. Sound stimuli consisted in 50-70 ms presentations of pure frequency tones or complex broadband sounds generated from animal calls or musical pieces. Onset and offset were smoothed, and the stimuli were played at different volumes. Thus, the data points represent population responses evoked by a short sound presentation, time-averaged over a 250 ms window. Responses were measured relative to spontaneous activity; hence, negative responses occurred if the stimulus-evoked firing rates were smaller than the spontaneous ones. While the precise timing of spikes can convey a large amount of information [36], in many studies this temporal dimension is neglected, and the representation of stimuli is considered in terms of spike counts in given time windows. This approach is legitimate in cases in which sensory coding occurs on a relatively slow time scale; here, we also assume this form of spike count coding. We denote the vector of population activity in trial T of stimulus s is by r (s, T), and the average response across trials for this stimulus by r (s) ≡ 〈r (s, T) 〉T. If not explicitly stated otherwise, here, and throughout, we consider time-averaged neural responses, both when referring to experimental data and in the theoretical analysis; i. e. , the vector r (s, T) represents the neural activity averaged across a chosen time window. To measure covariability between neurons i and j across trials, in response to a given stimulus, we use the noise covariances, defined as C i j (s) = cov (r i (s, T), r j (s, T) ) T = ⟨ r i (s, T) r j (s, T) ⟩ T - ⟨ r i (s, T) ⟩ T ⟨ r j (s, t) ⟩ T. (1) A measure of the strength of pairwise noise correlations across stimuli is the average correlation coefficient, defined as c i j N = 〈 C i j (s) C i i (s) C j j (s) 〉 s, (2) where 〈. 〉s denotes the average over all stimuli presented. This quantity is to be contrasted with the signal correlation, c i j S = cov (r i (s), r j (s) ) s var (r i (s) ) s var (r j (s) ) s, (3) which measures the similarity of average responses across stimuli. To quantify the way in which the orientation of the high-dimensional distribution changes across stimuli, we use the variance of the population activity projected along the direction of the (normalized) mean response, r ¯ (s) = r (s) / | r (s) |, i. e. , σ μ 2 (s) = ∑ i j C i j (s) r ¯ j (s) r ¯ i (s). (4) This quantity can be compared to the variance projected along the diagonal direction, d ¯ = (1, …, 1) T / N, σ d 2 (s) = ∑ i j C i j (s) d ¯ i d ¯ j, (5) which corresponds to a uniform averaging of the covariances. To compare these quantities across stimuli, σ μ 2 and σ d 2 are normalized by the sum of the variances, σ a l l 2 (s) = ∑ i C i i (s). If, for a given stimulus, all neurons are equally active on average, then r ¯ = d ¯ and σ d 2 = σ μ 2. As we show below, different circuit models predict distinct stimulus dependencies of σ μ 2 and σ d 2. The discrepancies are most apparent for the stimuli for which the average population response differs strongly from a uniform population response. A deviation from a uniform response can be measured by the angle between mean response and diagonal, or cos (d, r) = r ¯ T · d ¯. For a graphical illustration of these measures, see Fig 1A and 1B. Correlations in the activity of neurons can have other origins besides recurrent connections. In parallel with the recurrent network model, we consider two alternative prototypical models in which correlations originate from shared inputs or from shared gain fluctuations, respectively. The three different scenarios are illustrated in Fig 1C. In S1 Appendix we show how, formally, these models can also be cast as special cases of the recurrent model. In order to evaluate the influence of correlation on neural coding, we examine the discriminability of a set of stimuli from the population activity. In the models, a stimulus, s, evokes activity characterized by an average response vector, r (s), and a covariance matrix, C (s). We seek a simple measure for evaluating the possibility of attributing a given population response to one of two discrete stimuli, s1 and s2, unambiguously. For this, we assume that the high-dimensional distributions of responses are Gaussian, and project the two distributions on a single dimension in which we calculate the signal-to-noise ratio as our measure. The best projection follows from Fisher’s linear discriminant analysis: the most informative linear combination of neuron responses, denoted by w ⋅ r, is achieved if the vector w points in the most discriminant direction, w = (C (s 1) + C (s 2) ) - 1 (r (s 1) - r (s 2) ). (21) The mean and variance of the projected distributions onto the normalized direction, w ¯, are w ¯ T r (s i) and σ s i 2 = w ¯ T C (s i) w ¯, for i ∈ {1,2}. We can then define the signal-to-noise ratio, as S = | w ¯ T · (r (s 1) - r (s 2) ) | σ s 1 + σ s 2. (22) Larger values of S correspond to better discriminability. To quantify the effect of correlation on discriminability, we compare the quantity S, defined in Eq (22), with the quantity Sshuffled obtained from a ‘shuffled data set’, in which responses are shuffled across trials (in experimental data) or off-diagonal covariance elements, Ci ≠ j (s), are set to 0 (in models). According to this procedure, we compare the accuracy of the code in the full (experimental or model) data with that in artificially generated data in which noise correlations are removed but which keeps the average populations responses and single-neuron variances unchanged. A ratio Sshuffled/S smaller than unity indicates that correlation is beneficial to discriminability. We note that S is an approximation for a measure based on the optimal linear classifier, for which the threshold separating the two one-dimensional projected distributions has to be calculated numerically. A measure akin to S used in similar contexts is the linear Fisher information [23], valid for continuous stimuli. An advantageous property of S is its invariance under linear transformations: if all responses, r, are fed into another network whose output, Br, results from a product with an invertible matrix B, S does not change. This obtains because the mean responses are transformed into Br (s), while the covariances are transformed into BC (s) BT. Intuitively, a simple matrix multiplication is accommodated for by a corresponding change in the most discriminant direction w on which we project the population activity.
One population feature which distinguishes the possible mechanisms of generation of noise correlation is the relation between the population-averaged response, 〈r〉 (s) = 〈ri (s) 〉i = ∑i ri (s) /N, on the one hand, and the population-averaged variance, 〈Cii〉i (s) = ∑i Cii (s) /N, or the noise covariance averaged across pairs, 〈Cij (s) 〉i ≠ j = ∑i ≠ j Cij/N (N − 1), on the other hand. In this section, we analyze these relations for Poisson neurons in a recurrent network with random effective connections, as described in Methods. Using Eq (12) in which a baseline spiking rate has been included, which reads Cij = ∑k Bik Bjk (rk (s) + a + Vext, k) for the pairwise covariances, we derive the expression ⟨ C i i ⟩ i (s) ≈ N ⟨ B 2 ⟩ (⟨ r ⟩ (s) + a + ⟨ V ext ⟩) (23) for the average variance, and the expression ⟨ C i j ⟩ i ≠ j (s) ≈ N ⟨ B ⟩ 2 (⟨ r ⟩ (s) + a + ⟨ V ext ⟩) (24) for the average covariance. (See S1 Appendix for mathematical details and Fig 2 for numerical results.) The quantity 〈B〉 = ∑ij Bij/N2 is the average strength of the effective connections in neuron pairs and, correspondingly, 〈 B 2 〉 = ∑ i j B i j 2 / N 2 is the average of its square. The variance of the input, averaged across neurons, is 〈Vext〉, and a denotes a possible constant offset in the observed firing rates (firing rate baseline). Eq (24) yields a linear relation between the average spiking rate and the covariances. The factor of proportionality—the ‘slope’—has been derived, here, with the assumption that the elements of B are independent. If this is not the case, the slope will also depend on the correlations among the matrix the elements, Bij. However, the linear dependence between rates and (co-) variances itself, which results from the nature of the Poisson spike generation, is independent of this assumption. In the linear relations between average variance and spiking rate, and between average covariance and spiking rate, the ratio of the intercept and the slope is identical and equal to the strength of the external noise. In summary, in the recurrent network model, both internally generated noise and external noise are amplified (or attenuated) by the recurrent connections, and the parameters in the relations between (co-) variances and population response can be related to network and input parameters. In feed-forward networks, both shared input units and global gain fluctuations can yield noise correlations [7,9, 13]. Here, we compare the population signatures of the noise statistics in the case of these two alternatives with those that emerge in a recurrent network model. In a feed-forward network, pairwise covariances are given by Eq (17), which can be rewritten as C i j = δ i j (r i + a) + ∑ k F i k F j k V ext, k. (25) Note that, in contrast to the case of a recurrent network, the neural firing rates, ri, only affect diagonal entries, i. e. , variances. The average covariance can be expressed as ⟨ C i j ⟩ i ≠ j (s) ≈ N ⟨ F ⟩ 2 ⟨ V ext ⟩ (s), (26) and does not directly depend on the average population response, 〈r〉 (s). The relation between the average input variance, 〈Vext〉, and the average population response depends upon on the balance between positive and negative elements in rext, which we measure through the input variability, ρext = var (rext) /〈rext〉2 (see S1 Appendix). In a model network with global gain fluctuations, the average covariances scale quadratically with the population response, as ⟨ C i j ⟩ i ≠ j = V ext ⟨ r ⟩ 2. (27) This is a direct consequence of the fact that pairwise covariances are proportional to the product of the firing rates of the two neurons in the pair. By contrast, the single-cell variances and the population-averaged variance each have both quadratic and linear contribution in r and 〈r〉, respectively (see also e. g. [9]). (Further details are provided in S1 Appendix.) Additional discrepancies between the different scenarios can be related to the detailed structure of the matrix C, which determines the shape of the response distribution. By ‘shape’ we refer to the geometric orientation and extent of the multi-dimensional ellipsoid cloud that corresponds to the population responses, in the space spanned by the responses of the individual neurons. A two-dimensional sketch of such an ellipse, and the geometric interpretation of the quantities we use, are depicted in Fig 1A (see also Methods). Loosely speaking, the orientation of the response distribution depends on the network architecture: the long axis of the ellipsoid, i. e. , the direction of the largest variance, lies along the diagonal in the cases of the recurrent network model and the feed-forward model with shared inputs, but instead along the direction of the mean response in the case of the gain-fluctuation model (Fig 1B). (The “diagonal direction” corresponds to a population response pattern in which all neurons are equally active.) To capture this dependence quantitatively, we consider the projected variances, σ μ 2 and σ d 2 (see Fig 2, and Methods for details). Apart from differences in the relations between population-averaged response and variance, Fig 2A–2C, we find that, in the gain fluctuation model, but not in the other two models, the variance projected along the direction of the average response is approximately constant across stimuli (Fig 2D–2F). In recurrent and feed-forward network models in which the connections induce strong correlations, a large proportion of the variance lies along the diagonal direction, independently of the stimulus, in contrast to the outcome of the gain-fluctuation model (Fig 2G–2I). The variance projected along the direction of the average response is large only if the latter happens to be similar to the diagonal direction. For a given data set, a plot of the relations between population response and (co) variance can thus be used as a simple test for the consistency of the data with the different circuit models. In the simple scenarios we analyze here, average responses and covariances are related because they both depend on the network architecture. By expressing signal correlations, c i j S, and noise correlations, c i j N, in terms of network and input parameters, we can derive a relation between these two sets of quantities (Fig 3). For a random transfer matrix, B, all neuron pairs are statistically equivalent, and the strength of correlations can be characterized by the average of the pairwise signal and noise correlations. In a recurrent network model obtained from a random matrix B in which elements are independent, the network architecture affects noise and signal correlations through an effective quantity, namely, the signal-to-noise ratio of the elements of the transfer matrix, ρ = var (B) /〈B〉2: c N ≡ ⟨ c i j N ⟩ i ≠ j ≈ 1 1 + ρ, (28) c S ≡ ⟨ c i j S ⟩ i ≠ j ≈ 1 + (N - 1) c in 1 + ρ + (N - 1) c in (29) (see S1 Appendix, Eqs (19) and (26) ). Here, cin denotes the strength of signal correlations in the input across stimuli. Both signal and noise correlations are larger in networks with more homogeneous entries (smaller ρ). If the average input for pairs of neurons across stimuli is uncorrelated (no input signal correlations, cin = 0), noise and signal correlations are identical on average. However, due to a prefactor which grows with system size (N − 1 in Eq (29) ), even weak input signal correlations are strongly amplified and can yield a large effect, Fig 3A. Because noise in the population response to a given stimulus is produced internally in the process of spike generation, noise covariances are unaffected by this mechanism, so that strong signal correlations can coexist with weak noise correlations, Fig 3B. These relations depend on network parameters and hold for the average correlations (across pairs in a network). The pairwise noise and signal correlation coefficients, c i j N and c i j 2, can vary widely, but numerical calculations indicate that they are correlated (across pairs) in a given network, Fig 3C. For the corresponding relations in the other two models, see Table 2. Details of the derivations are reported in S1 Appendix, as well as in previous work on the gain-fluctuation model (see Discussion). Since the network architecture affects both signal and noise correlations, it is natural to ask how the discriminability of stimuli is affected in turn, as it depends on both quantities. Indeed, noise correlations can affect the coding properties of a population appreciably [24]. Noise correlations are referred to as favorable for the discrimination of a pair of stimuli when the shapes of the two correlated response distributions (corresponding to the two stimuli) are such that variability occurs predominantly in a direction orthogonal to the informative direction (Fig 4). The relevance of correlations can be quantified by comparing the discriminability of stimulus pairs in the case of the full population output and in the case in which noise correlations are removed by shuffling trials. A number of earlier studies have considered the influence of correlated variability on coding [23,26,28,29,32]. Most of these, however, did not consider the dynamical or mechanistic origins of correlations. Some recent investigations have related coding with correlated variability to questions of dynamics and circuits [7,9, 11,13,14], but these have focused on feed-forward architecture and, primarily, small populations of neurons. By contrast, we consider here larger, recurrent networks. In Fig 4, we analyze, in the recurrent network model, the interplay of signal and noise correlations that influences stimulus discrimination. A dominant portion of the variance occurs along the diagonal (Fig 4A), so that stimulus discriminability depends on the angle between the diagonal and the line connecting the two average population responses (corresponding to the two stimuli). In the case of a family of stimuli (Fig 4B), discriminability depends on the distribution of the many angles corresponding to different choices of pairs of stimuli. Here, we illustrate this quantity (Fig 4C), and we compare discriminability in the recurrent model to that for independent neurons (Fig 4D), in the case of a population with 60 neurons. We find that the more heterogeneous our recurrent connections, i. e. , the larger the parameter ρ, the more frequently small cosines, i. e. , favorable angles, occur (Fig 4C). We then compare discriminability (through the signal-to-noise ratio, S, defined in Eq (22) in the recurrent network model with that in a population of independent neurons obtained by shuffling across trials (Fig 4D). We find that the effect of shuffling depends on network heterogeneity as well (via the distribution of the noise correlations); however, when averaged over all pairs of stimuli, the effect of correlations is negligible and does not depend on the network’s heterogeneity. In the case of more homogeneous recurrent networks, both the beneficial and harmful effect of noise correlations are boosted, for different stimulus pairs. We note that similar effects can be observed in the model of feed-forward network with shared input and in the model of feed-forward network with gain fluctuation; see S1 Fig. In S2 Appendix we analyze idealized two- and four-population models of recurrent networks which provide further intuition on the relation between network structure and the representation of stimulus information. We illustrate the way in which the accuracy of stimulus representation depends on the synaptic strengths; among other things we show that, by adequately varying the within-population and cross-population synaptic strengths, stimulus representation can improve while firing rates and variance remain unchanged. Intriguingly, this improvement is accompanied by an increase of the noise entropy in the network, rather than the more commonly expected decrease of the noise entropy. We use the theoretical results presented in the previous sections to analyze the responses of populations of neurons to different stimuli. The data set contains the firing rates, collected during a certain time interval, in response to the presentation of different sound stimuli in a number of trials (see Methods and Ref. [35]). We will compare the properties of covariances (across trials) and average responses with the predictions of our different models. Based on the models, we can then evaluate the effect of network-generated variability on stimulus discrimination. The trial-to-trial variability in single-neuron output is large (supra-Poisson, Fig 5A); neurons with larger average response exhibit also a larger variability in their responses. This tendency is observed across different neurons responding to a given stimulus, for individual neurons across stimuli, and for the average population response across stimuli. The pairwise correlations in trial-to-trial variability are quantitatively large and pairs with high noise correlations tend to have strong signal correlations, Fig 5B. In other words, because signal correlations measure the similarity of average responses across different stimuli, neurons with similar tuning properties also have similar trial-to-trial variability. Additionally, in populations with strong average signal correlations, noise correlations are strong as well. These relations between noise and signal correlations can be reproduced in the recurrent network model (Fig 3). However, similar results may be possible also in alternative scenarios for the generation of correlated variability. We compare the different models in the next section. Motivated by our investigation of network models, we first examine the relation between the shape of the response distributions for each stimulus and the pattern of average responses for the entire set of stimuli. In Fig 6A and 6B, we display the normalized standard deviation projected on the direction of the mean response, σμ/σall, and the normalized standard deviation projected on the diagonal direction, σd/σall, as functions of the angle between average response and diagonal, represented as cos (d, r) (see Methods for the definitions of these quantities). We observe a strong dependence on cos (d, r) of the variance projected on the direction of the mean response, but not of the variance projected on the diagonal direction. The dependence of σμ/σall on the stimulus is much stronger in nearly all the measured populations, Fig 6C. This behavior is consistent with a network model, either feed-forward or recurrent, but not with a model with shared gain fluctuations, where a large part of the variance is consistently in the direction of the mean response (see Sec. “Population signatures of noise statistics in feed-forward network models”). We note, however, that in the data firing rates are measured relative to the spontaneous activity; this is reflected in the presence of negative values in the stimulus-evoked activity. Such an offset could affect our comparison because we do not know the true value of the mean response. To account for this possibility, we searched for the best possible offset, assuming the model of gain fluctuations for the stimulus-dependent covariances. We then corrected the firing rates by this offset, and evaluated the stimulus dependence of σμ and σd, as before (see S1 Appendix for details), but found no qualitative change in the results. However, we did not test for mixed models, and it is possible that part of the correlated variability can be explained by common gain fluctuations. Based on the results presented above, we conclude that, of our three scenarios, feed-forward and recurrent network models are more consistent with the data. In the following, we fit the parameters of these models to the data, to find which of these two provides a better fit. For this comparison, we analyze the dependence of the average variances, 〈Cij〉i ≠ j, and covariances, 〈Cii〉i, on the population-averaged response, 〈r〉 (s). In the recurrent network (Eqs (23) and (24) ), both variances and covariances increase linearly with 〈r〉, while in the feed-forward network (Eq (27) ), mean covariances are not expected to depend strongly on 〈r〉. To examine whether the experimental data is consistent with the feed-forward model, we fitted parameters to the activity of each experimentally observed population (see S1 Appendix), and generated a surrogate set of responses and covariance matrices, with a matching number of neurons and stimuli. We followed this procedure because, here and in the following, we do not fit a full connectivity matrix, but generate a matrix, F, with random entries from a distribution with parameters to fit the data at the population level. In particular, we obtained values for the variability in the input ensemble, ρext = var (rext) /〈rext〉2, and in the network connection strengths, ρ = var (F) /〈F〉2. We found that ρext, ρ > 1: for both input and network elements, the variance was (appreciably) larger than the mean (see Fig 7A and 7B). A large value of ρext is needed to explain the high variability of average responses across stimuli, while the variability of the network connections captured by the parameter ρ is related to the magnitude of the observed correlations. With these parameters, the statistics of the distribution of average responses are well reproduced (Fig 7C). As mentioned in Sec. “Population signatures of noise statistics in feed-forward network models”, for large values of ρext the feed-forward model does not predict a strong dependence of 〈Cij〉i ≠ j on 〈r〉 (s). We quantify this relation by the ratio of slope to intercept in a linear fit of the data; this ratio measures the strength of dependence of average covariances on population responses. In the feed-forward scenario, we find that the slopes of the linear fits in the model relative to the intercepts are too low in comparison to what is observed in the experiments (Fig 7D and 7E). The feed-forward model cannot reproduce both the large variability of neural responses across stimuli and the increase of the average covariance with the average response. In a recurrent model, this increase is expected, because noise generated by the neurons is propagated through the network: the covariances are proportional to the average response (see Eq (24) ). Moreover, it predicts that the ratios of intercept and slope in the linear behaviors of variances and covariances are the same (this ratio scales with the variance of the external input). Indeed, linear fits to these relationships reveal approximately consistent ratios of intercept and slope (panel F); the estimated parameters are summarized in Fig 7G and 7H. The external noise, estimated from the ratio intercept/slope, turns out to be of the same magnitude as the average rate. Interpreted in terms of the model, the noise resulting from Poisson spike generation thus contributes as much to neuron variance as the external input. This combined variability is propagated through the network, and, on average, multiplied by a factor N〈B〉2 > 1, corresponding to the slope in Eq (24). If this factor is larger than one, both average response and noise are amplified by the recurrent connections. In summary, we find that a model of recurrently connected neurons with random effective connections captures the observed activity statistics, in particular the relations between average response and average covariances as well as the consistent shape of population fluctuations across stimuli. Feed-forward networks models with either shared input or common gain fluctuation are not consistent with all of these observations. We note, however, that our conclusions are based on a relatively small data set; a larger number of trials and measurements of absolute values of firing rates would be desirable for a more stringent test. We quantify the influence of noise correlation on stimulus discrimination by the ratio Sshuffled/Soriginal, where the signal-to-noise ratio, S, is calculated for the data before and after shuffling trials. The quantity S is defined in Eq (22) and denotes, for a pair of stimuli, the difference in average response divided by the standard deviation of the responses, both projected on the most discriminant direction. Larger values indicate that stimuli are easier to discriminate; if Sshuffled/Soriginal is larger than one, then removing correlations by shuffling trials improves stimulus discrimination. Across pairs of stimuli, this signal-to-noise ratio varies strongly in the data (Fig 8A). On average, stimuli are slightly easier to discriminate in the shuffled data, i. e. , noise correlations are weakly harmful to the coding of the stimulus set used in the experiments Fig 8B. A closer analysis of the response distributions reveals that, to a large degree, the effect of shuffling can be explained by the relative locations of the two mean responses with respect to the diagonal, Fig 8C and 8D, as measured by cos (d, r (s1) − r (s2) ) (Fig 4). This is because, due to the noise correlations, the main direction of variability is along the diagonal. The overall effect of shuffling on stimulus discrimination depends on the relation between noise and signal correlations, Fig 8E and 8F: stronger signal correlations lead to a stronger dominance of angles that are unfavorable if noise is aligned along the diagonal direction, and in this case shuffling correlations away will benefit stimulus discrimination. In the above, we investigated the influence of noise correlation on stimulus coding. A more biologically relevant question pertains, rather, to the influence of circuit architecture and parameters on coding. These govern the response statistics, which in turn influence the coding performance. Upon varying circuit properties, in general both average responses and their higher-order statistics can change [18,30]. The overall effect of the network stimulus information during this transformation of the input into the spiking output depends on the noise generated intrinsically—by the Poisson spike-generation mechanism. Noise in the input-output transformation necessarily destroys some of the information contained in the input, so that this transformation is lossy. Stronger excitatory connections generically amplify both average responses and the trial-to-trial variability. Specifically, the variance of Poisson neurons scales with firing rate. In a feed-forward network, this is the only source of noise, and the input to the neurons increases with the strength of the feed-forward connections. As a result, the signal-to-noise ratio improves with stronger excitatory connections: the enhancement in the firing rates overcompensate the increase in noise. By contrast, in a recurrent network, noise from Poisson spike-generation is fed back into the network and this causes fluctuations of the firing rates themselves. These fluctuations are also amplified by the recurrent connections, and consequently the signal-to-noise ratio decays with stronger excitatory connections. Formally, the distinction between feed-forward and recurrent networks is embodied in dependence of the (co) variances on network parameters and average responses (Eqs 17 and 12). To examine the combined effects of recurrent connections on average responses as well as variability around these and its correlation, it is instructive to consider a simplified, recurrent network model; we discuss such a model in detail in S2 Appendix. While the theoretical framework is not a new one, the advantage of this simple model is that the effect of recurrent connections can be calculated explicitly. We show that, provided the recurrent connections are sufficiently strong, shuffling correlations will reduce stimulus discriminability. This effect arises because the shuffling removes the effect connections have on correlations, but not their effect on the average responses and single cell variances.
We studied correlated variability theoretically using the idealized framework of Poisson neurons and random effective networks. These simplifications allow us to focus on qualitative differences arising from different network architectures; in turn, the identified qualitative trends can be tested against experimental data without having to infer the detailed connectivity structure. On the down side, we cannot fit individual pairwise correlations. Also missing is a thorough investigation of the effects of more biologically realistic, non-random connections. In particular, specific connectivity structures which may give rise to different response regimes [40], such as weakly correlated activity, remain to be investigated. A broad range of quantitative work has been devoted to examining the origins of variability in the activity of neural populations. Recurrence has been invoked to explain population variability in the absence of microscopic stochasticity [41,42], and more recent work along has extended this line of thought to cover a greater range of response regimes of asynchronous [43–45]. Population-wide fluctuations have been identified as contributing strongly to the observed correlations between the responses of cortical neurons to sensory stimuli [10]. Global, multiplicative gain fluctuation has been proposed also as a mechanism for the generation of such correlations [11], and its implication on the form of the correlation has been analyzed and fitted successfully to cortical data [9]. Similar models capture noise correlations, and their stimulus dependence, in the retina [13,14], as well as the large correlations observed under anesthesia [12]. Finally, global gain fluctuation models have also been used to reproduce the relations between noise and signal correlations [7]. In general, it is difficult to differentiate the mechanism of shared input from that of common gain fluctuation, and even from that of recurrence, as the origin of correlated variability, based on measured activity. Here, instead of focusing on temporal dynamics [16,46,47], we approach the problem by considering the relation between the network architecture and the structure of population activity statistics, when a neural population is presented with an ensemble of stimuli. In principle, the observation of population responses to a battery of stimuli provides sufficiently many constraints to tease apart different network models, and infer the parameters in the corresponding connectivity matrices. In practice, and especially for large neural populations, the number of trials accessible in experiments constrains the precision of measured covariances and the resolution of the fits. Here, we took a macroscopic view, by considering measurable effects at the population level, rather than aiming to estimate the weights of individual (effective) connections. As such, we compare population quantities derived from our models to population quantities extracted from the data, rather than to individual, pairwise correlations or other such microscopic quantities. Our theoretical analyses approximate the spike generation mechanism in neurons by a Poisson process; as such, single-cell variability scales with the spiking rate. If the noise generated within the network were purely additive, and, hence, independent of spiking rates, it would be indistinguishable from external noise that is simply filtered through the network [38]. In the Poisson model, by contrast, the interplay of internally generated noise and variability due to external signals imprints its signature on the dependence of covariances on firing rates. Even if Poisson variability happens to be a faithful model of single-cell variability, still our models make the limiting assumption of linearly interacting Poisson processes. Clearly, nonlinear transfer functions can affect the relation between spiking rates and noise correlations in individual pairs of neurons [8,48]. The tractability of the model, however, allows us to obtain explicit relations between observable statistics, which can be used as a starting point to interpret the origin of correlated variability. We used a recurrent network with random effective connections to estimate the influence of the variability generated in the network dynamics on stimulus representation. Based on the paradigm that response variability amounts to noise that downstream neurons have to cope with, a number of authors have argued that noise correlations can benefit information coding, provided they suppress variability along the direction relevant for stimulus discrimination [24–26,29]. The effects of additive and multiplicative origins of noise correlation have been examined in Refs. [11,13,14]. As in earlier work, we quantified the effect of noise correlation on coding by comparing the true or model data with their modified versions obtained from shuffling trials: this allows for a comparison between the correlated population and a mean- and variance-matched independent population. In our work, however, we compared the outcome of this procedure in the case of three different network architectures, and in the context of large populations of (model) neurons presented with a battery of stimuli. Recent theoretical studies have examined the impact of noise and correlations generated in recurrent networks. Noise internally generated in a network via spike generation can in principle be averaged out in feed-forward or recurrent networks, depending on the connectivity of the network, by increasing the population size and the redundancy of the population code [30]. However, external noise and ‘sub-optimal processing’ [33] can limit the amount of information in the output activity, a fact that is reflected by the specific structure of the correlations (rather than their magnitudes) [32]. However, such structures are difficult to pinpoint in detail in measured activity [32]. In experiments, it thus remains difficult to evaluate the influence of network-generated noise on stimulus coding. In the present study, our goal was to take experimentally observed statistics of the population activity as a starting point, and to interpret the structure of the average responses and noise correlations based on outcomes of model networks. Varying network properties such as the architecture of connections changed both the pattern of average activity and the noise correlations, both of which influence the accuracy of the neural code. As our mathematical results indicate, the effect of recurrent dynamics on information coding depends primarily on the magnitude of the noise generated internally. Our toy model, as well as the investigation presented in Refs. [18,30], suggests that, generically, recurrent amplification does not improve information coding. The reason is essentially that, in a recurrent network, the noise inherent to the spike generation mechanism is fed back in the network as ‘input noise’, which in turn amplifies the ‘output noise’. This amplification harms the signal-to-noise ratio. By contrast, in a feed-forward network, only the ‘output noise’ in spike affects the code; in Poisson neurons, an increase in this variability is overcompensated by an increase in the firing rate, so that the signal-to-noise ratio improves. Our recurrent network model allows us to estimate the ‘amplification factor’ in an experimental population, as well as the consequent strength of the noise, provided simplifying assumptions such as that of a random effective network architecture. We tested the consistency of different scenarios involving interacting Poisson neurons with measured neural population responses, and we interpreted the observed variability in terms of a recurrent network model. The measured population activity was given as spike counts in 250 ms bins, so that correlations were defined over a relatively slow timescale and, therefore, did not take rapid temporal modulation (as noted in, e. g. , Ref. [49]) into account. Both experimental variability and correlations were high [50], so that the effects of correlations were potentially strong, and amenable to an analysis such as ours. In general, variability and correlations can be high in anesthetized animals [11], but noise and signal correlations were noted to be somewhat weaker in experiments that are comparable to the one we consider [51]. While it is difficult to rule out that part of the correlations result from experimental artifacts relating to the calcium imaging technique used in the experiments (e. g. , scattering of fluorescent light by the neuropil), intracellular recordings of rates were consistent with calcium recordings [35]. A related point concerns the identification of the deconvolved fluorescence signal with the neural response: if the latter is inferred up to a multiplicative factor, the numerical values of the inferred covariances will be affected by the square of this factor. Such a factor, however, would not change the functional form of the relations between firing rates and covariances, but only the numerical values of inferred parameters. Furthermore, calcium imaging operates on a slow time scale as compared to the dynamics of individual spikes. In this work, we were interested in the coarse population activity and variability, and therefore considered spike counts in a fixed window, thereby disregarding the precise temporal dynamics of neural activity. The latter may indeed be relevant, especially in auditory processing [52,53]. In comparing the data to our model results, we did not to attempt at inferring the strength of individual (effective) connections between neurons; this would require a much larger volume of data. Instead, we focused on the parameters of the distribution of effective, random connections. This approach, in the context of a recurrent network model, allowed us to capture not only the scaling of variances and covariances with firing rates, but also the relation between signal correlation and noise correlation, and the effect of shuffling away covariances on stimulus discrimination. Furthermore, differences among experimental populations of neurons could be attributed to changes in the effective connectivity within populations. Recent studies [7,9, 11–14] have proposed the mechanism of common gain fluctuation to explain correlated variability across a population. In this work, we identify similarities and differences of correlated variability that arises in recurrent models vs. feed-forward models such as the common-gain-fluctuation model. Estimated model parameters indicate recurrent amplification and suggests that the portion of the variability that can be traced back to spike generation in the population—noise harmful to coding—is comparable in magnitude to the external input noise. Interpreting correlations in the framework of a random network is obviously not a new idea; our results illustrate the fact that subtle differences in the output statistics can lead to very different interpretations on network architecture. Our data analysis suffers from an unknown offset due to putative baseline spiking rates, the limited volume of the data, and the low temporal resolution. In this context, we compared different prototypical scenarios, but did not consider mixed models. In experimental populations of neurons, the statistics of activity can obviously result from a combination of feed-forward and recurrent processes. An likely scenario is one in which a feed-forward network provides correlated input to a recurrent network from which recordings are carried out. For simple choices of model stimulus sets, pure network models will not explain all aspects of the data. Examining the relative contributions of recurrent and feed-forward processes may be relevant to understanding the origin and role of correlated variability. As a first step, here, we provide some characterizations of the outcome of each process taken individually.
The response of neurons to a stimulus is variable across trials. A natural solution for reliable coding in the face of noise is the averaging across a neural population. The nature of this averaging depends on the structure of noise correlations in the neural population. In turn, the correlation structure depends on the way noise and correlations are generated in neural circuits. It is in general difficult to identify the origin of correlations from the observed population activity alone. In this article, we explore different theoretical scenarios of the way in which correlations can be generated, and we relate these to the architecture of feed-forward and recurrent neural circuits. Analyzing population recordings of the activity in mouse auditory cortex in response to sound stimuli, we find that population statistics are consistent with those generated in a recurrent network model. Using this model, we can then quantify the effects of network properties on average population responses, noise correlations, and the representation of sensory information.
medicine and health sciences
neural networks
engineering and technology
nervous system
signal processing
signaling networks
random variables
neuroscience
covariance
mathematics
network analysis
computational neuroscience
coding mechanisms
computer and information sciences
animal cells
neural pathways
probability theory
cellular neuroscience
neuroanatomy
cell biology
anatomy
signal to noise ratio
neurons
biology and life sciences
cellular types
physical sciences
computational biology
Interpretation of correlated neural variability from models of feed-forward and recurrent circuits