article: It has previously been shown that generic cortical microcircuit models can perform complex real-time computations on continuous input streams, provided that these computations can be carried out with a rapidly fading memory. We investigate the computational capability of such circuits in the more realistic case where not only readout neurons, but in addition a few neurons within the circuit, have been trained for specific tasks. This is essentially equivalent to the case where the output of trained readout neurons is fed back into the circuit. We show that this new model overcomes the limitation of a rapidly fading memory. In fact, we prove that in the idealized case without noise it can carry out any conceivable digital or analog computation on time-varying inputs. But even with noise, the resulting computational model can perform a large class of biologically relevant real-time computations that require a nonfading memory. We demonstrate these computational implications of feedback both theoretically, and through computer simulations of detailed cortical microcircuit models that are subject to noise and have complex inherent dynamics. We show that the application of simple learning procedures (such as linear regression or perceptron learning) to a few neurons enables such circuits to represent time over behaviorally relevant long time spans, to integrate evidence from incoming spike trains over longer periods of time, and to process new information contained in such spike trains in diverse ways according to the current internal state of the circuit. In particular we show that such generic cortical microcircuits with feedback provide a new model for working memory that is consistent with a large set of biological constraints. Although this article examines primarily the computational role of feedback in circuits of neurons, the mathematical principles on which its analysis is based apply to a variety of dynamical systems. Hence they may also throw new light on the computational role of feedback in other complex biological dynamical systems, such as, for example, genetic regulatory networks.
  The neocortex performs a large variety of complex computations in real time. It is conjectured that these computations are carried out by a network of cortical microcircuits, where each microcircuit is a rather stereotypical circuit of neurons within a cortical column. A characteristic property of these circuits and networks is an abundance of feedback connections. But the computational function of these feedback connections is largely unknown. Two lines of research have been engaged to solve this problem. In one approach, which one might call the constructive approach, one builds hypothetical circuits of neurons and shows that (under some conditions on the response behavior of its neurons and synapses) such circuits can perform specific computations. In another research strategy, which one might call the analytical approach, one starts with data-based models for actual cortical microcircuits, and analyses which computational operations such “given” circuits can perform under the assumption that a learning process assigns suitable values to some of their parameters (e. g. , synaptic efficacies of readout neurons). An underlying assumption of the analytical approach is that complex recurrent circuits, such as cortical microcircuits, cannot be fully understood in terms of the usually considered properties of their components. Rather, system-level approaches that directly address the dynamics of the resulting recurrent neural circuits are needed to complement the bottom-up analysis. This line of research started with the identification and investigation of so-called canonical microcircuits [1]. Several issues related to cortical microcircuits have also been addressed in the work of Grossberg; see [2] and the references therein. Subsequently it was shown that quite complex real-time computations on spike trains can be carried out by such “given” models for cortical microcircuits ([3–6], see [7] for a review). A fundamental limitation of this approach was that only those computations could be modeled that can be carried out with a fading memory, more precisely only those computations that require integration of information over a timespan of 200 ms to 300 ms (its maximal length depends on the amount of noise in the circuit and the complexity of the input spike trains [8]). In particular, computational tasks that require a representation of elapsed time between salient sensory events or motor actions [9], or an internal representation of expected rewards [10–12], working memory [13], accumulation of sensory evidence for decision making [14], the updating and holding of analog variables such as for example the desired eye position [15], and differential processing of sensory input streams according to attentional or other internal states of the neural system [16] could not be modeled in this way. Previous work on concrete examples of artificial neural networks [17] and cortical microcircuit models [18] had already indicated that these shortcomings of the model might arise only if one assumes that learning affects exclusively the synapses of readout neurons that project the results of computations to other circuits or areas, without giving feedback into the circuit from which they extract information. This scenario is in fact rather unrealistic from a biological perspective, since pyramidal neurons in the cortex typically have in addition to their long projecting axon a large number of axon collaterals that provide feedback to the local circuit [19]. Abundant feedback connections also exist on the network level between different brain areas [20]. We show in this article that if one takes feedback connections from readout neurons (that are trained for specific tasks) into account, generic cortical microcircuit models can solve all of the previously listed computational tasks. In fact, one can demonstrate this also for circuits whose underlying noise levels and models for neurons and synapses are substantially more realistic than those which had previously been considered in models for working memory and related tasks. We show in the Theoretical Analysis section that the significance of feedback for the computational power of neural circuits and other dynamical systems can be explained on the basis of general principles. Theorem 1 implies that a large class of dynamical systems, in particular systems of differential equations that are commonly used to describe the dynamics of firing activity in neural circuits, gain universal computational capabilities for digital and analog computation as soon as one considers them in combination with feedback. A further mathematical result (Theorem 2) implies that the capability to process online input streams in the light of nonfading (or slowly fading) internal states is preserved in the presence of fairly large levels of internal noise. On the basis of this theoretical foundation, one can explain why the computer models of generic cortical microcircuits, which are considered in the section Applications to Generic Cortical Microcircuit Models, are able to solve the previously mentioned benchmark tasks. These results suggest a new computational model for cortical microcircuits, which includes the capability to process online input streams in diverse ways according to different “instructions” that are implemented through high-dimensional attractors of the underlying dynamical system. The high dimensionality of these attractors results from the fact that only a small fraction of synapses need to be modified for their creation. In comparison with the commonly considered low-dimensional attractors, such high-dimensional attractors have additional attractive properties such as compositionality (the intersection of several of them is in general nonempty) and compatibility with real-time computing on online input streams within the same circuit. The presentation of theoretical results for abstract circuit models in the Theoretical Analysis section is complemented by mathematical details in the Methods section, under the heading Mathematical Definitions, Details to the Proof of Theorem 1, and Examples, and the heading Mathematical Definitions and Details to the Proof of Theorem 2. Details of the computer simulations of more detailed cortical microcircuit models are discussed in Applications to Generic Cortical Microcircuit Models in the Methods section. A discussion of the results of this paper is given in the Discussion section.
  The dynamics of firing rates in recurrent circuits of neurons is commonly modeled by systems of nonlinear differential equations of the form or [22–25]. Here each xi, i = 1, …, n, is a real-valued variable that represents the current firing rate of the ith neuron or population of neurons in a recurrent neural circuit, and v (t) is an external input stream. The coefficients aij, bi denote the strengths of synaptic connections, and λi > 0 denotes time constants. The function σ is some sigmoidal activation function (nondecreasing, with bounded range). In most models of neural circuits, the parameters are chosen so that the resulting dynamical system has a fading memory for preceding inputs. If one makes the synaptic connection strengths aij in Equation 1 or Equation 2 so large that recurrent activity does not dissipate, the neural circuit tends to exhibit persistent memory. But it is usually quite difficult to control the content of this persistent memory, since it tends to be swamped with minor details of external inputs (or initial conditions) from the distant past. Hence this chaotic regime of recurrent neural circuits (see [62] for a review) is apparently also not suitable for biologically realistic online computations that combine new information from the current input with selected (e. g. , behaviorally relevant) aspects of external or internal inputs from the past. Recurrent circuits of neurons (e. g. , those described by Equations 1 or 2) are from a mathematical perspective special cases of dynamical systems. The subsequent mathematical results show that a large variety of dynamical systems, in particular also fading-memory systems of type Equation 1 or Equation 2, can overcome in the presence of feedback the computational limitations of a fading memory without necessarily falling into the chaotic regime. In fact, feedback endows them with universal capabilities for analog computing, in a sense that can be made precise in the following way (see Figure 1A–1C for an illustration): We examine in this section computational aspects of feedback in recurrent circuits of spiking neurons that are based on data from cortical microcircuits. The dynamics of these circuits is substantially more complex than the dynamics of circuits described by Equation 6, since it is based on action potentials (spikes) rather than on firing rates. Hence one can expect at best that the temporal dynamics of firing rates in these circuits of spiking neuron is qualitatively similar to that of circuits described by Equation 6. The preceding theoretical results imply that it is possible for dynamical systems to carry out computations with persistent memory without acquiring all the computational disadvantages of the chaotic regime, where the memory capacity of the system is dominated by noise. Feedback units can create selective “loopholes” into the fading-memory dynamics of a dissipative system that can only be activated by specific patterns in the input or circuit dynamics. In this way the potential content of persistent memory can be controlled by feedback units that have been trained to recognize such patterns. This feedback may arise from a few neurons within the circuit, or from neurons within a larger feedback loop. The task to approximate a suitable feedback function K is less difficult than it may appear on first sight, since it suffices in many cases to approximate a linear feedback function. The reason is that sufficiently large generic cortical microcircuit models have an inherent kernel property [8], in the sense of machine learning [38]. This means that a large reservoir of diverse nonlinear responses to current and recent input patterns is automatically produced within the recurrent circuit. In particular, nonlinear combinations of variables a, b, c, … (that may result from the circuit input or internal activity) are automatically computed at internal nodes of the circuit. Consequently, numerous low-degree polynomials in these variables a, b, c, … can be approximated by linear combinations of outputs of neurons from the recurrent circuit. An example of this effect is demonstrated in Figure 2G, where it is shown that the product of firing rates r3 (t) and r4 (t) and of two independently varying afferent spike train inputs can be approximated quite well by a linear readout neuron. The kernel property of biologically realistic cortical microcircuit models is apparently supported by the fact that these circuits have many additional nonlinearities in addition to those that appear in Equations 1,2, and 6. One formal difference between neurons in the mean field model (Equation 6) and more realistic models for spiking neurons is that the input to a neuron of the latter type consists of postsynaptic potentials, rather than of firing rates. Hence the time-varying input x (t) to a readout neuron is in this section not a vector of time-varying firing rates, but a smoothed version of the spike trains of all presynaptic neurons. This smoothing is achieved through application of a linear filter with an exponentially decaying kernel, whose time constant of 30 ms models time constants of receptors and postsynaptic membrane of a readout neuron in a qualitative fashion. Thus, if w is a vector of synaptic weights, then w · x (t) models the impact of the firing activity of presynaptic neurons on the membrane potential of a readout neuron. We refer in the following to those neurons where the weights of synaptic connections from neurons within the circuit are adapted for a specific computational task (rather than chosen randomly from distributions that are based on biological data, as for all other synapses in the circuit) as readout neurons. The output of a readout neuron was modeled in most of our simulations simply by a weighted sum w · x (t) of the previously described vector x (t). Such output can be interpreted as the time-varying firing rate of a readout neuron. However, we show in Figure 2 that these readout neurons can (with a moderate loss in performance) also be modeled by spiking neurons, exactly like the other neurons in the simulated circuit. This demonstrates that not only those circuits that receive feedback from external readout neurons, but also generic recurrent circuits in which a few neurons have been trained for a specific task, acquire computational capabilities for real-time processing that are not restricted to computations with fading memory. Theorem 2 suggests that the training of a few of its neurons enables generic neural circuits to employ persistent internal states for state-dependent processing of online input streams. Previous models for nonfading memory in neural circuits [13,39–41] proposed that it is implemented through low-dimensional attractors in the circuit dynamics. These attractors tend to freeze or to entrain the whole state of the circuit, and thereby shut it off from the online input stream (although independent local attractors could emerge in local subcircuits under some conditions [40]). In contrast, the generation of nonfading memory through a few trained neurons does not entail that the dynamics of the circuit be dominated by their persistent memory states. For example, when a readout neuron gives during some time interval a constant feedback K (x (t) ) = c, this only constrains the circuit state x (t) to remain in the sub-manifold {x: K (x) = c} of its high-dimensional state space. This sub-manifold is in general high-dimensional. In particular, if K (x) is a linear function w · x, which often suffices as we will show; the dimensionality of the sub-manifold {x: K (x) = c} differs from the dimension of the full state space only by 1. Hence several such sub-manifolds have in general a high-dimensional intersection, and their intersection still leaves sufficiently many degrees of freedom for the circuit state x (t) to also absorb continuously new information from online input streams. These sub-manifolds are in general not attractors in a strict mathematical sense. Rather, their effective attraction property (or noise-robustness) results from the subsequently described training process (“teacher forcing”). This training process produces weights w which have the property that the resulting feedback moves on a trajectory of circuit states that goes through states x~ (t) in the neighborhood of the sub-manifold {x: K (x) = c}, closer to this sub-manifold. We simulated generic cortical microcircuit models consisting of 600 integrate-and-fire (I&F) neurons (for Figures 2 and 3), and circuits consisting of 600 Hodgkin–Huxley (HH) neurons (for Figure 4), in either case with a rather high level of noise that reflects experimental data on the high conductance state in vivo [42]. These circuits were not constructed for any particular computational task. In particular, sparse synaptic connectivity between neurons was generated (with a biologically realistic bias towards short connections) by a probabilistic rule. Synaptic parameters were chosen randomly from distributions that depend on the type of pre- and postsynaptic neurons (in accordance with empirical data from [43,44]). More precisely, we used biologically realistic models for dynamic synapses whose individual mixture of paired-pulse depression and facilitation (depending on the type of pre- and postsynaptic neuron) was based on these data. It has previously been shown in [6,8] that the presence of such dynamic synapses extends the timespan of the inherent fading memory of the circuit. However the computational tasks that are considered in this paper require, apart from a nonfading memory, only a fading memory with a rather short timespan (to make the estimation of the current firing rate of input spike trains feasible). Therefore, the biologically more realistic dynamic synapses could be replaced in this model by simple static synapses, without a change in the performance of the circuit for the subsequently considered tasks. All details of the simulated microcircuit models can be found in Details of the Cortical Microcircuit Models. Details of the subsequently discussed computer experiments are given in the sections Technical Details of Figure 5, Technical Details of Figure 2, and Technical Details of Figure 3. We tested three different types of computational tasks for generic neural circuits with feedback. The same neural circuit can be used for each task, only the organization of input and output streams needs to be chosen individually (see Figure 5). The following procedure was applied to train readout neurons, i. e. , to adjust the weights of synaptic connections from neurons in the circuit to readout neurons for specific computational tasks (while leaving all other parameters of the generic microcircuit model unchanged): 1) first those readout neurons were trained that provide feedback, then the other readout neurons; 2) during the training of readout neurons that provide feedback, their actual feedback was replaced by a noisy version of their target output (“teacher forcing”); 3) each readout neuron was trained by linear regression to output at any time t a particular target value f (t). Linear regression was applied to a set of datapoints of the form 〈x (t), f (t) 〉 for many timepoints t, where x (t) is a smoothed version of the spike trains of presynaptic neurons (as defined before). Note that teacher forcing, with noisy versions of target feedback values, trains these readouts to correct errors resulting from imprecision in their preceding feedback (rather than amplifying errors). This training procedure is responsible for the robustness of the dynamics of the resulting closed-loop circuits, in particular for the “attractor” properties of the effectively resulting high-dimensional attractors. In our first computer experiment, readout neurons were trained to turn a high-dimensional attractor on or off (Figure 2D), in response to bursts in two of the four independent input spike trains. More precisely, eight neurons were trained to represent in their firing activity at any time the information: in which of the input streams, 1 or 2, had a burst most recently occurred? If it had occurred most recently in stream 1, they were trained to fire at 40 Hz, and if a burst had occurred most recently in input stream 2, they were trained not to fire. Hence these neurons were required to represent the nonfading state of a simple FSM, demonstrating in an example the computational capabilities predicted by Theorem 2. Figure 2G demonstrates that the circuit retains its kernel property in spite of the feedback injected into the circuit by these readouts. But beyond the emulation of a simple FSM, the resulting generic cortical microcircuit is able to combine information stored in the current state of the FSM with new information from the online circuit input. For example, Figure 2E shows that other readouts from the same circuit can be trained to amplify their response to specific inputs if the high-dimensional attractor is in the “on” state. Readouts can also be trained to change the function that they compute if the high-dimensional attractor is in the on state (Figure 2F). This provides an example for an online reconfigurable circuit. The readout neurons that provide feedback had been modeled in this computer simulation like the other neurons in the circuit: by I&F neurons with in vivo–like background noise. Hence they can be viewed equivalently as neurons within an otherwise generic circuit. Another difficult problem in computational neuroscience is to explain how neural circuits can implement a parametric memory, i. e. , how they can hold and update an analog value that may represent, for example, an intended eye position that a neural integrator computes from a sequence of eye-movement commands [45], an estimate of elapsed time [9], or accumulated sensory evidence [14]. Various designs have been proposed for parametric memory in recurrent circuits, where continuous attractors (also referred to as line attractors) hold and update an analog value. But these approaches are inherently brittle [41], and have problems in dealing with high noise or online circuit inputs. On the other hand, Figure 3 shows that dedicated circuit constructions are not necessary, since feedback from readout neurons in generic cortical microcircuits models can also create high-dimensional attractors that hold and update an analog value for behaviorally relevant timespans. In fact, due to the high-dimensional character of the resulting high-dimensional attractors, two such analog values can be stored and updated independently (Figure 3C and 3D), even within a fairly small circuit. In this example, the readouts that provide feedback were simply trained to increase or reduce their feedback at each timepoint. Note that the resulting circuit activity is qualitatively consistent with recordings from neurons in cortex and striatum during reward expectation [10–12]. A similar ramp-like rise and fall of activity as shown in Figure 3C, 3D, and 3F has also been recorded in neurons of posterior parietal cortex of the macaque in experiments where the monkey had been trained to classify the duration of elapsed time [9]. The high dimensionality of the continuous attractors in this model makes it feasible to constrain the circuit state to stay simultaneously in more than one continuous attractor, thereby making it in principle possible to encode complex movement plans that require specific temporal relationships between individual motor commands. Our model for parametric memory in cortical circuits is consistent with high noise: Figure 4G shows the typical trial-to-trial variability of a neuron in our simulated circuit of HH neurons with in vivo–like background noise. It qualitatively matches the “wide diversity of neural firing drift patterns in individual fish at all states of tuning” that was observed in the horizontal occulomotor neural integrator in goldfish [15], and the large trial-to-trial variability of neurons in prefrontal cortex of monkeys reported in [10]. In addition, this model is consistent with the surprising plasticity that has been observed even in quite specialized neural integrators [15], since continuous attractors can be created or modified in this model by changing just a few synaptic weights of neurons that are immediately involved. It does not require the presence of long-lasting postsynaptic potentials, NMDA receptors, or other specialized details of biological neurons or synapses, although their inclusion in the model is likely to provide additional temporal stability [13]. Rather it points to complementary organizational mechanisms on the circuit level, which are likely to enhance the controllability and robustness of continuous attractors in neural circuits. The robustness of this learning-based model can be traced back to the fact that readout neurons can be trained to correct undesired circuit responses resulting from errors in their previous feedback. Furthermore, such error correction is not restricted to linear computational operations, since the previously demonstrated kernel property of these generic circuits allows even linear neurons to implement complex nonlinear control strategies through their feedback. As an example, we demonstrate in Figure 4 that even under biologically realistic high-noise conditions a linear readout can be trained to update a continuous attractor (Figure 4D), to filter out input activity during certain time intervals independent of the current state of the continuous attractor (Figure 4E), or to combine the time-varying analog variable encoded by the current state CA (t) of the continuous attractor with a time-varying variable r1 (t) that is delivered by an online spike input. Hence, intention-based information processing [16] and other tasks that involve a merging of external inputs and internal state information can be implemented in this way. Figure 4C shows that a high-dimensional attractor need not entrain the firing activity of neurons in a drastic way, since it just restricts the high-dimensional–circuit dynamics x (t) to a slightly lower dimensional manifold of circuit states x (t) that satisfy w · x (t) = f (t) for the current target output f (t) of the corresponding linear readout. On the other hand, Figure 4E shows that the activity level CA (t) of the high-dimensional attractor can nevertheless be detected by other linear readouts, and can simultaneously be combined in a nonlinear manner with a time-varying variable r2 (t) from one afferent circuit input stream, while remaining invariant to the other afferent input stream. Finally, the same generic circuit also provides a model for the integration of evidence for decision making that is compatible with in vivo–like high noise conditions. Figure 4H depicts the timecourse of the same neural integrator as in Figure 4D, but here for the case where the rates r1, r2 of the two input streams assume in eight trials eight different constant values after the first 100 ms (while assuming a common value of 65 Hz during the first 100 ms). The resulting timecourse of the continuous attractor is qualitatively similar to the meandering path towards a decision threshold that has been recorded from neurons in area LIP where firing rates represent temporally integrated evidence concerning the dominating direction of random dot movements (see Figure 4A in [14]).
  We have presented a theoretically founded model for real-time computations on complex input streams with persistent internal states in generic cortical microcircuits. This model does not require a handcrafted circuit structure or biologically unrealistic assumptions such as symmetric weight distributions, static synapses that do not exhibit pair-pulsed depression or facilitation, or neuron models with low levels of noise that are not consistent with data on in vivo conditions. Our model only requires the assumption that adaptive procedures (synaptic plasticity) in generic neural circuits can approximate linear regression. Furthermore, in contrast to classical learning paradigms for attractor neural networks, it is here not required that a large fraction of synaptic parameters in the circuit are changed when a new computational task is introduced or a new item is stored in working memory. Rather, it suffices if those neurons that provide the circuit output and a few neurons that provide feedback are subject to synaptic plasticity. Such minimal circuit modifications have the advantage that thereby created attractors of the circuit dynamics are high-dimensional. We have shown that the circuit state can simultaneously be in several of such high-dimensional attractors, and still retain sufficiently many degrees of freedom to absorb and process new information from online input streams. In particular, we have shown in Figures 2 and 4 how bottom-up processing can be reconfigured dependent on discrete internal states (implemented through high-dimensional attractors) by turning certain input channels on or off, and by changing the computational operations that are applied to input variables. Furthermore we have shown in Figure 4 that analog variables, which are extracted from an online input stream, can be combined in real-time computations with analog variables that are stored in high-dimensional continuous attractors. This provides in particular a model for the implementation of intention-based information processing [16] in cortical microcircuits. It remains open how learning signals can induce neurons in a biological organism to compute specific linear feedback functions. But at least we have reduced this problem to the feasibility of perceptron-like learning (or more abstractly: to linear regression) for single neurons. Subsequent research will have to determine whether these learning requirements (which can be partially reduced to spike-timing dependent plasticity [46]) can be justified on the basis of results on unsupervised learning and reinforcement learning [47] in biological organisms. Whereas it was previously already known that one can construct specific circuits that have universal computational capabilities for real-time computing on analog input streams, Theorems 1 and 2 of this article imply that a large variety of dynamical systems (in particular generic cortical microcircuits) can acquire through feedback such universal capabilities for computations that map time-varying inputs to time-varying outputs. It should be noted that these universal computational capabilities differ from the well-known but much weaker universal approximation property of feedforward neural networks (see [34]), since not only the static output of an arbitrary continuous static function is approximated, but also the dynamic response of arbitrary differential equations of higher-order to time-varying inputs. The theoretical results of this article also provide an explanation for the astounding computational capability and flexibility of echo state networks [17]. In addition they can be used to analyze computational aspects of feedback in other biological dynamical systems besides neural circuits. Several such systems, for example, genetic regulatory networks, are known to implement complex maps from time-varying input streams (e. g. , external signals) onto time-varying outputs (e. g. , transcription rates). But little is known about the way in which these maps are implemented. Whereas feedback in biological dynamical systems is usually only analyzed and modeled from the perspective of control, we propose that an analysis of its computational aspects is likely to yield a better understanding of the computational capabilities of such systems.
  Definition of feedback equivalence. We recall that a smooth mapping is one for which derivatives of all orders exist (infinite differentiability), and that a diffeomorphism T: T: ℜn → ℜn is a smooth mapping for which there exists a well-defined smooth inverse T−1: ℜn → ℜn. Definition (see [33], Definition 5. 3. 1). Two n-dimensional systems x′ = f (x) + g (x) v and (with smooth vector fields are called feedback equivalent (over the state space ℜn) if there exists 1) a diffeomorphism T: ℜn → ℜn, and 2) smooth maps α, β: ℜn → ℜ with β (x) ≠ 0 for all x ∈ ℜn, such that, for each x ∈ ℜn and (where T* denotes the Jacobian of T). Definition of the class Sn. Recall that a linear system x′ = Ax + bu is controllable if it is possible to drive any state x0 to any other state x1 using an input (see [33], Definition 3. 1. 6). Controllability is a generic property of systems, and amounts to the requirement that the matrix (b, Ab, …, An−1b) has full rank, where n is the dimension of the system (see [33], Theorem 2). Note that the linear system (Equation 7) satisfies this requirement, and hence is controllable. We take Sn to be the class of n-dimensional systems (Equation 3) that are (globally) feedback linearizable, that is to say, the systems (Equation 3) for which there exists some linear controllable system that is feedback equivalent to Equation 3 (see [33], Definition 5. 3. 2). An n-dimensional system is feedback linearizable if and only if it is feedback-equivalent to the system (Equation 7) (see [33], Lemma 5. 3. 5). Therefore, we have the following: Lemma: A system (Equation 3), with smooth vector fields f = 〈f1, …, fn〉 and g = 〈g1, …, gn〉 belongs to Sn if and only if there exists a diffeomorphism T: ℜn → ℜn and two smooth maps α, β: ℜn → ℜ, with β (x) ≠ 0 for all x ∈ ℜn, such that, for each x ∈ ℜn: and where T* denotes the Jacobian of T and An interpretation of the property given in the above Lemma, that will be used in the proof of Theorem 1 in the section Details to the Proof of Theorem 1, is as follows (see [33], Chapter 5, for more discussion): For each input μ (t) and each solution z (t) of the vector function x (t) = T−1 (Z (t) ) satisfies Equation 3 with the input v (t) = α (x (t) ) + β (x (t) ) μ (t), where Details to the proof of Theorem 1. In this section, we prove the simulation result that is claimed in Theorem 1. Take any system (Equation 3) in Sn and any system (Equation 4) to be simulated. Using T, α, β as in the Lemma in section Definition of the Class Sn that characterizes the class Sn, we define: and we let h (x) be the first coordinate of T (x). In the special case where Equation 3 describes the dynamics of a circuit according to Equation 6, α is a linear function, β is a constant, and T is an invertible linear map from ℜn to ℜn. Next, pick an external input u (t), t ≥ 0, and a solution z (t) of the forced system (Equation 4). From the interpretation of feedback linearization given earlier (in the last part of Definition of the Class Sn), it follows that for any inputs u (t) and u0 (t) (in particular, one could take u0 ≡ 0), and each solution z (t) of (that is, we use μ (t) = G (z (t), z′ (t), z″ (t), …, z (n−1) (t) ) + u (t) + u0 (t) as the input to z (n) = μ), the vector function x (t) = T−1 (Z (t) ) satisfies Equation 3 with input Furthermore, Z (t) = T (x (t) ) means that z (t) = h (x (t) ), as required for the notion of simulation. This almost proves the simulation result, except for the fact that there is no reason for the initial value x (0) = T−1 (Z (0) ) to be zero, since z (t) is an arbitrary trajectory. This is where the input u0 plays a role. Let ξ: = T (0). We will show that, given any solution z (t) and any input u (t), there is some input u0 (t), with u0 (t) ≡ 0 for all t ≥ 1, so that the solution of with y (0) = ξ has the property that y (t) = z (t) for all t ≥ 1. (Where z (t) is the desired trajectory to be simulated, with u0 ≡ 0.) Then letting x (t) = T−1 (Y (t) ) instead of T−1 (Z (t) ) means that x (0) and still h (x (t) ) = y (t) = z (t) for all t ≥ 1. Consider now an arbitrary solution z (t) of Equation 4 and let ζ be the vector with entries We next pick a scalar differentiable function ϕ such that ϕ (i) (0) = ξi+1 and ϕ (i) (1) = ξi+1 for i = 0, …, n − 1. (It is easy to see that such functions exist. For example, one may simply consider the linear system p′ = Anp + bnq with states p and input q. This is a completely controllable linear system (cf. [33] Chapter 3), so we just pick an input q (t) that steers ξ into ζ, and finally let ϕ (t) be the first coordinate of p (t). Now we let for t < 1, and u0 (t) ≡ 0 for t ≥ 1, and claim that the solution of Equation 10 with y (0) = ξ has the property that y (t) = z (t) for all t ≥ 1. Since u (t) + u0 (t) = u (t) for all t ≥ 1, we only need to show that y (i) (1) = z (i) (1) for every i = 0, …, n − 1. To see this, in turn, and using uniqueness of solutions of differential equations, it is enough to show that y (t): = ϕ (t) satisfies on the interval [0,1] and has derivatives at t = 0 as specified by the vector ξ. But this is indeed true by construction. Finally, we remark that if | u (t) | ≤ c and | z (i) | (t) ≤ c for all t ≥ 0, then x (t) = T−1 (Z (t) ) is bounded in norm by a constant that only depends on c (since T−1 is continuous, by definition of diffeomorphism), and the numbers bi: = z (i) (1) are also bounded by a constant that depends only on c, so K (x (t), u (t) + u0 (t) ) also is. Corollary 3. Analogous results can be shown for the simulation of systems consisting of any number k of higher order differential equations as in Equation 4. In this case fixed systems of first-order differential equations of a form as in Equation 3, but with k memoryless feedback functions K1, …Kk that depend on the simulated higher-order system, can be shown to be able to simulate the dynamic response of arbitrary higher-order systems of differential equations. Lie brackets. The study of controllability and other properties of nonlinear systems is based upon the use of Lie bracket formalism and theory ([33], Chapter 4). We need this formalism to show in the section Application to Neural Network Equations that the class Sn includes some neural networks of the form Equation 6. For any two vector fields f and g, denotes the Lie bracket of f and g. Recall that the Lie bracket of two vector fields is a vector field that characterizes the effective direction of movement obtained by performing this “commutator” motion: follow the vector field f for t time steps, then g for t time steps, then f backward in time for t time steps, and finally g backward in time for t time steps, for small t > 0. To be more precise, denote formally by etf the flow associated to f, and similarly for g. Consider the following curve, for any initial state x0: Applying repeatedly this expansion: (and similarly for g), we obtain that as t → 0, from which it follows that γ′ (0) = [f, g] (x0), which means that the direction of [f, g] is followed when performing the commutator motions. Using the possible noncommutativity of the vector fields, one generates in this manner genuinely new directions of movement in addition to those provided by the linear combinations of f and g. Well-known examples are provided by the Lie bracket of two rotations around orthogonal axes, which is a rotation around the remaining axis (see for example [33], page 150), or the motions involved in parking an automobile (see for example [33], Example 4. 3. 13). Iterations of Lie brackets play a key role. Let us introduce, for any given vector field f, the operator adf, which maps vector fields into vector fields by means of the formula adf (g): = [f, g]. Iterations of the operator adf are defined in the obvious way: and. It is also useful to consider an operator Lf that acts on scalar functions. We use the notation Lfϕ, for any (smooth) vector field f and (smooth) function ϕ, to denote the Lie derivative of ϕ along f, that is, ∇ϕ · f. The function Lfϕ, which is again smooth, is nothing more than the directional derivative of the function ϕ in the direction of the vector field f, in the sense of elementary calculus. One can also consider iterated applications of the operator Lf. A characterization of Sn via lie brackets. With these notations, we are ready to present a Lie geometric characterization of the class Sn. The next theorem follows by combining the proofs of Proposition 5. 3. 9 and of Theorem 15 in [33] (with 0 = X = ℜn in the notations of that text). Theorem 4. The system x′ = f (x) + g (x) v is globally feedback linearizable if and only if there exists a smooth function having everywhere nonzero gradient and satisfying the following properties: 1) for each x ∈ ℜn, the vectors are linearly independent; 2) for each x ∈ ℜn and each j =0, …, n − 2, ; 3) the map is a bijection ℜn → ℜn. Observe that the conditions amount to the existence of a well-behaved solution ϕ of a set of first-order linear partial differential equations. Existence of a solution of this form is not trivial to verify. To study solvability, in control theory one considers the following conditions: This last condition means that the Lie bracket of any two of the vector fields, for i ∈ {0, …, n − 2}, should be, for each x, a linear combination of these same n − 1 vectors. One then has the following result (see Theorem 15 in [33]), which is a consequence of Frobenius' Theorem in partial differential equation theory: A system satisfies both conditions (LI) and (INV) at a state x if and only if it is feedback linearizable in some open set containing x. This provides a useful and complete characterization of local feedback linearizability, and in particular a necessary condition for global feedback linearizability. In examples, often these conditions lead one to a globally defined solution, see, e. g. , example 5. 3. 10 in [33]). Application to neural network equations. Let us now show with the help of Theorem 4 that the class Sn includes some fading-memory systems of the form Equation 6. Indeed, consider any system as follows: where the λi ≠ λj for each i ≠ j are all positive, diag (λ1, . . . , λn) is the resulting diagonal matrix, and the column vector b = col (b1, …, bn) has nonzero entries: bi ≠ 0 for all i. (Such a system, which has the form Equation 6 with ϕ (Ax) ≡ 0, consists of n first-order linear differential equations in parallel, and is obviously fading-memory.) It is easy to see that, up to signs (−1) i, we have for i > 0, and the linear independence of follows from the fact that these constant vectors form a Vandermonde matrix. Then we can pick ϕ (x) as a linear map x → ax, where a is any vector in ℜn that is orthogonal to all of the vectors The map is represented then also by a Vandermonde matrix, so it is a bijection. Hence, conditions 1) −3) of Theorem 4 are satisfied, which implies that the system (Equation 11) belongs to the class Sn. As a further example, we now consider the following system, which also has the general form of the neural network Equation 6: where ϕ is a scalar function, smooth but otherwise arbitrary for now, and a as well as the λi are constants, also arbitrary for now. We will analyze this example using the Lie formalism described in the section Lie Brackets. The system has the form x′ = f (x) + g (x) v, with n = 3, and f and g are the following vector fields: Note that the Jacobian g* of g is identically zero, which simplifies the computation of Lie brackets. We calculate adfg (x) = [f, g] (x) = −f* (x) g (x) and [g, adfg] = (adfg) * (x) g (x) as follows: The involutivity condition says that the set of vector fields {g, adfg} should be involutive, which means that [g, adfg] (x) should be in the span of g (x) and adfg (x) for all x. Let us evaluate g, adfg, [g, adfg] at the particular points for which x3 = 0, so that we obtain, respectively, three vectors v1, v2, v3 that depend on x2 only: If [g, adfg] (x) is in the span of g (x) and adfg (x) for all vectors x, then, in particular, v3 (x2) must belong to the span of v1 (x2) and v2 (x2) for all x2. This means that there is, for each x2, a scalar r (x2) such that If a ≠ 0, it follows that ϕ″ (x2) = aϕ″ (x2) for all x2. So, if also a ≠ 1, we conclude that ϕ″ (x2) must vanish for all x2. Thus, the system in our example (assuming a ∉ {0,1}) is feedback linearizable only if ϕ is a linear function. On the other hand, consider now the cases a = 0 or a = 1. Then, the involutivity condition becomes the requirement that there should exist a scalar function r such that which can be achieved provided only that the function ϕ′ is everywhere nonzero (which is true if ϕ is, for example, a standard sigmoidal function), simply by taking r (x) = ϕ″ (x2 + x3) / ϕ′ (x2 + x3). The linear independence condition amounts to showing that the set of vectors {g, adfg, ad2fg} is linearly independent. Computing the determinant of the matrix that has these vectors as columns, when a = 0 we obtain −ϕ′ (x2) [ϕ′ (x2 + x3) ]2, which is everywhere nonzero, provided that we again assume that ϕ has an everywhere nonzero derivative. Thus, the Lie-theoretic conditions for feedback linearization are satisfied, for any choice of λi, when a = 0. In the case a = 1, the same computation gives a determinant of λ1 − λ2) [ϕ′ (x2 + x3) ]2, so the Lie-theoretic conditions for feedback linearization are satisfied, for any choice of λi such that λ1 ≠ λ2.
summary: Circuits of neurons in the brain have an abundance of feedback connections, both on the level of local microcircuits and on the level of synaptic connections between brain areas. But the functional role of these feedback connections is largely unknown. We present a computational theory that characterizes the gain in computational power that feedback can provide in such circuits. It shows that feedback endows standard models for neural circuits with the capability to emulate arbitrary Turing machines. In fact, with suitable feedback they can simulate any dynamical system, in particular any conceivable analog computer. Under realistic noise conditions, the computational power of these circuits is necessarily reduced. But we demonstrate through computer simulations that feedback also provides a significant gain in computational power for quite detailed models of cortical microcircuits with in vivo–like high levels of noise. In particular it enables generic cortical microcircuits to carry out computations that combine information from working memory and persistent internal states in real time with new information from online input streams.
keywords: neuroscience
  none
title: Computational Aspects of Feedback in Neural Circuits