A NE W APPR OACH TO CHAOT IC SY ST E MS CHAR ACT E R IZAT ION AND IT S IMPL ICAT IONS FOR B IOL OGY
In this paper, starting from a review of the neurophysiological evidence of chaotic behavior, we discuss the possibility of using chaos to design a chaotic neural net able to store information in each of its inner pseudo cycles. After a short discussion about the intrinsic limitations of memory in classical convergent neural net architectures, we sketch the advantages that the use of a chaotic memory could imply also from the computational (T uring machine) standpoint. Hence, we present our new technique for cycle extraction from a chaotic attractor, discussing its several computational advantages. T he power of this technique will be shown for its capacity of allowing in polynomial time, acting directly on the time series, without embedding dimensions and without combinatorial searches of the delay times, the extraction of all the pseudo periods of a chaotic dynamics, so to make possible a complete statistics of all the cycles, of whatever period, constituting the skeleton of a chaotic attractor. INTRODUCTION: THREE PARADIGMS IN MODELING NEURAL NETS AS DYNAMIC SYSTEMS We can distinguish among three different classes of dynamic systems in modeling neural networks: 1. Convergent one: every trajectory converges asymptotically to some equilibrium; IRAFS International Research Area on Foundations of the Sciences and PULReLab PUL Research Laboratory, Pontifical Lateran University, Rome. 2 Antonio Luigi Perrone 2. Oscillatory one: every trajectory is asymptotic to a periodic (perhaps stationary) orbit; 3. Chaotic one: 'most' trajectories do not tend to periodic orbits. A lthough the biological systems practically never display a convergent dynamics, nevertheless the great majority of neural net models are convergent because of the exceeding difficulty in studying the opposite cases (HIR SCH 1989) Hence, limiting ourselves to chaotic models that are our actual center of interest, we want to discuss in this paper from the cognitive standpoint the main questions related to the informational and computational use of unstable (chaotic) and non stationary dynamics. Indeed, if nature makes use so largely of nonstationary and unstable dynamics in biological realm, evidently some intrinsic informational value is hidden in this strategy. SOME E V IDE NCE OF OSCIL L A TOR Y AND CHAOTI C B E HAV IOR S IN NE UR A L SY STE MS Since 70's many experimental studies were devoted to unveil the presence of chaotic dynamics both in the behavior of single neurons (see (HOL DE N 1986) for a review) and of neuron arrays in the sensomotory cortex of mammalians. L imiting ourselves to the more interesting studies on chaotic behaviors of neural systems, before all W. Freeman and his collaborators demonstrated the presence of chaotic activity in the olfactory system of rabbit (FR E E MAN 1987; SK AR DA FR E E MAN 1987; E ISE NB E R G 1989; Y AO FR E E MAN 1990) and in the visual cortex of primates (FR E E MAN V AN DIJK 1987) According to these studies, the output of olfactory bulb displays in the rest state a chaotic basis with a very large dimensionality, almost noisy. In the presence of a learned odor, the behavior displays an oscillatory transient of large amplitude. Hence the behavior 'converges' toward a coherent dynamics spatially and temporally almost periodical. On the contrary, if the odor is new, the bulb activity becomes chaotic with a low dimensionality (Correlation Dimension (CD) 4 with an average frequency 40 Hz) As I T suda suggests (T SUDA 1992) the chaos seems to work as a "novelty filter" in this case. Following the already quoted T suda' s review, another series of studies about the presence of chaotic behavior in mammalian brains were published by A B abloyantz and her collaborators (B AB L OY ANTZ 1986; B AB L OY ANTZ DE STE X HE 1986; DE STE X HE 1988) about the presence of chaotic behaviors in human E E G during sleeping and epilepsy states. Other important experimental evidences in this connection are illustrated in many studies devoted to enhance the presence of stable A New Approach to Chaotic Systems Characterization and Its Implications for Biology 3 and unstable oscillatory behaviors in visual cortex for the definition of mechanisms of coding, integration and storing of information on a temporal basis. For instance, there is evidence of synchronization behaviors among homologous neurons (i.e. devoted to process the same class of visual stimuli) located in different columns even very distant among them (DINSE 1990; ECKHORN 1988; 1990; ENGEL 1990a.b; GRAY 1989; 1990) What Tsuda rightly emphasizes is that the chaotic behavior of the neural systems generally display a massively not uniform character of the statistical density of the orbits with phenomena of statistical resonance. This evidence has been interpreted by Tsuda himself in the framework of the more general phenomenon of the so called "chaotic itinerancy" i.e. the capacity of some complex systems to pass through different chaotic or more stable attractors of their dynamics by changing some control parameters of the dynamics itself. From the cognitive standpoint, these complex phenomena has been generally interpreted as evidence of the presence of dynamic mechanisms in the brain of memory storage and of pattern integration. On the other hand, the absence in the mammalian brain of any "grandmother cell" i.e. of any spatial mechanism for the information storing and integration, requires the presence of some dynamic mechanism on a temporal basis for these essential cognitive tasks. Indeed, both the phenomenon of the so called " functional segregation" of neurons also at the highest layers of the associative cortex, for instance, in the visual system (Livingstone Hubel 1988) i.e. the presence of neurons specialized for processing only one class of visual stimuli (e.g. colors, orientation, movement, etc. of the visual object and the amazing amount of information that each brain must store and process, require necessarily some dynamic mechanism for the semantic integration of the perceptual representation of the external object and for information storing. In this direction, we want to recall many works quoted by Tsuda (SHIMIZU 1985; KOERNER 1987; GRAY 1989; DINSE 1990) as well as many works of Russian neurophysiologists devoted to modeling through the paradigm of the oscillatory networks the neural mechanisms for the dynamic integration of information between the limbic system and the sensomotory cortex (KOVALENKO 1984; KIRILLOV 1986; BORYSIUK KIRILLOV 1992; BORYSIUK 1992) SOME CURRENT INTERPRETATIONS OF THE ROLE OF CHAOS IN NEURAL SYSTEMS The common background of these interpretations To understand the more common interpretations about the role of chaos in neural information processing as far as it is modeled via the neural network paradigm, it is necessary to illustrate the common physical framework in which 4 Antonio Luigi Perrone these interpretations are generally inserted. The theoretical interest among physicists during the last ten years for studying chaotic behaviors in neural network models has been stimulated by a work of G. Parisi (PARISI 1986) He proposed the possibility of using a multiplicity of complex dynamic behaviors such as limit cycles and chaotic attractors in neural nets by the asymmetrical coupling of neurons in a classical Hopfield net (HOPFIELD 1982) These behaviors have been systematically studied in classical simulation works such as (SOMPOLINSKY 1988; BAUER MARTIENSSEN 1989) So, according to Amit (AMIT 1986) while a "stable" net such as a classical Hopfield net has the tendency to reinforce patterns already learned instead of learning new ones, a chaotic net would could help the learning of new patterns. In fact, in a Hopfield net, recognition occurs only when the input is sufficiently close to one of the stable (fixed point) attraction basins of the net dynamics each corresponding to a memorized pattern. If this does not occur and/or the number of the memorized patterns approximate its critical value a 0.14) where a is the ratio between the number N of the net neurons and he number p of the memorized patterns the net falls in a "confused" state, i.e. it falls in a stable oscillatory state (limit cycle) among two or more fixed points. Moreover, also for patterns very close to the memorized patterns but in presence of noise, the behavior of the net can become metastable. I .e. it begins to oscillate in a stable state very far from the right one. T he suggestion of Parisi is to make these confused states of the net "time dependent" that is "unstable" or chaotic, so to give the net a dynamic tool (i.e. the distinction between state and unstable states) to distinguish between adequate (stable states) and confused (chaotic, unstable states) in its recognition tasks. In the original Parisi model, this is obtained by adding to the classical Hopfield net with symmetrical weights in which the learning occurs, a second set of asymmetrical weights with no learning. When some input is presented to the net, it begins a search in memory. If it fails, the net automatically switches to the chaotic behavior so to prevent the learning of this input. Practically, the chaos is used by Parisi as a "novelty filter" Several works were inspired by Parisi original idea. Perhaps the more elegant one was proposed by T suda himself (T SUDA 1992) Always following the Parisi idea to separate between a sub net with symmetrical couplings, even though very different from an Hopfield one, and a sub net with asymmetrical couplings, T suda net is able to display a richer behavior. Indeed, chaos is used not only as a "novelty filter" but it is able also to display dynamic links of chaotic type among several stable behaviors of the first sub net in which the information is stored. In other terms, chaos is used also to make easier and faster the searching among different memorized patterns. In this way,