reaction - diffusion systems are important models for various nonlinear processes in nonequilibrium physics and biological systems  .turing first proposed a reaction - diffusion model to explain pattern formation in developmental biology  .various pattern formation has been studied in theoretical models  .there is a characteristic wavelength in the turing pattern .kondo and asai observed that the number of stripes in the skin pattern of a tropical fish increases to keep the characteristic wavelength constant , when the fish grows with time  . in the skin pattern ,the periodic pattern is constructed of pigment cells with different colors .the different types of pigment cells are mutually competitive .on the other hand , there are several observations that various types of cells are differentiated during the developmental process , but the number ratio of the different cell types does not change very much as the body size increases .a typical example is the cellular slime mold dictyostelium discoideum .the amoebic state changes to the fruiting body via the migrating slug state , if the breeding condition becomes worse .prespore cells and prestalk cells are differentiated in the process .the number ratio is kept almost constant even if the body size is changed  .the ratio control and the pattern formation are performed in two steps . in an early stage ,the ratio of the two - type cells is regulated by dif ( differentiation inducing factor ) , and later prestalk cells move into a tip region within the slug through a chemotaxis by the camp  . the two cell types , i.e. , the prespore and the prestalk are competitive in this system .another example is the differentiation of blood cells from the stem cells in the bone marrow .the number ratio of the red blood cells , the white blood cells , and the platelets is kept to be roughly constant .another typical rule of the cell differentiation is a cascade control of the differentiation process by complicated gene networks . in the segmentation process of the drosophila, the differentiation proceeds from a large scale to a small scale in a hierarchical manner .a cascade network of genes and proteins such as bicoid protein , gap genes and pair - rule genes are identified in detail  .several competitive relations between two genes are known also in this process .for example , the gene engrailed ( en ) for the posterior compartmental specification and the gene wingless ( wg ) for the anterior compartment are mutually competitive .several theoretical models of the hierarchical gene network were proposed for the segmentation process  .the analyses of the specific gene network and the pattern formation for each system are important in the developmental biology . however , in this paper , we consider a very simple cascade model of competitive reaction - diffusion equations and propose a mechanism of the ratio control from a view point of the nonlinear dynamics .   for eq .( 2 ) at and ( solid curve ) , ( dashed curve ) , and ( dotted curve ) . ( b )stationary profiles of ( solid curve ) and ( dashed curve ) at and ., width=340 ]   for eq .( 4 ) at and ( solid curve ) , ( dashed curve ) , and ( dotted curve ) . ( b )stationary profiles of ( solid curve ) and ( dashed curve ) at .( c ) time evolution of ( solid curve ) and ( dashed curve ) from and . , width=529 ]   ( solid curve ) and ( dashed curve ) at and for eq .( b ) size ratio of the -dominant region as a function of for and .the rhombi denote numerical results and the dashed curve is ., width=340 ]if two cell - types are mutually competitive , only one type of cells are locally activated by the lateral inhibition .the two cell - types are bistable , and a homogeneous state of one - type cell will appear . however , if some long - range enhancement of the competitive type is included in the system , two types of cells will appear in a large system .meinhardt and gierer proposed a model of the mutual induction of locally exclusive states , and generated a stripe - like pattern  .we study first a simplified version of their model .the model is written as coupled reaction - diffusion equations : where are positive parameters , and obey the hill type equation , and are produced respectively from and .the hill type equation is often used in the biochemistry and the gene regulatory network  .the enhancement and the suppression of the reaction are expressed in the numerator and the denominator in the first term of on the right - hand side of the hill equation .that is , both and acts as activators for themselves by the terms of or , and acts as inhibitors for the other by the term of and .the two components and compete with each other when is sufficiently large .the terms and in the denominators express the effect of the saturation .we assume that and correspond to proteins ( morphogens ) determining competitive cell types such as the pigment cells of different colors in the skin pattern , or the prestalk and the prespore in the slime mould .the term of and implies that and are activators respectively for and .we assume that .it means that the morphogens and diffuse rapidly and work as long - range activators respectively for and .  there is a uniform solution to eq .it is expected that is depressed when increases locally and vice versa , because and are mutually competitive . if the perturbation of the form : are assumed owing to the competitive relation , the deviations and satisfy the eigenvalue can be calculated from eq .figure 1(a ) displays for ( solid curve ) , ( dashed curve ) and ( dotted curve ) at and .( the parameter values and are not realistic biological ones , but we study the nonlinear system as just a model system in this paper . ) the turing type instability is expected for a finite wavenumber . we have performed a one - dimensional numerical simulation . the system size , and the periodic boundary conditions are imposed .figure 1(b ) shows stationary profiles of ( solid curve ) and ( dashed curve ) at and . a spatially periodic pattern appears , in which the -dominant region and -dominant region reciprocate spatially . this is a turing pattern in the competitive reaction - diffusion systems .note that the component with large diffusivity acts as activators for the competitor , and it facilitates the competitor in spatially distant regions .it is different from the usual turing pattern in usual activator - inhibitor systems , where the inhibitor with large diffusivity inhibits the activator .the mechansim of the pattern formation by the long - range activation of the competitor is essentially the same as the model proposed by meinhardt and gierer almost 30 years ago  .kondo and asai performed a numerical simulation of the usual type of activator - inhibitor system to explain the skin pattern of the tropical fish  , but our model might be more plausible for the pattern formation , because the pigment cells are mutually competitive .next , we generalize the model equation ( 1 ) to a model equation in which the whole system is separated into two regions of -dominant region and -dominant region .if is infinitely large and a steady state is obtained for the component , eq .( 1 ) is reduced to where is the spatial average of , i.e. , in a one - dimensional system and in a two - dimensional system .it is because becomes uniform owing to the infinitely large diffusivity of in eq .this is our original model . for this model equation , the small deviation from the uniform solution satisfies for , because is assumed to be zero owing to the uniformity of . on the other hand , for , fugure 2(a )displays the eigenvalues for ( solid curve ) , ( dashed curve ) and ( dotted curve ) at .the eigenvalue at is negative for these parameter values , therefore , the fourier mode with is stable .the uniform solution is unstable for and .the eigenvalue takes the largest value for the smallest wave number .figure 2(b ) shows a stationary solution for for the one - dimensional system with periodic boundary conditions of size .the initial condition was and .for this initial condition , the whole space is separated into the -dominant region at and the -dominant region at .there are domain walls between the two domains .however , even if the initial condition is random , the whole space is separated into the two regions with the same size , although the position of the -dominant region becomes random .it is because takes a maximum value at .if the initial condition is and , four domains , i.e. , two -dominant regions and two -dominant regions appear initially , however , there is an attractive interaction between the domain walls and finally the two - domain structure is obtained as shown in fig.2(c ) .the time evolution is similar to the coarsening process in the one - dimensional ginzburg - landau ( gl ) equation :  .however , it is different from the gl system in that the final state is a uniform state in the gl system and the final state is a two - domain state in our system .it is a unique point in our model that the two - domain structure of equal size appears naturally for any system size .equation ( 3 ) can be generalized into an asymmetric model : where and are assumed to take different values .figure 3(a ) diplays stationary profiles of ( solid curve ) and ( dashed curve ) at and for .the sizes of the -dominant region and the -dominant region are not the same in this asymmetric model . however , the maximum and the minimum values of and and the width of the domain walls are almost the same , which are expressed as and .if and are assumed to takes constant values and , a domain wall between a -dominant region and a -dominant region moves as the -dominant region shrinks , which is similar to the motion of the domain wall in the asymmetric ginzburg - landau equation : .however , if the -dominant region shrinks , decreases and increases in time in our system .this is due to a negative feedback effect involved in our system .finally , the domain walls become stationary , when is satisfied .if the system size is sufficiently large , the ratio of the domain size of the -dominated region is evaluated from the condition as because and , if the width of domain walls is not taken into consideration . if , is approximated at .thus , the size ratio is determined by the ratio of the parameters and and does not depend on the system size .we studied the control of the domain size in the ginzburg - landau type equation in the previous work , which was the same mechanism as the present one  .figure 3(b ) displays numerically obtained size ratio s as a function of at , and for .the dashed curve denotes .fairly good agreement is seen .the ratio control can be attained by the choice of the parameters and .this is a mechanism of the ratio control of our differentiation model .the feedback effect via the -variable with infinitely large diffusivity controls the ratio of the different cell types . in this asymmetric model ,a two - domain structure appears naturally , and the ratio of the domain size is uniquely determined by the system parameters .it might be applicable to the ratio control in the early stage of the slime mold .( ) in the cascade model similar to eq .( 8) in a one - dimensional space .the system is composed of fourteen elements in three layers .the system size is .the parameter values are and .( c ) stationary profiles of ( ) at and ., width=529 ]next , we construct a new cascade model of the competitive reaction - diffusion equations as shown in fig .4(a ) based on eqs . ( 3 ) and ( 6 ) , although another cascade model based on a more complicated version of eq .( 1 ) was proposed by meinhardt to study the hierarchical subdivision into gap - gene , pair - rule gene and segment polarity gene in the drosophila  . in fig .4(a ) , a system of fourteen elements in three layers is shown .each element denotes a protein ( morphogen ) with number which determines the cell type .the horizontal dashed lines represent the competitive interaction between two elements as in the previous model , and the solid lines from the upper layer to the lower layer represent an active interaction from a upper - level element to a lower - level element .a negative feedback from the lower - level to the upper level might be important , but the feedback effect is not considered in our model for the sake of simplicity .the active and competitive interactions are represented by the hill type equations .for example , a model equation for a system of six elements in two layers is written as where is the coupling constant of the active interaction from the upper layer to the lower layer .owing to the competitive interaction , is almost zero in the -dominant region . at the domain , and are also almost zero because and are activated by . on the other hand ,the -dominant region is separated into the -dominant region and the -dominant region because of the competitive interaction between and .the size ratio of the two regions are also determined by the ratio of and .we have performed numerical simulations of a system of fourteen elements in the three layers .it is a one - dimensional system with system size .figure 4(b ) displays stationary profiles of ( ) for the three layer system at and .firstly , the whole space is separated into the two domains : a -dominant region and a -dominant region .next , the -dominant region is separated into the two domains : a -dominant region and a -dominant region , and the -dominant region is separated into the two domains : a -dominant region and a -dominant region . similarly , the -dominant region is separated into -dominant and -dominant regions , the -dominant region is separated into -dominant and -dominant regions , the -dominant region is separated into -dominant and -dominant regions , and , the -dominant region is separated into -dominant and -dominant regions as a result , a hierarchical pattern appears .the domain size decreases as and 1/8 as the layer is lowered .this is a kind of binary - tree decomposition of the whole space . if the parameter is assumed to take a different value for each layer , and is set to be for the layer , peak values of take almost the same value for each layer , because in the layer . for the well tuned parameter values of , a self - similar binary decomposition of dominant regions will occur .if , a multi - fractal - like pattern appears , because the size ratio is in the first layer , in the second layer , and in the third layer .figure 4(b ) displays a stationary profiles of ( ) for and at and .inhomogeneous decomposition of dominant regions is clearly seen .-dominant regions ( ) are depicted with different colors in a square of size .the parameter values are and at and .a hierarchical structure is clearly seen .the colored regions imply that satisfies .( a ) -dominant and -dominant regions .( b ) , , , and -dominant regions .( c ) , , , , , , , and -dominant regions .the number indicates the cell type . ,width=529 ]   of -dominant regions satisfying for and 8 . ,width=188 ]   dominant region satisfying for and 6 at ( a ) , ( b ) , and ( c ) . the snapshot at is shown in fig .5(b ) . the number indicates the cell type ., width=453 ]  finally , we have performed a numerical simulation in two dimensions .the system size is , and the periodic boundary conditions are imposed .the network of the interaction is the same as before in fig .4(a ) , that is , the number of elements are fourteen and the layer number is three .the parameter values are and at and .the initial conditions are random between 0.5 and 0.6 .figure 5(a),(b ) and ( c ) display patterns at which represent the dominant region satisfying for each using different color for ( a ) and 2 , ( b ) , and ( c ) .these states are final stationary states after the transient time evolution as shown later in fig .6 and fig .the whole region is divided into two domains : a -dominant region and a -dominant region .the -dominant region is a circular blue region which is located near the four edges in fig .the area ratio of the -dominant region and the -dominant region is , which is close to .the -dominant region is subdivided into two domains : a -dominant region and a -dominant region .the area ratio of the two regions is , which is also close to the ratio .similarly , the -dominant region is further subdivided into a -dominant region and a -dominant region .the area ratio of the two regions is , which is also close to the ratio .thus , the hierarchical differentiation is clearly seen .and it is shown that the area ratio is determined by the ratio of and .figure 6 displays the time evolution of the number ratio of the -dominant regions for and 8 .the number ratio becomes a stationary values at for and 2 , at for and , at for and 8 .that is , the stationary state is attained sequentially from the upper layer to the lower layer. figures 7(a ) , ( b ) and ( c ) show three snapshot patterns of the -,- , - and -dominant regions at and .the dominant regions are initially small and random , and the size and the location are determined gradually .figure 7(c ) is almost similar to fig .5(b ) , which means that the time evolution is almost stationary at .competitive interactions and hierarchical interactions are typical in gene networks . we have proposed a simple cascade model of competitive reaction - diffusion equations for the cell differentiation .we have first reconfirmed a spatially - periodic pattern in a simple one - dimensional competitive reaction diffusion equations , which was originally proposed by meinhardt and gierer .it is a kind of the turing pattern , but the origin of the formation of the periodic pattern is the long - range enhancement of the competitor .if the diffusion constant for -variable is assumed to infinity , we get a new model , in which a two - domain structure appears naturally from an random initial condition , and the size ratio of the two domains can be controlled by system parameters owing to the negative feedback effect of the domain size .next , we have generalized the model to a hierarchical model .the -dominant regions appear in a cascade manner from the upper layer to the lower layer .the ratio of the -dominant regions are well controlled by the system parameters .our simple cascade model is not a realistic model based on biological experiments , however , it is a useful model to consider the turing pattern , the ratio control , and the hierarchical differentiation in a unified manner .we expect that our model might be applicable to specific cell differentiation processes by changing the reaction network and modifying the system parameters suitably .99 g. nicolis and i. prigogine , _ self - organization in nonequilibrium systems _ ( john wiley and sons , new york 1977 ) .a. turing , phil.trans .r. soc.london b * 327 * , 37 ( 1952 ) .h. meinhardt , _ models of biological pattern formation _ ( academic press , london , 1982 ) .j. d. murray , _ mathematical biology _ ( springer - verlag , new - york,1989 ) .s. kondo and r. asai , nature * 376 * , 765 ( 1995 ) .t. bonner and m. k. slifkin , am .* 36 * , 727 ( 1949 ) .r. r. kay , seminars in cell and devel .* 10 * , 577 ( 1999 ) . y. maeda and m. maeda , exp . cell res . *84 * , 88 ( 1974 ) .s. f. gilbert et al . , _ developmental biology _ ( sinauer , 1997 ) .h. meinhardt , j. cell .suppl . * 4 * , 357 ( 1986 ) .e. mjolsness , d. h. sharp , j. reinitz , j. theor . biol . * 152 * , 429 ( 1991 ) .h. meinhardt and a. gierer , j. theor . biol .* 85 * , 429 ( 1980 ) .a. v. hill , j. physiol .* 40 * , r4 ( 1910 ) .m. b. elowitz and s. leibler , nature * 403 * , 335 ( 2000 ) .k. kawasaki and t. ohta , physica a * 116 * , 573 ( 1982 ) .h. sakaguchi , phys .e * 64 * , 047101 ( 2001 ) .

Abstract: we propose a kind of reaction - diffusion equations for cell differentiation , which exhibits the turing instability . if the diffusivity of some variables is set to be infinity , we get coupled competitive reaction - diffusion equations with a global feedback term . the size ratio of each cell type is controlled by a system parameter in the model . finally , we extend the model to a cascade model of cell differentiation . a hierarchical spatial structure appears as a result of the cell differentiation . the size ratio of each cell type is also controlled by the system parameter .


at its most abstract level , the sp theory is intended to model _ any _ kind of system for processing information , either natural or artificial .the theory is turing - equivalent in the sense that it can model the operation of a universal turing machine but , unlike earlier theories of computing , the sp theory provides an account of a range of phenomena in perception and cognition , including the analysis and production of natural language , ` fuzzy ' recognition of patterns and objects , probabilistic kinds of reasoning , solving problems by reasoning and by analogy , and unsupervised learning .the theory also provides a new perspective on a range of concepts in computing , logic and mathematics .  in work to date , the sp theory has been developed in purely abstract terms without reference to the anatomy or physiology of neural tissue . the main purpose of this paper is to consider possible ways in which the abstract concepts that have been developed within the sp theory may be mapped on to structures and mechanisms in the brain . to the extentthat this is successful , we may achieve a ` neural ' version of the theory called ` sp - neural'that inherits the insights that are provided by the abstract version .  to anticipate a little, it is proposed that ` patterns ' in the sp theory are realised with structures resembling hebb s concept of a ` cell assembly ' .by contrast with that concept :  * it is proposed that any one neuron can belong in one assembly and _ only _ one assembly. however , any given assembly may be ` referenced ' from other assemblies somewhat in the same way that any given web page may be referenced by urls on other web pages .. ] this device provides solutions to many of the problems associated with cell assemblies , it allows information to be stored in a compressed form , and it provides a robust mechanism by which assemblies may be connected to form hierarchies , grammars and other kinds of knowledge structure . * the mechanisms for learning in sp - neural are significantly different from the well - known principles of hebbian learning .however , they are compatible with the observations that hebb s theory was designed to explain and , at the same time , they provide explanations for observations that the hebbian theory is not able to explain .the next section describes the sp theory in outline with just sufficient detail for present purposes .section [ neural_realisation_section ] considers how the elements of the sp theory may be realised with neural mechanisms .section [ discussion_section ] considers a selection of issues that arise in validating the proposals against empirical data and alternative theories .section [ conclusion_section ] summarises explanatory benefits , empirical predictions and future avenues for research .in this section , the main elements of the sp theory are described with many details omitted .the focus is on those aspects of the theory that are relevant to the proposals that follow .the sp theory is founded on two quasi - independent areas of thinking :  * a long tradition in psychology that many aspects of perception and cognition may be understood in terms of information compression ( see , for example , * ? ? ?* ; * ? ? ?* ; * ? ? ?* ; * ? ? ?* ; * ? ? ?* ; * ? ? ?* ; * ? ? ?* principles of _ minimum length encoding _ , pioneered by and others , that focus on the intimate connection that exists between information compression and the inductive prediction of the future from the past ( see also * ? ? ?* ; * ? ? ?* ) .  in broad terms ,the sp theory is conceived as an abstract system or model that works like this .it receives ` new ' data from its environment and adds these data to a body of stored knowledge called ` old ' . at the same time, it tries to compress the information as much as possible by searching for full or partial matches between patterns and merging or ` unifying ' patterns or parts of patterns that are the same .in the course of trying to compress information , the system builds _ multiple alignments _ , as described below .  generally speaking, new information may be equated with sensory information but information within the system may sometimes play the same rle , as described in section [ production_section ] , below .two computer models of the sp system have been developed :  * sp61 is a partial realisation of the theory that builds multiple alignments and calculates probabilities of multiple alignments but does not transfer any information from new to old : all its old information must be supplied by the user .this model , which is relatively robust and stable , is described quite fully in .* sp70 realises all the main elements of the theory , including the transfer of information from new to old .this model , and its application to unsupervised learning , is described in .more work is required to realise the full potential of this model .all information in the system is expressed as arrays or _patterns _ of symbols , where a _ symbol _ is simply a ` mark ' that can be compared with any other symbol to decide whether it is the ` same ' or ` different ' . in work done to date, the focus has been on one - dimensional strings or sequences of symbols but it is envisaged that , at some stage , the ideas will be generalized to patterns in two dimensions .this very simple ` universal ' format for knowledge has been adopted with the expectation that it would facilitate the representation of diverse kinds of knowledge and their seamless integration . notwithstandingthe simplicity of the format , the processing mechanisms provided within the sp system mean that it is possible to model such things as grammatical rules ( with ` context - sensitive ' power ) , if - then rules , discrimination networks and trees , class - inclusion hierarchies ( with inheritance of attributes ) , and part - whole hierarchies .an example will be seen in section [ ma_section ] , below , and others may be found in the sources cited above .  another motive for adopting this format for knowledge is the observation that much of our knowledge derives ultimately from sensory inputs , especially vision , and most sensory inputs map naturally on to sequences or two - dimensional arrays .this idea sits comfortably with the observation that the cortex is , topologically , a two - dimensional structure and that there is a relatively direct mapping between the retina and each of the areas of the visual cortex , between the skin and the somatosensory cortex , and likewise for other senses .it seems reasonable to suppose that this kind of mapping is a general feature of the way the brain handles information .the main elements of the multiple alignment concept as it has been developed in this research are illustrated in the example presented here .for the sake of clarity and to save space , this example is relatively simple .however , this should not be taken to represent the limits of what the system can do .more complex examples may be found in and the other sources cited above .  in any realistic case ,the number of possible alternative alignments is far too large to be searched exhaustively .it is necessary to use heuristic methods that prune away large parts of the search space , trading accuracy for speed .the sp61 and sp70 models both use forms of ` hill climbing ' with measures of compression to guide the search .  given a new pattern representing the sentence ` t h e c a t s l e e p s ' and a set of old patterns representing grammatical rules , the sp system builds multiple alignments like the one shown in figure [ alignment_1 ] .the aim is to create multiple alignments that allow the new pattern to be encoded economically in terms of the old patterns as described in section [ evaluation_of_alignments ] , below . out of the several alignments that sp61 has built in this case ,the one shown in figure [ alignment_1 ] is the best .  .... 0t h e           c a t             s l e e p  s  0              | | |           | | |             | | | | |  |  1             | | |           | | |        < vstem 2 s l e e p > |  1              | | |           | | |        |  |        | |2             | | |           | | |    < v vs < vstem       > s > 2              | | |           | | |    | | |              | 3 s num   ; < np    | | |           | | |   > < v |              > 3    |    | | |     | | |           | | |   |   |               4   |    | | |     | | |< nstem 3 c a t >  |   |               4    |    | | |     | | |      |  |      |  |   |               5   |    | | |     | | |  < n ns < nstem     > > |   |               5    |    | | |     | | |  | | |           | |   |               6   |    | | |  < d 0 t h e > | | |           | |   |               6    |    | | |  | |     | | | |           | |   |               7   |    | < np < d     > < n |           > >   |               7    |    |             |               |               8  num sng ;             ns              vs              8 ....  by convention , the new pattern is always shown in row 0 of any alignment , as can be seen in figure [ alignment_1 ] .the old patterns are shown in the rows below the top row , one pattern per row .the order of the old patterns across the rows is entirely arbitrary and without any special significance .  a pattern like ` np d   ' in row 7 of the alignment expresses the idea that a noun phrase ( ` np ' ) is composed of a determiner ( ` d ' ) followed by a noun ( ` n ' ) . in a context - free phrase - structure grammar , this would be expressed with a rule like ` np d n ' .if we ignore row 8 , the whole alignment may be seen to achieve the effect of a context - free parsing , dividing the sentence into its constituent words , identifying ` t h e c a t ' as a noun phrase , marking each word with its grammatical class , and , within the verb ` s l e e p s ' , marking the distinction between the stem and the suffix .ignoring row 8 , the alignment in figure [ alignment_1 ] is equivalent to the tree - structured parsing shown in figure [ tree_parsing_figure ] .although some of the patterns in the alignment are similar to rules in a context - free phrase - structure grammar , the whole system has the expressive power of a context - sensitive system .this is illustrated in row 8 of the alignment where the pattern ` num sng ; ns vs ' marks the ` number ' dependency between the singular noun in the subject of the sentence ( ` c a t ' ) and the singular verb ( ` s l e e p s ' ) . a basic context - free phrase - structure grammar , without augmentation , can not handle this kind of dependency .   within each pattern in old , there is a distinction between _ identification _( i d ) symbols and _ contents _ ( c ) symbols .the former serve to identify the pattern or otherwise define its relationship with other patterns , while the latter represent the contents or substance of the pattern .for example , in the pattern ` np d  n ' in row 7 of figure [ alignment_1 ] , the id - symbols are the initial left bracket ( ` ' ) , the symbol ` np ' that follows it and the terminating right bracket ( ` ' ) .all other symbols in the pattern are c - symbols . in the pattern ` nstem 3 c a t ' , the id - symbols are the initial and terminal brackets , as before , plus the symbols ` nstem ' and ` 3 ' , while the c - symbols in that pattern are ` c ' , ` a ' and ` t ' . in the pattern ` num sng ; ns vs ' , the first three symbols are id - symbols and the last two are c - symbols.  in sp61which does not attempt any learning the old patterns are supplied to the system by the user and each symbol within each old pattern is marked by the user to show whether it is an id - symbol or a c - symbol . in sp70which creates patterns and adds them to old in the course of learning the distinction between id - symbols and c - symbols within any one pattern is marked by the system when the pattern is created .   a combination of id - symbols , derived from an alignment like the one shown in figure [ alignment_1 ] , can provide an abbreviated code for the entire sentence pattern ( or other pattern ) held in new .a ` good ' alignment is one where this code is relatively small in terms of the number of bits of information that it contains .the ` compression score ' for an alignment is the difference between the size ( in bits ) of the new pattern in its original form and its size after it has been encoded .  the procedure for deriving an encoding from an alignment is quite simple: scan the alignment from left to right looking for columns containing a single instance of an id - symbol , not matched to any other symbol .the encoding is simply the symbols that have been found by this procedure , in the same order as they appear in the alignment .the encoding derived in this way from the alignment in figure [ alignment_1 ] is ` s sng 0 3 2 ' .this is smaller than the new pattern in terms of the number of symbols it contains .it is even smaller when the size of the new pattern and the size of the code are measured in terms of the numbers of bits they contain , as described in .the ` compression score ' or ` compression difference ' for an alignment is : where is the size ( in bits ) of the new pattern in its original form and is its size in bits after it has been encoded .an attractive feature of the sp system is that , _ without any modification _ , it can support the production of language ( or other patterns of knowledge ) as well as its analysis .if sp61 is run again , with the sentence in new replaced by the encoded form of the sentence ( ` s sng 0 3 2 ' ) as described in section [ evaluation_of_alignments ] , the best alignment found by the system is exactly the same as before except that row 0 contains the encoded pattern and each symbol in that pattern aligned with matching symbols in the rows below .the original sentence has , in effect , been recreated because the alignment contains the words of the sentence in their correct order .this is an example of the possibility noted near the beginning of section [ sp_theory_section ] ) where the rle of ` new ' information is played by information within the system rather than by sensory data .it is envisaged that the production of sentences from meanings may be modelled in a similar way . instead of using a code pattern like `s sng 0 3 2 ' to drive the production process , some kind of semantic structure may be used instead .the sp system as a whole is a system that learns by assimilating ` raw ' information from its environment and distilling the essence of that information by a process of information compression .the main elements of the theory are now realised in the sp70 computer model .this model is able to abstract simple grammars from appropriate data without any kind of external ` teacher ' or the provision of ` negative ' samples or the grading of samples from simple to complex ( cf . * ? ? ?* ) . in short ,it is an _ unsupervised _ model of learning .some reorganisation is needed to overcome certain weaknesses in the model as it stands now .although some reorganisation is needed to overcome certain weaknesses in sp70 as it stands now , the overall structure appears to be sound . in the model ,learning occurs in two stages :  1 .  _creation of old patterns_. as the system receives new patterns , a variety of old patterns are derived from them as explained below .some of these patterns are ` good ' in terms of principles of minimum length encoding but many of them are ` bad ' .selection of ` good ' patterns_. by a process of sifting and sorting through the old patterns , the system abstracts one or more subsets , each one of which is relatively good in terms of the principles of minimum length encoding .the patterns in the first one or two of these subsets may be retained and the remaining patterns may be discarded .it is envisaged that , in future versions of the model , these two processes creation of old patterns and selection amongst them will be repeated many times while new patterns are being received so that the system can gradually bootstrap a set of old patterns that are relatively useful for the economical encoding of new data .  to get the flavour of the way in which old patterns are created ,consider a simple example .if the current pattern in new is ` t h e b o y r u n s ' and the repository of old patterns is empty , the system discovers that there is no way to encode the new pattern economically in terms of old information so it augments the new pattern with i d symbols ( which converts it into ` % 1 t h e b o y r u n s ' ) and adds the augmented pattern to old .when old has accumulated a range of patterns like this , it can begin to create multiple alignments . if , at that stage , the current pattern in new is ` t h e g i r l r u n s ' , multiple alignments created by the system will include one like this :  .... 0    t h e g i r l r u n s  0     | | |     | | | |  1< % 1 t h e b o y  r u n s > 1 ....  from this alignment , the system can derive some additional old patterns by extracting coherent sequences of matched symbols and unmatched symbols , adding system - generated id - symbols , and creating another pattern that ties everything together .  in this case, the result is five patterns like this :  .... < % 2 t h e > < % 3 r u n s > < % 4 0 b o y > < % 4 1 g i r l > < % 5 < % 2 > < % 4 > < % 3 > > ....  the first four of these patterns represent words while the last one is an ` abstract ' pattern that describes the overall structure of two original sentences in terms of id - symbols in the lower - level patterns .notice how ` b o y ' and ` g i r l ' have been assigned to the same class ( identified by the symbol ` % 4 ' ) , very much in the tradition of distributional linguistics ( see , for example , * ? ? ?* ; * ? ? ?notice also how the system picks out entities that we naturally regard as ` discrete ' ( see also * ? ? ?* ; * ? ? ?* ; * ? ? ?similar principles may explain how we learn to see the visual world in terms of discrete objects .the flexible pattern matching that is built into the process of finding multiple alignments means that sp70 can find correlations that bridge arbitrary amounts of intervening structure .it is not necessary to restrict the learning process to second- or third - order correlations ( _ cf ._ ) .apart from the parsing and production of language and the unsupervised learning of knowledge , the sp system has proved to be a versatile model of several other aspects of perception and cognition :  * _ fuzzy pattern recognition and best - match retrieval of information_. the system provides a model for the way we can recognise things despite errors of omission , commission and substitution and the way we can retrieve information from fragmentary clues . * _ recognition at multiple levels of abstraction_. given appropriate patterns in its store of old information , the system can be used to model the recognition of things at several different levels of abstraction simultaneously .some unknown entity may be recognised as an individual , ` tibs ' , and at the same time it may be recognised as a cat , a mammal , a vertebrate and an animal ( see also the example described at the end of section [ sharing_section ] ) . *_ probabilistic and exact forms of reasoning_. the system supports probabilistic ` deduction ' , abduction , nonmonotonic reasoning and chains of reasoning .it also provides a model for classical ` exact ' forms of reasoning . *_ solving problems by reasoning and by analogy_. given appropriate patterns , the system can do such things as finding a route between two places or solving problems by analogy .further information may be found in and earlier publications cited there .the sp system expresses a pair of ideas that are fundamental in the representation and processing of knowledge and are one of the corner - stones of the proposals in this paper :  * _ identification_. any entity or concept may be given a ` name ' , ` label ' , ` code ' , ` tag ' or _ identifier _ by which it may be identified .examples in everyday life include the name of a person , town , country , book , theorem , poem , type of animal or plant , period of history , and many more .+ the concept of identification is not restricted to unique identification of specific things .it also applies to the identification of _ classes _ of things like ` furniture ' , ` people ' , ` animals ' and so on * _reference_. whenever we wish to refer to any such entity or class , we can do so by means of a _ copy _ of the relevant name , label or identifier .every copy is a _ reference _ to the given entity or class and in each case there may be many such references .examples in everyday life include referring to someone by their name , or referring to a town , country , book etc by its name . in a paper like this one , ` ' is a reference to the bibliographic details at the end of the paper and those details are themselves a reference to the book itself .being able to refer to any concept by a relatively brief name is a great aid to succinct communication .imagine how cumbersome and difficult things would be if we had to give a full description of everything and anything whenever we wanted to talk about it like the slow language of the ents in tolkien s _ the lord of the rings_. identification and reference are a powerful aid to the compression of information and it should not surprise us to find that the same pair of ideas lies at the heart of computer - based ` zip ' programs for compressing information .  in the sp system , any given pattern has an identifier in the form of one or more id - symbols .one or more copies of those symbols in other patterns or in the same pattern serve as references to that pattern . in figure[ alignment_1 ] , the identifier for the pattern ` np d  n  ' in row 7 are the symbols ` ' and ` np ' at the beginning of the pattern and the symbol ` ' at the end . a reference to that pattern appears as ` np ' in the pattern ` s num ; np  v ' in row 3 . in a similar way , ` % 2 ' and ` % 3 ' in the pattern ` % 5 % 2  % 4  % 3  ' from section [ sp70_section ] may be seen as references to ` % 2 t h e ' and ` % 3 r un s ' , respectively , and ` % 4 ' is a reference to the class \{` % 4 0 b o y ',` % 4 1 g i r l '}.this section describes how the sp concepts may be realised in terms of neural structures and processes .much of the discussion in this section relates to particular domains mainly vision and language but it should be emphasised that the proposals are intended to apply to sensory data and knowledge of all kinds visual , auditory , tactile etc both individually and in concert .  in this section and the rest of the paper , the focus will be on the analysis of sensory data rather than the production of sentences or other patterns of knowledge . in every figure, arrows on neural connections will show the direction of flow of sensory signals . a full discussion of the way the system may model the creation or production of motor patterns is outside the scope of this paper . in brief, it is envisaged that codes or ` meanings ' may take on the rle of new information as described in section [ production_section ] and they may serve to create neural analogues of multiple alignments by precisely the same processes that are described in section [ creating_neural_mas ] , below .   apart from the kinds of ` internal ' data just mentioned , new information in the sp system corresponds to sensory data .the discussion in this subsection is mainly about vision but similar principles seem to apply to other sensory modalities .  in mammalian vision ,patterns of light entering the eye are received by the retina and transmitted via the lateral geniculate body to layers 3 and 6 of the visual cortex and beyond .although sensory information is received initially in analogue form , it is widely accepted that the information is converted at an early stage into something like david marr s concept of a ` primal sketch ' . in the early stages ,sensory data is converted into the digital language of on - centre cells , off - centre cells and the like , and in later stages it is interpreted in terms of digital features such as lines at particular angles , motion in a particular direction , and so on ( see , for example , * ? ? ?* ; * ? ? ?each neuron in the cortex that responds to a particular feature of the sensory data may be regarded as the neural equivalent of an sp symbol and we shall call it a _neural symbol_. an array of neural symbols that registers a sensory pattern in any sensory cortex we shall call a _ receptor array_.whatever its three dimensional configuration , we shall assume that it is topologically equivalent to a one - dimensional sequence or a two - dimensional array .a receptor array equates with the part of the sp system that receives new patterns from the system s environment and passes it on for processing within the system .notice that , if a receptor array is to respond to a wide variety of patterns which is clearly true of the visual cortex and other sensory cortices then the complete ` alphabet ' of neural symbols must be available at every location within the array and thus repeated many times across the receptor array as shown schematically in figure [ receptor_array_figure ] . in accordance with this expectation , it has been found that with the type of neuron that responds selectively to a short line segment at a particular angle the complete range of orientations is repeated across the cortex within each of a large number of fairly small ` orientation columns ' .it seems likely that a similar organisation may exist in the somatosensory cortex , mirroring the way in which receptors that respond selectively to heat , cold , touch , pressure and so on are repeated across areas of the skin ( * ? ? ?* chapter 18 ) .so far , we have assumed that a new pattern arrives all at once like an image projected on to the retina .but it is clear that streams of sensory information are being received in all sensory modalities throughout our lives and that we are very sensitive to patterns in the temporal dimension within those streams of information , especially in hearing .it is assumed here that temporal sequences in hearing , vision and other modalities is captured spatially in arrays of neurons , probably in the cortex .an old pattern in the sp system may be realised by a network of interconnected cells in the cortex , similar to donald hebb s concept of a _ cell assembly _ ( see also * ? ? ?because there are significant differences between the present proposals and hebb s original concept , the neural realisation of an old pattern will be referred to as a _ pattern assembly_. those differences are described in this subsection and ones that follow , and they are further discussed in section [ discussion_section ] .an sp _ symbol _ in an old pattern may be realised within a pattern assembly by a single neuron or , perhaps , a small network of interconnected neurons . to simplify discussion in the rest of this paper, we shall assume that every symbol is realised by a single neuron and , as before , we may refer to any such neuron as a _neural symbol_. a neuron within a pattern assembly that corresponds to a c - symbol within an sp pattern will be called a ` c neural symbol ' or , more simply , a _c - neuron_. likewise , a neuron that represents an id - symbol will be called an ` i d neural symbol ' or an _ id - neuron_.  as with receptor arrays , it is assumed that , regardless of the three - dimensional configuration of the neurons in pattern assemblies , each such assembly is topologically equivalent to a sequence of neurons or a two - dimensional array of neurons .unlike the hebbian concept of a cell assembly , it is envisaged that the neurons in any one pattern assembly lie close together within the cortex .it is envisaged that the connections between neurons will be two - way excitatory connections mainly between immediate neighbours but there may also be excitatory connections between neural symbols that are a little more distant . in general , all the interconnections amongst neurons in a pattern assembly will be relatively short. however , there will be connections between pattern assemblies as described in section [ connections_section ] and those connections may be very much longer .the suggested configuration of pattern assemblies just described is consistent with braitenberg s suggestion ( pp .181 - 182 ) that there is strong correlation of neural activity within each 1 mm column in the human cortex and that each column may be connected to any other column .we may suppose that each pattern assembly is confined within one column but connections between pattern assemblies may span the whole cortex .as with sp patterns , pattern assemblies may be roughly graded from ` low level ' to ` high level ' .low level pattern assemblies are those representing small perceptual details such as formant transitions and allophone in the case of speech or small perceptual ` motifs ' such as corners of objects , textures , or colour combinations in the case of vision .high level pattern assemblies would represent abstractions like the ` sentence ' pattern in row 3 of figure [ alignment_1 ] or high - level classes like ` the world ' , ` people ' or ` the animal kingdom ' . as with sp patterns ,the gradation is not a strict hierarchy because any given pattern assembly may have a direct connection to any other pattern assembly , not necessarily ones that are immediately ` above ' or ` below ' .it is envisaged that each neural symbol within a receptor array sends signals to one or more c - neurons in one or more pattern assemblies .most of these would be pattern assemblies that may be classified as low level but in principle any pattern assembly may receive signals from a receptor array .connections leaving the neural symbols of a receptor array are shown at the top of figure [ receptor_array_figure ] .each c - neuron within each pattern assembly may receive signals from one or more receptor arrays or from one or more pattern assemblies or from both kinds of structure .normally , these signals would come from lower level structures but in principle any one pattern assembly may receive signals from any of the entire set of pattern assemblies , including itself .each id - neuron within each pattern assembly may send signals to one or more c - neurons .normally , these signals would go to higher level structures but in principle any one pattern assembly may send signals to any of the entire set of pattern assemblies , including itself .these kinds of interconnections amongst pattern assemblies are shown schematically in figure [ connections_figure ] , following the conventions shown in the legend to the figure .an aspect of the proposals that is not illustrated in the figure is that there will be low level pattern assemblies representing structures in every sensory modality vision , hearing , touch etc and high - level pattern assemblies may receive inputs from any of these lower level pattern assemblies .in this way , any high - level concept like ` person ' may be described in terms of all the relevant sensory modalities .in many cases , the connections that are needed to bring diverse sensory modalities together in this way will be relatively long , as previously indicated .an important feature of the connections between a receptor array and a pattern assembly and the connections between pattern assemblies is that _ the symbols at each end of every connection are always a match for each other_. in effect , the rle of each c - neuron is _ defined _ by its inputs . for example , the neural symbol ` a ' in the pattern assembly ` nstem 3 c a t ' in figure [ connections_figure ] may be seen to represent ` a ' because it receives inputs from other neural symbols that represent ` a ' . because the rle of each c - neuron within each pattern assembly is defined in this way ,it is not necessary to provide a repeated alphabet of symbols at each location within each pattern assembly like the repeated alphabet of symbols that is necessary at each location of a receptor array ( section [ new_patterns_and_symbols ] ) . in a similar way, the rle of each id - neuron is defined by the pattern assemblies to which it connects .the fact that connections between two pattern assemblies are always between symbols that match each other means that the proposed system of inter - connections amongst pattern assemblies provides a neural realisation of the relationship between identification and reference , as described in section [ id_ref_section ] .for example , the symbols ` d ' in the pattern ` np d  n  ' in figure [ connections_figure ] are connected to the matching symbols in the pattern ` d 0 t h e ' and it is envisaged that there would be similar connections to other patterns representing words in the grammatical class of ` determiners ' .thus ` d ' , as a reference to that grammatical class , has neural connections to patterns that represent the class.  it will be convenient to use the term _ neural reference _ for one or more neurons that is the neural equivalent of a reference , as described in section [ id_ref_section ] .   given a structure of receptor arrays , pattern assemblies and their interconnections as described so far , how can it function to identify the neural equivalent of one or two ` good ' multiple alignments like the one shown in figure [ alignment_1 ] ?it is envisaged that each pattern assembly will respond to incoming stimulation in a manner that is functionally coherent and similar to the responses of an individual neuron . below a certain threshold, there will be relatively little response amongst neurons in the pattern assembly .but when inputs exceed the threshold , the pattern assembly will ` fire ' and all the neurons in the assembly will become active together ( ` ignition ' as described by ) .it is envisaged that the neural equivalent of a multiple alignment will be identified when each of its pattern assemblies and the connections between them are relatively active as suggested by the connections shown with unbroken lines in figure [ connections_figure ] .other pattern assemblies and other connections will be relatively quiet , as suggested by the connections shown with broken lines in the same figure .it seems possible that there may also be a system of inhibitory connections amongst pattern assemblies perhaps mediated by structures below the cortex that would dampen down excessive neural activity and would have the effect of forcing a decision amongst competing alternatives so that the ` winner takes all ' .a mechanism of that sort operates in the innervation of the basilar membrane , where position along the length of the membrane encodes the frequency of sound and lateral inhibition has the effect of strengthening the response where the signal is strongest whilst suppressing the response in neighbouring regions .another mechanism that may help to prevent excessive excitation of pattern assemblies are inhibitory connections amongst the input fibres to pattern assemblies , as described in section [ order_relations_perception ] , next .   as the system has been described so far , a stimulus like ` b o y ' would produce the same level of response in a pattern assembly like ` n 2 y o b ' as it would in a pattern assembly like ` n 3 b o y ' .in other words , there is nothing in what has been proposed to keep track of the ordering or arrangement of elements in a pattern .our ability to recognise patterns despite some jumbling of their elements ( e.g. , solving anagrams ) is consistent with the scheme that has been described .but the fact that we can see the difference between an anagram and its solution shows that something else is needed .some kind of mechanism is required to distinguish between two kinds of situation :  1 .fibres leave a pattern assembly or one portion of a receptor array and arrive together , in the same relative positions , at one portion of another pattern assembly .fibres that conform to this rule will be described as _coherent_. 2 .fibres arrive at one portion of a pattern assembly from a variety of different sources or , if they come from one source , their relative positions are not preserved .this kind of arrangement will be described as _incoherent_.  when signals arrive at a pattern assembly , they should produce a stronger response in the first case than in the second . a possible mechanism to ensure that this happens would be lateral connections amongst the fibres of a coherent bundle that would have the effect of increasing the rate of firing in the bundle if the majority of them are firing together .when any one fibre in the bundle is firing , then signals will be carried to neighbouring fibres via the lateral connections and these signals should lower the threshold for firing and thus increase the rate of firing in any of the neighbouring fibres that are already firing . an alternative scheme that should achieve the same effectis inhibitory connections amongst fibres that are incoherent .if signals arrive in a temporal sequence , then other possible mechanisms include ` synfire chains ' and temporal ` sequence detectors ' , as described by .however , these appear to be less suitable for keeping track of order in spatially - distributed visual patterns , seen in a single glance .   it is envisaged that learning in sp - neural would occur in a manner that is similar to sp70 ( section [ sp70_section ] ) :  * when there is a good match between sensory patterns and stored pattern assemblies , the sensory data will be encoded in terms of those pattern assemblies much as in section [ evaluation_of_alignments ] .but if there is no good match for sensory data or only a partial match , then new pattern assemblies may be created like the patterns described in section [ sp70_section ] . *the creation of a new pattern assembly is unlikely to mean growing new neurons from scratch .although there is evidence that new nerve cells can grow in mature brains , it seems likely that the bulk of neurons in the brain are present at birth and that the creation of a new pattern assembly is largely a process of assigning pre - existing neurons to it , with the creation or breaking of connections as outlined below . *associated with each pattern assembly will be some physiological analogue of the frequency measure associated with each sp pattern counting the number of times each pattern assembly has been recognised in sensory inputs .pattern assemblies that have been frequently recognised should have a lower threshold for excitation or should be otherwise more responsive to incoming stimulation . *periodically , pattern assemblies may be evaluated in terms of their usefulness for encoding sensory data .those that are proving useful may be retained and those that are not proving useful may be purged .the purging of a pattern assembly would not mean the literal destruction of the neurons in the assembly .it would be largely a matter of breaking the connections with other pattern assemblies so that the neurons in the pattern assembly can be made available for later re - use in some other pattern assembly .  with regard to the making and breaking of connections , the lateral connections amongst the neural symbols of a new pattern assemblymay already be in place before it is created and the new assembly may be brought into existence by the relatively simple process of breaking connections at the boundaries .something like this is suggested by evidence for the progressive weakening of connections between cells that do not normally fire at the same time except that , in the present proposal , connections at the boundaries would be broken completely when a pattern assembly is first created .we shall return to this and related points in section [ learning_discussion ] .each new pattern assembly would also need appropriate connections with receptor arrays and other pattern assemblies . at first sight , this suggests the growing of new fibres , many of which would be quite long .it is much more plausible to suppose , as hebb suggested , that there are pre - established long connections between different parts of the cortex and that new connections are established by the growth of short connecting links to the main connections ( much as a new telephone may normally be installed without the necessity of laying a new cable all the way to the exchange ) .it may not even be necessary to postulate the growth of short links .it is possible that pre - existing links could be switched on or off somehow according to need .something like this might enable us to account for the speed with which we can establish memories for names , faces , events and so on ( see section [ learning_discussion ] , below ) .this section considers a selection of issues in the evaluation of the sp - neural proposals as they relate to empirical phenomena and alternative proposals , mainly hebb s original concept of a ` cell assembly ' and versions of that concept that have been proposed subsequently . to avoid terminological confusion, i will use the term ` cell assembly ' for all versions of that concept ( including pattern assemblies ) unless a particular version is the focus of interest .hebb proposed that cell assemblies could become associated and that assemblies that are associated might eventually merge to become a single assembly .he also envisaged an hierarchical organisation for conceptual structures and the possibility that low - level concepts ( and their corresponding cell assemblies ) might be shared by two or more higher - level concepts ( and their corresponding cell assemblies ) ( see also * ? ? ? * ) .  in this connection , raises some pertinent questions :  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ `` how do associations between cell assemblies differ from internal associations that are responsible for the assemblies properties ?it does not seem likely that both these processes can be the result of similar synaptic changes as is usually assumed .if they were , the interassembly associations would soon become intraassembly loops .a related puzzle is that parts are not submerged in the whole . doors and windows are integral parts of my concept of a house , but they are also robust , stand - alone concepts . '' ( p. 71 ) . __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  later on the same page he writes : `` perhaps the toughest problem of all concerns the fact that we have many associations with almost every engram .... the brain must be a veritable rat s nest of tangled associations , yet for the most part we navigate through it with ease . '' provides a possible answer to the way in which parts may be distinguished from wholes :  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ `` if two assemblies a and b are repeatedly active at the same time they will tend to become ` associated , ' so that a excites b and vice versa . if they are always active at the same time they will tend to merge in a single systems that is , form a single assembly but if they are also active at different times they will remain separate ( but associated ) systems .( this means that exciting part of a , for example , has a very high probability of exciting all of a , but a definitely lower probability of exciting a separate assembly , b ; a may be able to excite b only when some other assembly , c , also facilitates activity in b ) . ''( p. 105 ) ._ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  this may be part of the answer but it does not get to the bottom of the problem . there is a need to recognise that sharing of structures can be done in three distinct ways as illustrated in figure [ sharing_figure ] and described here :  * _ literal sharing_. the neurons in the lower - level cell assembly ( ` a ' in the figure ) are also part of two or more higher - level cell assemblies ( ` b ' and ` c ' in the figure ) . *_ sharing by reference_. the lower - level cell assembly ( ` a ' ) is _ outside _ the higher - level cell assemblies ( ` b ' and ` c ' ) but each of the latter contains one or more neurons ( shown in the figure as a single neuron marked with a small ` a ' ) that serves as a proxy , agent or representative for the lower - level cell assembly and is connected to that assembly .the proxy is a neural reference , as described in section [ connections_section]functionally equivalent to the concept of a reference described in section [ id_ref_section ] .+ if sharing of structures is always achieved in this way as proposed in this paper then any one neuron belongs in one cell assembly and _ only _ one cell assembly . * _ sharing by copying_. in each of two or more higher - level cell assemblies ( ` b ' and ` c ' ) there is a _ copy _ of the lower - level cell assembly ( ` a ' ) . as with sharing by reference , there is no need to postulate that any one neuron may belong in two or more cell assemblies .it is reasonably clear that hebb and all subsequent authors have not intended the third sense in which cell assemblies might share structure .although it may have a rle in some aspects of cognition , i shall say no more about it in this paper .  with regard to the first two senses in which cell assemblies may share structure , they have not to my knowledge been clearly differentiated either in hebb s writings or in any other writings about cell assemblies .hebb s descriptions of the cell assembly concept imply that any one neuron may belong in two or more cell assemblies in accordance with the first sense of structure sharing .this seems to be generally accepted by people writing later and it has often been made explicit ( see , for example , , , , and ) .but as a means of representing conceptual structures and their inter - relations , literal sharing is problematic .for example , it is difficult to see why assemblies b and c in the top part of figure [ sharing_figure ] should not simply merge into a single cell assembly .perhaps more serious is the problem of keeping track of the relative positions or ordering of shared components in two or more configurations .  consider a pack of playing cards containing the ace , king , queen and jack of clubs , all together and in that order .this configuration may be represented within a cell assembly as shown in part ( a ) of figure [ cards_sharing_literal_figure ] . here ,each card is represented by a cell assembly and the order of the cards within the pack is shown by connections between those cell assemblies .as it stands , the representation is unambiguous .but we run into difficulties if , using literal sharing of structures , we try to add a representation of the pack after it has been shuffled . part ( b ) of figure [ cards_sharing_literal_figure ] shows the result if the same four cards fall together in the pack but in the order queen , ace , jack and king .the addition of connections to represent that order causes the whole structure to become ambiguous .if the connections are non - directional , the ace , for example , may be followed immediately by any of the other three cards . and even if directional links are allowed , the ace may be followed immediately by the king or the jack .the problem can be overcome if , instead of using literal sharing of structures , we use sharing by reference . parts ( a ) and ( b ) of figure [ cards_sharing_reference_figure ] each show how an ordering of the playing cards may be represented unambiguously by a sequence of neurons , each one of which is a neural reference to a cell assembly for the corresponding card ( shown in the middle of the figure ) .neural references also provide a neat solution to the other problems raised by milner :  * associations between pattern assemblies may be encoded by building a new pattern assembly containing neural references to the pattern assemblies that are to be associated .these associations are then internal to the new pattern assembly and the original pattern assemblies retain their identity . * in a similar way , there can be a stand - alone cell assembly for each component of a house while the assembly for the house itself comprises a collection of neural references to the components . in this way, the concept of a house does not become muddled with concepts for doors , windows etc .* it is true that there must be a ` rat s nest ' of associations in the brain but neural references allow these to be encoded in a way that does not disturb the integrity of each concept .the coherence of a web page is not affected by links to that page from other pages and it makes no difference how many such pointers or references there may be .  some of the power of sharing by reference can be seen in figure [ class_part_figure ] .the figure shows a hierarchy of classes from the relatively abstract ` animal ' ( ` a ' near the bottom of the figure ) , via ` vertebrate ' ( ` v ' ) , ` mammal ' ( ` m ' ) , ` cat ' ( ` c ' ) , to a specific cat ` tibs ' shown near the top of the figure . at the same time, the figure shows the part - whole relations between each class and the descriptive elements of the class . in a more elaborate example , these elements may themselves be broken down into a hierarchy of parts and subparts .as previously noted , arrows on the connections between pattern assemblies show the direction of flow of sensory signals .    in the figure ,each c - neuron in the body of a pattern assembly functions as a proxy for or neural reference to the pattern assembly from which it receives sensory signals .the whole structure achieves the effect of ` inheritance ' in an object - oriented system : the individual cat called ` tibs ' inherits the feature ` purrs ' from the ` cat ' pattern assembly , the features ` suckles young ' and ` furry ' are inherited from the ` mammal ' pattern assembly , ` backbone ' is inherited from the ` vertebrate ' assembly , and ` eats ' and ` breathes ' are inherited from the ` animal ' pattern assembly .an alternative to the foregoing is to suppose that cell assemblies may form hierarchies and other kinds of structure by temporal correlations amongst their neurons : `` a set of units can be bound into a block by synchronising their fast activity fluctuations .several such blocks can coexist if their activity is desynchronised relative to each other ... '' and `` to represent that attribute a is bound to attribute b and attribute c to attribute d , the cells for a and b fire in synchrony , the cells for c and d fire in synchrony , and the ab set fires out of synchrony with the cd set . '' .ideas of this kind reviewed by  are clearly very different from the sp - neural proposals but there is insufficient space here for any kind of weighing of the pros and cons .id - neurons in the present proposals are similar to the ` cardinal cells ' described by , except that there may be more than one for each concept .constructs such as neural reference and multiple alignment are also distinctive in the present proposals .the chief objection to the idea of cardinal cells is that , if ` grandmother ' is represented by a single cell , we will not be able to access our knowledge of her if that cell dies .but as barlow points out , a small amount of replication will give considerable protection against this kind of catastrophe .as suggested in section [ old_patterns_and_symbols ] , each neural symbol may be a small cluster of neurons rather than a single neuron .another possibility is that all our concepts are replicated two or three times in the brain .arguments and calculations that would take too much space to reproduce here suggest that , with the sp - neural scheme for representing knowledge , this kind of replication is feasible .hebb s proposals for learning are summarised in his much - quoted suggestion that :  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ `` when an axon of cell a is near enough to excite a cell b and repeatedly or persistently takes part in firing it , some growth process or metabolic change takes place in one or both cells such that a s efficiency , as one of the cells firing b , is increased .'' ._ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  this idea forms the basis for learning in many varieties of artificial neural network . apart from the elegant simplicity of the idea , another possible reason for its popularityis that it accords with statistical models of learning where two things need to co - occur with a relatively high frequency before the association between them can be seen to be significant .and the hebbian view of learning is broadly in accordance with the slow build up of our knowledge throughout childhood and beyond .and yet this view of learning is in direct conflict with everyday observations of what we can and do learn from a single occurrence or experience .if we are involved in some kind of accident which does not cause us to lose consciousness , we can remember the sequence of events and recount them with little difficulty .a possible objection here is that , where strong emotions are involved , we may rehearse our memories many times and thus strengthen the links amongst the corresponding cell assemblies .but the same objection carries much less force if we are asked to recall things that carry less emotional charge . on any given evening, we normally have no difficulty in recalling the humdrum events of the day and this without any apparent rehearsal .  because the slow growth of cell assemblies does not account for our ability to remember things immediately after a single exposure, hebb adopted a ` reverberatory ' theory for this kind of memory .but , as points out , it is difficult to understand how this kind of mechanism could explain our ability to assimilate a previously - unseen telephone number .each digit in the number may be stored in a reverberatory assembly but this does not explain how we remember the _ sequence _ of digits in the number .  in the learning scheme outlined in sections [ sp70_section ] and [ neural_learning_section ] ,new pattern assemblies can be created in response to a single sensory input .this is consistent with our ability to remember unique events and sequences of events . and, if we suppose that synapses can be switched on or off in seconds or fractions of a second according to need , we may be able to account for the speed with which immediate memories can be established .this model for short - term memory does not suffer from the weakness in the reverberatory model that milner identified : we may remember the sequence of digits in a telephone number by creating a new pattern assembly that represents the number .but an ability to lay down new pattern assemblies relatively quickly does not explain why it takes several years to learn something like a language . the principles on which sp70 is based suggest why learning a language , and similar kinds of learning , take quite a lot of time .the abstract ` space ' of alternative grammars for any natural language is astronomically large and , even using heuristic techniques like those in the sp models , it takes time to search amongst the many possibilities . finding a tolerably good grammar for any natural language is a very complex problem and it can not be solved in an instant .   a possible objection to the sp - neural proposalsis that they are inconsistent with the ` constancy ' phenomena in perception .these include :  * _ size constancy_. we can recognise an object despite wide variations in the size of its image on the retina and we judge its size to be constant despite these variations . * _ brightness constancy_. we can recognise something despite wide variations in the absolute brightness of the image on our retina ( and , likewise , we judge its intrinsic brightness to be constant ) . * _ colour constancy_. in recognising the intrinsic colour of an object , we can make allowances for wide variations in the colour of the light which falls on the object and the consequent effect on the colour of the light that leaves the object and enters our eyes .if the pattern recorded in a receptor array was merely a copy of sensory input there would indeed be wide variations in the size of the visual pattern projected by a given object and similar variations in brightness and colours .the suggestion here is that much of the variability of sensory input from a given object has been eliminated at a stage before the information reaches the receptor array .lateral inhibition in the retina emphasises boundaries between relatively uniform areas . the redundant information within each uniform area is largely eliminated which means that it is , in effect , shrunk to the minimum size needed to record the attributes of that area . since this minimum will be the same regardless of the size of the original image, the overall effect should be to reduce or eliminate variations in the sizes of images from a given object .  in a similar way ,the ` primal sketch ' created by lateral inhibition should be largely independent of the absolute brightness of the original image because it is a distillation of _ changes _ within an image , independent of absolute values . andadaptation in the early stages of visual processing should to a large extent prevent variations in brightness having an effect on patterns reaching the receptor array .edwin land s ` retinex ' theory suggests that colour constancy is achieved by processing in the retina and in the visual cortex ( see * ? ? ?this is consistent with the idea that this source of variability has been removed at a stage before sensory input is compared with stored patterns .an important motivation for the cell assembly concept was to explain the phenomenon of ` stimulus equivalence ' or ` generalization'our ability to recognise things despite variations in size , shape , position of the image on the retina , and so on .however , writes that `` generalization is certainly an important and puzzling phenomenon , but discrimination is equally important and tends to be ignored in theories of neural representation .any theory of engram formation must take into account the relationship between category and instance the ability we have to distinguish our own hat , house , and dog from hats , houses and dogs in general . ''( p. 70 ) .both stimulus equivalence and discrimination can be explained in terms of sp - neural .the theory accounts for generalization in four main ways :  * the way in which perceptual constancies may be accommodated in the present proposals was considered in section [ constancies ] . * as with the original cell assembly concept , a pattern assembly can respond provided a reasonably large subset of its neurons has received inputs .this seems to accommodate our ability to recognise things despite omissions , additions or substitutions in sensory inputs .* in the present scheme , each neuron or feature detector in a receptor array is connected to each one of the pattern assemblies ( mainly low level assemblies ) that contain the corresponding feature .thus , each of these pattern assemblies may respond to appropriate input regardless of the position of the input on the receptor array . * owing to the provision of neural references in the present proposals , it is possible to create pattern assemblies that represent abstractions from sensory input .an example from figure [ connections_figure ] is the pattern assembly ` np d  n  ' .this pattern assembly represents the abstract structure of a noun phrase via references to the class of determiners ( ` d ' ) and the class of nouns ( ` n ' ) .it may be activated by a range of _ alternative _ inputs : any noun phrase that conforms to the determiner - noun pattern .the sp theory provides a neat account of our ability to distinguish specific instances from the categories in which they belong .as mentioned in section [ other_aspects_section ] , the sp system can build multiple alignments which include several different levels of abstraction , including a level that represents a specific instance .examples may be seen in and earlier publications .the way in which a hierarchy of classes ( including a specific instance ) may be represented in sp - neural is illustrated in the example described at the end of section [ sharing_section ] ( figure [ class_part_figure ] ) .one of the strengths of the sp system is its ability to represent syntactic structures of natural languages and to process them in the parsing and production of sentences .although sp - neural is not complete , it already suggests solutions to the challenges for any neural theory of language processing that have been identified by :  1 .how can centre - embedded sequences such as ` abcc ' be represented ? 2 .how can discontinuous constituents and distributed words be realised ( e.g. , _switch ... on _ ) ?how is it possible to specify the syntactic relatedness of distant elements in a string ( e.g. , noun and verb agreement ) ? 4 .how can repeated use of the same word or lexical category within a sentence be modelled and stored ?a related question is how recursive structures might be modelled and stored ? 5 .how can lexical categories such as ` noun ' , ` verb ' and ` adjective'be realised in a neuronal network ?  the answers suggested here have similarities to and differences from those proposed by pulvermller in chapters 8 to 12 ( _ ibid . _ ) .there is insufficient space here for a detailed comparison so i will merely outline the way in which these questions may be answered in sp - neural .the concept of neural reference provides a key to all five questions .sp - neural suggests that questions ( a ) , ( b ) , ( d ) and ( e ) may be answered in a way that is a close analogue of the organisation of a context - free phrase - structure grammar , while the ` context sensitive ' feature of the sp theory ( section [ context_sensitive_section ] ) may provide an answer to question ( c ) .figure [ center_embedding_figure ] shows how , using neural references , ` b ... b ' may be embedded in ` a ... a ' and how ` c ... c ' may be embedded in ` b ... b ' .    in a similar way , the ` embedding ' of a noun - phrase pattern assembly within an assembly representing the phrase _ switch ... on _ may be achieved by means of a neural reference to the noun phrase placed between _ switch _ and _ on _ , as shown in figure [ discontinous_structures_figure ] .this figure also shows how grammatical categories such as ` determiner ' ( ` d ' ) and ` noun ' ( ` n ' ) may be represented in sp - neural .the pattern assembly representing the structure of a noun phrase contains neural references to each of these categories and each neural reference is connected to pattern assemblies representing the members of the category .figure [ repeated_structures_figure ] shows how a noun phrase ( ` np ' ) may be repeated within a sentence by the provision of _ two _ neural references to a pattern assembly representing the structure of a noun phrase .the repetition is expressed by the neural references , not by the structure itself .    andfigure [ recursion_figure ] shows how the recursion in phrases like _ the very very ... fast car _ may be represented in sp - neural .the first of the two pattern assemblies identified as ` x ' contains a neural reference to itself and this expresses the recursive nature of ` very ... ' .notice that , for the production of language but perhaps not for the analysis of language , it is also necessary to provide a ` null ' member of the ` x ' category so that it is possible to escape from the recursive loop .these pattern assemblies are close analogues of the rules in a phrase - structure grammar that would be needed to describe this kind of language structure .the suggested answer to question ( c ) within sp - neural hinges on the ability of the sp system to represent and process long - distance dependencies as indicated in section [ context_sensitive_section ] ( see also * ? ? ?* ; * ? ? ?it is envisaged that pattern assemblies representing grammatical structures may include ones like the pattern shown in row 8 of figure [ alignment_1 ] .this pattern expresses the idea that , within a sentence , a singular noun in the ` subject ' position must be followed by a singular verb in the position of the main verb .there will also be a similar pattern representing plural dependencies .it is envisaged that , within sp - neural , these pattern assemblies will exert their influence in much the same way as the corresponding patterns in the sp system .the broad outlines of sp - neural are reasonably clear but there are still several areas of uncertainty in the details .if this neural version of the sp theory can be completed satisfactorily , then it will inherit all the considerable explanatory force of the sp theory in several areas of perception and cognition including the way diverse forms of knowledge may be represented and integrated , the analysis and production of language , fuzzy recognition of objects , recognition at multiple levels of abstraction , retrieval of memories from fragmentary clues , various forms of probabilistic reasoning and exact reasoning , unsupervised learning , and the solving of problems by reasoning and by analogy .  even though sp - neural is not complete, it already offers useful insights into the kinds of structures and functions that we may expect to find in brains and nervous systems .the theory makes a number of predictions that are , in principle , falsifiable although they represent a considerable challenge for current techniques and technologies .the main predictions of the theory are these :  * as in hebb s theory and other theories in that tradition , sp - neural proposes that knowledge is stored in functionally - coherent and inter - connected assemblies of neurons in the cortex . however , the theory is distinctive in most of the points that follow . *it is proposed that any one neuron belongs in one pattern assembly and _ only _ one assembly .* it is envisaged that , while there may be long connections _ between _ pattern assemblies , the neurons within any one assembly lie close together within the cortex , perhaps confined to one column , with short connections between them , mainly between immediate neighbours .these are not a necessary part of the proposals but they seem likely . * relationships amongst pattern assemblies ,including part - whole relations , class - inclusion relations and associations between pattern assemblies , are mediated by the use of neurons that are functionally equivalent to _ references _ between structures , as described in the text .* when sensory information is received , recognition of patterns and objects is achieved by the excitation of pattern assemblies and connections between them in a manner that is functionally equivalent to the creation of multiple alignments in the sp theory .* there may be inhibitory connections between assemblies to dampen excessive excitation and sharpen competition between assemblies .inhibitory connections between afferent fibres may help to preserve information about the order of features or events in sensory inputs .* it is proposed that learning is achieved by the creation of pattern assemblies and selection amongst them , in the same way that patterns are created and selected in the sp theory .assemblies can be created and destroyed by the making and breaking of short connections , as described in text . * to record short - term memories ,pattern assemblies may be created within seconds or fractions of a second , perhaps by the activation and de - activation of pre - existing synaptic connections .  finding direct neurophysiological evidence that may confirm of confute these predictions is not likely to be easy but the proposals in this paper may suggest new avenues for investigation .uncertainties in the theory need to be resolved , ideally by observations of brain structure and functioning .meanwhile , computer simulation techniques may help to shape the theory .i am very grateful for constructive comments on earlier versions of this paper that i have received from horace barlow , jim maxwell legg , peter milner , friedemann pulvermller and guillaume thierry .the responsibility for all errors and oversights is , of course , my own .i am also grateful to daniel wolff for a very useful discussion of the sharing problem .barlow , h. b. ( 1959 ) .sensory mechanisms , the reduction of redundancy , and intelligence . in hmso ,editor , _ the mechanisation of thought processes _ , pages 535559 . her majesty s stationery office , london .bienenstock , e. and geman , s. ( 1995 ) .compositionality in neural systems . in arbib , m. ,editor , _ the handbook of brain theory and neural networks _ , pages 223226 .mit / bradford books , elsevier , amsterdam .huyck , c. r. ( 2001 ) .cell assemblies as an intermediate level model of cognition . in wermter , s. , austin , j. , and willshaw , d. , editors , _ emergent neural computational architectures based on neuroscience _ , pages 383397springer , heidelberg .von der malsburg , c. ( 1987 ) .synaptic plasticity as basis of brain organization . in changeux ,p . and konishi , m. , editors , _ the neural and molecular bases of learning , dahlem konferenzen_. john wiley & sons , chichester .wolff , j. g. ( 1988 ) . learning syntax and meanings through optimization and distributional analysis . in levy ,y. , schlesinger , i. m. , and braine , m. d. s. , editors , _ categories and processes in language acquisition _, pages 179215 .lawrence erlbaum , hillsdale , nj .copy : www.cognitionresearch.org.uk/lang_learn.html#wolff_1988 .wolff , j. g. ( 1999b ) .probabilistic reasoning as information compression by multiple alignment , unification and search : an introduction and overview ., 5(7):418462 .copy : http://arxiv.org/abs/cs.ai/0307010 .wolff , j. g. ( 2000 ) .syntax , parsing and production of natural language in a framework of information compression by multiple alignment , unification and search ., 6(8):781829 .copy : http://arxiv.org/abs/cs.ai/0307014 .wolff , j. g. ( 2001 ) .information compression by multiple alignment , unification and search as a framework for human - like reasoning . , 9(1):205222 .first published in the _ proceedings of the international conference on formal and applied practical reasoning ( fapr 2000 ) _, september 2000 , issn 14694166 .copy : www.cognitionresearch.org.uk/papers/pr/pr.htm .wolff , j. g. ( 2002a ) .mathematics and logic as information compression by multiple alignment , unification and search .technical report , cognitionresearch.org.uk .copy : http://arxiv.org/abs/math.gm/0308153 .wolff , j. g. ( 2002b ) .unsupervised learning in a framework of information compression by multiple alignment , unification and search .technical report , cognitionresearch.org.uk .copy : http://arxiv.org/abs/cs.ai/0302015 .wolff , j. g. ( 2003a ) .information compression by multiple alignment , unification and search as a unifying principle in computing and cognition ., 19(3):193230 .copy : http://arxiv.org/abs/cs.ai/0307025 .wolff , j. g. ( 2003b ) .unsupervised grammar induction in a framework of information compression by multiple alignment , unification and search . in dela higuera , c. , adriaans , p. , van zaanen , m. , and oncina , j. , editors , _ proceedings of the workshop and tutorial on learning context - free grammars _ , pages 113124 .this workshop was held in association with the 14th european conference on machine learning and the 7th european conference on principles and practice of knowledge discovery in databases ( ecml / pkdd 2003 ) , september 2003 , cavtat - dubrovnik , croata .copy : http://arxiv.org/abs/cs.ai/0311045 .wolff , j. g. ( 2004 ) .integration of ` rules ' and ` similarity ' in a framework of information compression by multiple alignment , unification and search . .commentary on `` the rules versus similarity distinction '' by e. m. pothos in the same issue .

Abstract: this paper describes how the elements of the sp theory may be realised with neural structures and processes . to the extent that this is successful , the insights that have been achieved in the sp theory - the integration and simplification of a range of phenomena in perception and cognition - may be incorporated in a neural view of brain function .   these proposals may be seen as a development of hebb s concept of a ` cell assembly ' . by contrast with that concept and variants of it , the version described in this paper proposes that any one neuron can belong in one assembly and only one assembly . a distinctive feature of the present proposals is that any neuron or cluster of neurons within a cell assembly may serve as a proxy or reference for another cell assembly or class of cell assemblies . this device provides solutions to many of the problems associated with cell assemblies , it allows information to be stored in a compressed form , and it provides a robust mechanism by which assemblies may be connected to form hierarchies , grammars and other kinds of knowledge structure .  drawing on insights derived from the sp theory , the paper also describes how unsupervised learning may be achieved with neural structures and processes . this theory of learning overcomes weaknesses in the hebbian concept of learning and it is , at the same time , compatible with the observations that hebb s theory was designed to explain .