In this paper, the early design of our self-organized agent-based simulation model for exploration of synaptic connections that faithfully generates what is observed in natural situation is given. While we take inspiration from neuroscience, our intent is not to create a veridical model of processes in neurodevelopmental biology, nor to represent a real biological system. Instead, our goal is to design a simulation model that learns acting in the same way of human nervous system by using findings on human subjects using reflex methodologies in order to estimate unknown connections.
In the literature, there are many models for the self-organization of neuronal networks. @cite_9 developed a toolkit for computational neuroscientists to explore developmental changes in biological neural networks. However, details of the methodology used (e.g., how the initial random network is constructed) and of simulation parameters (e.g., how the threshold parameter for pruning is obtained) are not clear. @cite_4 present an approach to self-organization in a dynamic neural network by assembling cooperative neuro-agents. However, their intent is not to explore synaptic connectivity. @cite_2 addresses the development of brain-inspired models that will be embedded in robotic systems to support their cognitive abilities. However, this work focuses on brain slices rather than reflex pathways and aims to improve cognitive capabilities of robotic systems rather than exploring synaptic functional connectivity.
@cite_9 @cite_4 @cite_2
2742598047 176707287 2168012262
We address the development of brain-inspired models that will be embedded in robotic systems to support their cognitive abilities. We introduce a novel agent-based coevolutionary computational framework for modeling assemblies of brain areas. Specifically, self-organized agent structures are employed to represent brain areas. In order to support the design of agents, we introduce a hierarchical cooperative coevolutionary (HCCE) scheme that effectively specifies the structural details of autonomous, yet cooperating system components. The design process is facilitated by the capability of the HCCE-based design mechanism to investigate the performance of the model in lesion conditions. Interestingly, HCCE also provides a consistent mechanism to reconfigure (if necessary) the structure of agents, facilitating follow-up modeling efforts. Implemented models are embedded in a simulated robot to support its behavioral capabilities, also demonstrating the validity of the proposed computational framework. Our researches on adaptive systems are inspired by their ability of building by themselves a representation of their surrounding world. Using cooperation as a local criterion of self-organization, we study in a network of neuro-agents the evolution of the system and in the same way, the emergence of a functioning coherent with the environmental feed-back. In this paper we expose the abilities of the neuro-agents that give them the autonomy required to build an ab nihilo topology. And finally we want to emphasize the dynamic of the network organization that is basically its best property of adaptation to a changing environment. The work presented here concerns our approach to self-organization in a dynamic neural network. We present some experiments of emergent learning based on our knowledge of complex systems through multi-agent systems technology and cellular biology. In fact we are most interested in the mechanisms involved in the construction of a network able to stabilize its structure in a given environment, than in the efficiency of the learning supported by this structure. Obviously those two ontogenetic processes are strongly coupled and appear simultaneously as mutual consequences. Under those conditions, we conceive artificial systems presenting an inherent complexity close to biological systems as they are open and as their structure evolves from nothing (at least an empty shell) up to an organized, stable and functional network of neuro-agents. Our working hypothesis tells minimal entities of such a system can be built and given with sufficient behavioral potentialities in order to organize themselves to satisfy the whole system activity. In this study, we want to show that a coherent and useful system can emerge without any global model for its functioning, solely from activity and organizational rules of its sub-parts; only agents are designed, not the topology of the network they will belong to. Our thought process supposes four distinct steps that description will be formerly extended. Firstly the neuro-agent unit has to be designed, their properties, their abilities (calculus, communication) and their knowledge. Secondly, initial system is constituted of a minimal number of unlinked neuro-agents. Those pre-built neuro-agents are only receptors (inputs) and actuators (outputs) of the network. Thirdly the global system has to be plunged in a dynamic and open environment in which it will have to adapt to; regarding neuromimetic systems, this phase suppose Evolutionary Robotics (ER) is one of promising approaches to design robot controllers which essentially have complicated and or complex properties. In most ER research, the sensory-motor mappings of robots are represented as artificial neural networks, and their connection weights (and sometimes the structure of the networks) can be optimized in the parameter spaces by using evolutionary computation. However, generally, the evolved neural controllers could be fragile in unexperienced environments, especially in real worlds, because the evolutionary optimization processes would be executed in idealized simulators. This is known as the gap problem between the simulated and real worlds. To overcome this, the author focused on evolving an on-line learning ability instead of weight parameters in a simulated environment. According to recent biological findings, actually, the kinds of on-line adaptation abilities can be found in real nervous systems of insects and crustaceans, and it is also known that a variety of neuromodulators (NMs) play crucial roles to regulate the network characteristics (i.e. activating blocking changing of synaptic connections). Based on this, a neuromodulatory neural network model was proposed and it was utilized as a mobile robot controller. In the paper, the detail behavior analysis of the evolved neuromodulatory neural network is also discussed. Sensorimotor control and learning are fundamental prerequisites for cognitive development in humans and animals. Evidence from behavioural sciences and neuroscience suggests that motor and brain development are strongly intertwined with the experiential process of , where internal body representations are formed and maintained over time. In order to guide our movements, our brain must hold an internal model of our body and constantly monitor its configuration state. How can sensorimotor control using such low-level body representations enable the development of more complex cognitive and motor capabilities? Although a clear answer has still not been found for this question, several studies suggest that processes of mental simulation of action-perception loops are likely to be executed in our brain and are dependent on internal body representations. Therefore, the capability to re-enact sensorimotor experience might represent a key mechanism behind the implementation of higher cognitive capabilities, such as behaviour recognition, arbitration and imitation, sense of agency and self-other distinction. Addressed mainly to researchers on autonomous motor and mental development in artificial agents, this work aims at gathering the latest development in the study on exploration behaviours, on internal body representations, on internal models, and on mechanisms for internal sensorimotor simulations. Relevant studies in human and animal sciences are discussed and a parallel to similar investigations in robotics is presented. Recent research shows that some brain areas perform more than one task and the switching times between them are incompatible with learning and that parts of the brain are controlled by other parts of the brain, or are “recycled”, or are used and reused for various purposes by other neural circuits in different task categories and cognitive domains. All this is conducive to the notion of “programming in the brain”. In this paper, we describe a programmable neural architecture , biologically plausible on the neural level, and we implement, test, and validate it in order to support the programming interpretation of the above-mentioned phenomenology. A programmable neural network is a fixed-weight network that is endowed with auxiliary or programming inputs and behaves as any of a specified class of neural networks when its programming inputs are fed with a code of the weight matrix of a network of the class. The construction is based on the “pulling out” of the multiplication between synaptic weights and neuron outputs and having it performed in “software” by specialised multiplicative-response fixed subnetworks. Such construction has been tested for robustness with respect to various sources of noise. Theoretical underpinnings, analysis of related research, detailed construction schemes, and extensive testing results are given. Characterization of neuronal death and neurogenesis in the adult brain of birds, humans, and other mammals raises the possibility that neuronal turnover represents a special form of neuroplasticity associated with stress responses, cognition, and the pathophysiology and treatment of psychiatric disorders. Multilayer neural network models capable of learning alphabetic character representations via incremental synaptic connection strength changes were used to assess additional learning and memory effects incurred by simulation of coordinated apoptotic and neurogenic events in the middle layer. Using a consistent incremental learning capability across all neurons and experimental conditions, increasing the number of middle layer neurons undergoing turnover increased network learning capacity for new information, and increased forgetting of old information. Simulations also showed that specific patterns of neural turnover based on individual neuronal connection characteristics, or the temporal-spatial pattern of neurons chosen for turnover during new learning impacts new learning performance. These simulations predict that apoptotic and neurogenic events could act together to produce specific learning and memory effects beyond those provided by ongoing mechanisms of connection plasticity in neuronal populations. Regulation of rates as well as patterns of neuronal turnover may serve an important function in tuning the informatic properties of plastic networks according to novel informational demands. Analogous regulation in the hippocampus may provide for adaptive cognitive and emotional responses to novel and stressful contexts, or operate suboptimally as a basis for psychiatric disorders. The implications of these elementary simulations for future biological and neural modeling research on apoptosis and neurogenesis are discussed. Peer-to-peer (P2P) systems provide a new solution to distributed information and resource sharing because of its outstanding properties in decentralization, dynamics, flexibility, autonomy, and cooperation, summarized as DDFAC in this paper. After a detailed analysis of the current P2P literature, this paper suggests to better exploit peer social relationships and peer autonomy to achieve efficient P2P structure design. Accordingly, this paper proposes Self-organizing peer-to-peer social networks (SoPPSoNs) to self-organize distributed peers in a decentralized way, in which neuron-like agents following extended Hebbian rules found in the brain activity represent peers to discover useful peer connections. The self-organized networks capture social associations of peers in resource sharing, and hence are called P2P social networks. SoPPSoNs have improved search speed and success rate as peer social networks are correctly formed. This has been verified through tests on real data collected from the Gnutella system. Analysis on the Gnutella data has verified that social associations of peers in reality are directed, asymmetric and weighted, validating the design of SoPPSoN. The tests presented in this paper have also evaluated the scalability of SoPPSoN, its performance under varied initial network connectivity and the effects of different learning rules. Biological systems such as rats have special brain structures which process spatial information from the environment. They have efficient and robust localization abilities provided by special neurons in the hippocampus, namely place cells. This work proposes a biologically plausible architecture which is based on three recently developed techniques: Reservoir Computing (RC), Slow Feature Analysis (SFA), and Independent Component Analysis (ICA). The bottom layer of our RC-SFA architecture is a reservoir of recurrent nodes which process the information from the robot's distance sensors. It provides a temporal kernel of rich dynamics which is used by the upper two layers (SFA and ICA) to autonomously learn place cells. Experiments with an e-puck robot with 8 infra-red sensors (which measure distances in [4–30] cm) show that the learning system based on RC-SFA provides a self-organized formation of place cells that can either distinguish between two rooms or to detect the corridor connecting them. We propose a mathematical theory for how networks of neurons in the brain self-organize into functional networks, similarly to the self-organization of supply networks in a free-market economy. The theory is inspired by recent experimental results showing how information about changes to output synapses can travel backward along axons to affect a neuron's inputs. In neuronal development, competition for such retroaxonal'' signals determines which neurons live and which die. We suggest that in adults, an analogous form of competition occurs between neurons, to supply their targets with appropriate information in exchange for a payment'' returned to them backward along the axon. We review experimental evidence suggesting that neurotrophins may constitute such a signaling pathway in the adult brain. We construct a mathematical model, in which a small number of consumer'' neurons receive explicit fast error signals while a larger number of producer'' neurons compete to supply them with information, guided by retroaxonal signals from the consumers and from each other. We define a loss function to measure network performance and define the worth'' of a producer to be the increase in loss that would result if that neuron were to fall silent. We show how slow retroaxonal signals can allow producers to estimate their worth, and how these estimates allow the network to perform a form of parallel search over multiple producer cells. We validate our approximations and demonstrate the proposed learning rule using simulations. Convolutional Neural Networks(CNNs) are both computation and memory intensive which hindered their deployment in many resource efficient devices. Inspired by neural science research, we propose the synaptic pruning: a data-driven method to prune connections between convolution layers with a newly proposed class of parameters called Synaptic Strength. Synaptic Strength is designed to capture the importance of a synapse based on the amount of information it transports. Experimental results show the effectiveness of our approach empirically. On CIFAR-10, we can prune various CNN models with up to 96 connections removed, which results in significant size reduction and computation saving. Further evaluation on ImageNet demonstrates that synaptic pruning is able to discover efficient models which are competitive to state-of-the-art compact CNNs such as MobileNet-V2 and NasNet-Mobile. Our contribution is summarized as follows: (1) We introduce Synaptic Strength, a new class of parameters for convolution layer to indicate the importance of each connection. (2) Our approach can prune various CNN models with high compression without compromising accuracy. (3) Further investigation shows, the proposed Synaptic Strength is a better indicator for kernel pruning compare with the previous approach both in empirical results and theoretical analysis. The analysis of brain connectivity is a vast field in neuroscience with a frequent use of visual representations and an increasing need for visual analysis tools. Based on an in-depth literature review and interviews with neuroscientists, we explore high-level brain connectivity analysis tasks that need to be supported by dedicated visual analysis tools. A significant example of such a task is the comparison of different connectivity data in the form of weighted graphs. Several approaches have been suggested for graph comparison within information visualization, but the comparison of weighted graphs has not been addressed. We explored the design space of applicable visual representations and present augmented adjacency matrix and node-link visualizations. To assess which representation best support weighted graph comparison tasks, we performed a controlled experiment. Our findings suggest that matrices support these tasks well, outperforming node-link diagrams. These results have significant implications for the design of brain connectivity analysis tools that require weighted graph comparisons. They can also inform the design of visual analysis tools in other domains, e.g. comparison of weighted social networks or biological pathways. How can we build agents that keep learning from experience, quickly and efficiently, after their initial training? Here we take inspiration from the main mechanism of learning in biological brains: synaptic plasticity, carefully tuned by evolution to produce efficient lifelong learning. We show that plasticity, just like connection weights, can be optimized by gradient descent in large (millions of parameters) recurrent networks with Hebbian plastic connections. First, recurrent plastic networks with more than two million parameters can be trained to memorize and reconstruct sets of novel, high-dimensional 1000+ pixels natural images not seen during training. Crucially, traditional non-plastic recurrent networks fail to solve this task. Furthermore, trained plastic networks can also solve generic meta-learning tasks such as the Omniglot task, with competitive results and little parameter overhead. Finally, in reinforcement learning settings, plastic networks outperform a non-plastic equivalent in a maze exploration task. We conclude that differentiable plasticity may provide a powerful novel approach to the learning-to-learn problem. The brain processes information through many layers of neurons. This deep architecture is representationally powerful, but it complicates learning by making it hard to identify the responsible neurons when a mistake is made. In machine learning, the backpropagation algorithm assigns blame to a neuron by computing exactly how it contributed to an error. To do this, it multiplies error signals by matrices consisting of all the synaptic weights on the neuron's axon and farther downstream. This operation requires a precisely choreographed transport of synaptic weight information, which is thought to be impossible in the brain. Here we present a surprisingly simple algorithm for deep learning, which assigns blame by multiplying error signals by random synaptic weights. We show that a network can learn to extract useful information from signals sent through these random feedback connections. In essence, the network learns to learn. We demonstrate that this new mechanism performs as quickly and accurately as backpropagation on a variety of problems and describe the principles which underlie its function. Our demonstration provides a plausible basis for how a neuron can be adapted using error signals generated at distal locations in the brain, and thus dispels long-held assumptions about the algorithmic constraints on learning in neural circuits. Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial neural networks tend to eschew precisely designed codes, dynamics or circuits in favor of brute force optimization of a cost function, often using simple and relatively uniform initial architectures. Two recent developments have emerged within machine learning that create an opportunity to connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion and various forms of short- and long-term memory storage. Second, cost functions and training procedures have become more complex and are varied across layers and over time. Here we think about the brain in terms of these ideas. We hypothesize that (1) the brain optimizes cost functions, (2) the cost functions are diverse and differ across brain locations and over development, and (3) optimization operates within a pre-structured architecture matched to the computational problems posed by behavior. In support of these hypotheses, we argue that a range of implementations of credit assignment through multiple layers of neurons are compatible with our current knowledge of neural circuitry, and that the brain's specialized systems can be interpreted as enabling efficient optimization for specific problem classes. Such a heterogeneously optimized system, enabled by a series of interacting cost functions, serves to make learning data-efficient and precisely targeted to the needs of the organism. We suggest directions by which neuroscience could seek to refine and test these hypotheses. We propose a novel network model of spiking neurons, without preimposed topology and driven by STDP (Spike-Time-Dependent Plasticity), a temporal Hebbian unsupervised learning mode, based on biological observations of synaptic plasticity. The model is further driven by a supervised learning algorithm, based on a margin criterion, that has effect on the synaptic delays linking the network to the output neurons, with classification as a goal task. The network processing and the resulting performance are completely explainable by the concept of polychronization, recently introduced by Izhikevich Izh06NComp . On the one hand, our model can be viewed as a new machine learning concept for classifying patterns by means of spiking neuron networks. On the other hand, as a model of natural neural networks, it provides a new insight on cell assemblies, a fundamental notion for understanding the cognitive processes underlying memory. Keywords. Spiking neuron networks, Synaptic plasticity, STDP, Delay learning, Classifier, Cell assemblies, Polychronous groups. Recent years have witnessed an increasing interest in neuron–glia communication. This interest stems from the realization that glia participate in cognitive functions and information processing and are involved in many brain disorders and neurodegenerative diseases. An important process in neuron–glia communications is astrocyte encoding of synaptic information transfer—the modulation of intracellular calcium (Ca2 + ) dynamics in astrocytes in response to synaptic activity. Here, we derive and investigate a concise mathematical model for glutamate-induced astrocytic intracellular Ca2 + dynamics that captures the essential biochemical features of the regulatory pathway of inositol 1,4,5-trisphosphate (IP3). Starting from the well-known two-variable (intracellular Ca2 + and inactive IP3 receptors) Li–Rinzel model for calcium-induced calcium release, we incorporate the regulation of IP3 production and phosphorylation. Doing so, we extend it to a three-variable model (which we refer to as the ChI model) that could account for Ca2 + oscillations with endogenous IP3 metabolism. This ChI model is then further extended into the G-ChI model to include regulation of IP3 production by external glutamate signals. Compared with previous similar models, our three-variable models include a more realistic description of IP3 production and degradation pathways, lumping together their essential nonlinearities within a concise formulation. Using bifurcation analysis and time simulations, we demonstrate the existence of new putative dynamical features. The cross-couplings between IP3 and Ca2 + pathways endow the system with self-consistent oscillatory properties and favor mixed frequency–amplitude encoding modes over pure amplitude–modulation ones. These and additional results of our model are in general agreement with available experimental data and may have important implications for the role of astrocytes in the synaptic transfer of information. Networks of coupled dynamical systems have been used to model biological oscillators1,2,3,4, Josephson junction arrays5,6, excitable media7, neural networks8,9,10, spatial games11, genetic control networks12 and many other self-organizing systems. Ordinarily, the connection topology is assumed to be either completely regular or completely random. But many biological, technological and social networks lie somewhere between these two extremes. Here we explore simple models of networks that can be tuned through this middle ground: regular networks ‘rewired’ to introduce increasing amounts of disorder. We find that these systems can be highly clustered, like regular lattices, yet have small characteristic path lengths, like random graphs. We call them ‘small-world’ networks, by analogy with the small-world phenomenon13,14 (popularly known as six degrees of separation15). The neural network of the worm Caenorhabditis elegans, the power grid of the western United States, and the collaboration graph of film actors are shown to be small-world networks. Models of dynamical systems with small-world coupling display enhanced signal-propagation speed, computational power, and synchronizability. In particular, infectious diseases spread more easily in small-world networks than in regular lattices. This paper presents a new approach to neural modeling based on the idea of using an automated method to optimize the parameters of a synaptic learning rule The synaptic modi cation rule is considered as a parametric function This function has local inputs and is the same in many neurons We can use standard optimization methods to select appropriate parameters for a given type of task We also present a theoretical analysis permitting to study the generalization property of such parametric learning rules By generalization we mean the possibility for the learning rule to learn to solve new tasks Experiments were performed on three types of problems a biologically inspired circuit for conditioning in Aplysia Boolean functions linearly separable as well as non linearly separable and classi cation tasks The neural network architecture as well as the form and initial parameter values of the synaptic learning function can be designed using a priori knowledge This paper provides an easy to follow tutorial on the free-energy framework for modelling perception developed by Friston, which extends the predictive coding model of Rao and Ballard. These models assume that the sensory cortex infers the most likely values of attributes or features of sensory stimuli from the noisy inputs encoding the stimuli. Remarkably, these models describe how this inference could be implemented in a network of very simple computational elements, suggesting that this inference could be performed by biological networks of neurons. Furthermore, learning about the parameters describing the features and their uncertainty is implemented in these models by simple rules of synaptic plasticity based on Hebbian learning. This tutorial introduces the free-energy framework using very simple examples, and provides step-by-step derivations of the model. It also discusses in more detail how the model could be implemented in biological neural circuits. In particular, it presents an extended version of the model in which the neurons only sum their inputs, and synaptic plasticity only depends on activity of pre-synaptic and post-synaptic neurons. Information processing in the cerebral cortex involves interactions among distributed areas. Anatomical connectivity suggests that certain areas form local hierarchical relations such as within the visual system. Other connectivity patterns, particularly among association areas, suggest the presence of large-scale circuits without clear hierarchical relations. In this study the organization of networks in the human cerebrum was explored using resting-state functional connectivity MRI. Data from 1,000 subjects were registered using surface-based alignment. A clustering approach was employed to identify and replicate networks of functionally coupled regions across the cerebral cortex. The results revealed local networks confined to sensory and motor cortices as well as distributed networks of association regions. Within the sensory and motor cortices, functional connectivity followed topographic representations across adjacent areas. In association cortex, the connectivity patterns often showed abrupt transitions between network boundaries. Focused analyses were performed to better understand properties of network connectivity. A canonical sensory-motor pathway involving primary visual area, putative middle temporal area complex (MT+), lateral intrapariet al area, and frontal eye field was analyzed to explore how interactions might arise within and between networks. Results showed that adjacent regions of the MT+ complex demonstrate differential connectivity consistent with a hierarchical pathway that spans networks. The functional connectivity of pariet al and prefrontal association cortices was next explored. Distinct connectivity profiles of neighboring regions suggest they participate in distributed networks that, while showing evidence for interactions, are embedded within largely parallel, interdigitated circuits. We conclude by discussing the organization of these large-scale cerebral networks in relation to monkey anatomy and their potential evolutionary expansion in humans to support cognition.