Progress in science depends on the effective exchange of ideas among scientists. New ideas can be assessed and criticized in a meaningful manner only if they are formulated precisely. This applies to simulation studies as well as to experiments and theories. But after more than 50 years of neuronal network simulations, we still lack a clear and common understanding of the role of computational models in neuroscience as well as established practices for describing network models in publications. This hinders the critical evaluation of network models as well as their re-use. We analyze here 14 research papers proposing neuronal network models of different complexity and find widely varying approaches to model descriptions, with regard to both the means of description and the ordering and placement of material. We further observe great variation in the graphical representation of networks and the notation used in equations. Based on our observations, we propose a good model description practice, composed of guidelines for the organization of publications, a checklist for model descriptions, templates for tables presenting model structure, and guidelines for diagrams of networks. The main purpose of this good practice is to trigger a debate about the communication of neuronal network models in a manner comprehensible to humans, as opposed to machine-readable model description languages. We believe that the good model description practice proposed here, together with a number of other recent initiatives on data-, model-, and software-sharing, may lead to a deeper and more fruitful exchange of ideas among computational neuroscientists in years to come. We further hope that work on standardized ways of describing—and thinking about—complex neuronal networks will lead the scientific community to a clearer understanding of high-level concepts in network dynamics, and will thus lead to deeper insights into the function of the brain.
Philosophers of science have yet to develop a robust definition and interpretation of models and simulations [39]–[42]. Most of that debate focuses on models in physics, but Peck [31] gives an interesting review of models and simulations in ecology, while Aumann [32] thoroughly discusses requirements of successful modeling of ecological systems; Wooley and Lin [43] give an overview of modeling and simulation in biology. The only comparable assessment of the role of models and simulations in computational neuroscience is part of a book chapter by Clark and Eliasmith [44]. A recent appraisal of the role of models in neuroscience [45]–[47], based on a general reappraisal of the role of computational models by Humphreys [48], has mostly focused on connectionist models. We shall not attempt to provide a general analysis of models and simulations in computational neuroscience here. Our aim is more practical: to promote standards for the description of neuronal network models in the literature, to further sharing of knowledge and facilitate critique. Thus, our focus is narrower yet than that of Eliasmith and Anderson [49], Ch. 1. 5, who proposed a “Methodology” of neural engineering. For our purposes, we adopt a quite restricted working definition of a model: Several aspects of this definition deserve note: The model is first of all a mental model formed in the brain of a researcher. It is her hypothesis about the function of a part of the brain. Heinrich Hertz expressed this idea first in his textbook “Prinzipien der Mechanik” in 1894: Scientific progress depends critically on the ability of neuroscientists to communicate models, i. e. , hypotheses, among each other: When Anna presents her model to Bob and Charlie—will both build the same mental model in their minds as Anna? Or will some nuances be lost, some aspects interpreted differently, some parts misunderstood? Only a precise, unambiguous notation for models will allow Anna, Bob and Charlie to discuss their individual understandings of the model and thus to truly share models. Efficient communication dictates that scientists should use a common notation to describe their models, as it is demanding to thoroughly acquaint ourselves with any advanced notation. It is tempting to consider implementations of neuronal network models in a specific simulator software as a sufficient model description, as it is explicit, specific and describes structure and dynamics. We believe this to be a fallacy. Implementations come most often in the form of scripts or computer programs, which tend to be difficult to reverse engineer: It is simply not possible to infer the overall network structure from the bits and pieces of a large script. Secondly, most simulation scripts rely on properties hidden in a simulator, which may even change as a simulator evolves over time. Translating a given implementation first to a mental model and then to a second simulator software for independent testing, opens for errors in both translation steps. We believe that while scientific productivity benefits from sharing simulation code through repositories such as ModelDB [50] and standard languages such as NeuroML [51], implementations do not fill the need for precise human-readable model descriptions in the scientific literature. Based on experiences in systems biology, Wimalaratne et al. [36] stress that it is crucial to identify biophysical concepts as logical abstractions in order to create meaningful and re-usable model implementations. It is also worth mentioning that the translation of a mathematical model into a computer program is lossy and irreversible. The translation is lossy due to the finite precisions of computers. For example, most real numbers cannot be represented on a computer. This is obviously problematic in the analysis of chaotic systems where small errors have a big influence on the state trajectories of the system. The translation is generally not reversible, because the commonly used programming languages are not accessible to formal analysis. It is generally not even possible to prove that a function, implemented in a common language such as C++, is correct. In some cases, one may even have to add equations to models in the computer implementation to preserve stability and obtain results in agreement with experimental observation [42], [52]. While mathematical model descriptions can be treated with formal methods, their computer implementations generally cannot. This means that if we want to validate the claims about a model, we must start from the description in the scientific publication. If we start from the model implementation of the authors, we can never refute that the model may be faulty or doing something entirely different than what was claimed in the publication. Taking a given implementation of a model or hypothesis and simply executing it again does not constitute independent testing, nor does it fulfill the criterion of falsifiability: the same program run twice should yield identical results.
A complete model description must cover at least the following three components: (i) The network architecture, i. e. , the composition of the network from areas, layers, or neuronal sub-populations. (ii) The network connectivity, describing how neurons are connected among each other in the network. In most cases, connectivity will be given as a set of rules for generating the connections. (iii) The neuron and synapse models used in the network model, usually given by differential equations for the membrane potential and synaptic currents or conductances, rules for spike generation and post-spike reset. Model descriptions should also contain information about (iv) the input (stimuli) applied to the model and (v) the data recorded from the model, just as papers in experimental neuroscience do, since a reproduction of the simulations would otherwise become impossible. Neuronal network models are usually described by a combination of five means: prose (text), equations, figures, tables and pseudocode. We shall discuss these in turn. Prose is a powerful means of communicating ideas, intentions and reasons. It is flexible and, if used carefully, precise. Unfortunately, prose can easily—often unintentionally—become ambiguous. Previous knowledge and ideas in the mind of the reader will shape the reader' s understanding of a textual description of a model and may lead to misunderstandings. Prose that strives to be strictly unambiguous and provide all required detail, on the other hand, will often be difficult to read. Mathematical notation (equations) is compact and unambiguous. Suitably chosen notation compresses complex relationships in concise expressions, which allow for further manipulation in our mind, as illustrated by the matrix exponentiation in the Introduction. The now common mathematical notation emerged alongside the great scientific achievements of Newton, Leibniz and others between the 17th and 19th century [53], [54]. Unfortunately, not all mathematical notation is understood easily, and variations in notation, as is common in computational neuroscience (cf. Table 2), can present serious obstacles to effective communication. Figures communicate the architecture and connectivity of network models well, since vision is the dominating sense in most humans. Most readers will first scan the figures in a paper to get an overview of what the paper is about, using figure captions as a guide, and read the full text of the paper only later. Thus, figures and captions will shape the initial idea a reader forms about a neuronal network model, and the ideas thus established may be difficult to correct through textual description. Specifying complex networks precisely in figures can be difficult, and disciplines depending strongly on exact diagrams, such as mechanical and electrical engineering, have developed precise standards for such diagrams (see, e. g. , [55]). Systems biologists have yet to arrive at a definite standard for depicting their models, but they at least have an open debate about graphical representations [56]–[59]. Tables are a useful means of organizing data, especially model parameters. Data presented in table form is far more accessible than data dispersed throughout a text, facilitating, e. g. , comparisons of parameter choices between different papers and proof-reading of simulation scripts against papers. Pseudocode is often used to present algorithms in concise, human readable form, without resorting to a specific programming language. It will be an efficient means of communication only if the pseudocode notation is sufficiently well established to be unambiguous. The placement of model descriptions within a scientific publication depends on the focus of the paper and the journal it is published in. Traditionally, model descriptions were either given in the body text of a paper, or in an appendix. It has now become common to give only brief model overviews in the paper itself, and to relegate detailed model descriptions to supplementary material published online, or even to place simulation code online in community repositories such as ModelDB.
Figure 1 summarizes the placement of the description of architecture, connectivity and neuron and synapse models, respectively, across all papers; for details, see Tables S1, S2 and S3 in the Supporting files. All papers present at least an overview of the model they investigate in the main body of the paper. Details are frequently provided in supplementary material available online, especially in more recent papers; appendices are used to a lesser degree. Model descriptions in some papers are incomplete in the sense that the authors refer to other publications for details of neuronal dynamics in particular. Within the body text of the paper, model descriptions were placed in the “Methods” sections in 10 of the 14 papers surveyed, even though the neuronal network model is in itself a product of significant scientific analysis and synthesis [32]. As such, it would rather belong in the “Results” section of a paper. Whether the placement of the model description in the “Methods” section genuinely reflects the way in which authors perceive their models, or rather is a consequence of editorial policies shaped by “wet” neuroscience, is not clear at present. It is interesting to note in this context that papers in theoretical physics generally do not follow the strict “methods-results-discussion” pattern. We would like to point out two interesting aspects of the placement of model descriptions. First, the text of a paper manuscript, including the appendix, undergoes thorough peer review and copy editing, ensuring high standards in content and presentation. It is not, at present, clear whether all material published as supplementary material receives the same scrutiny in the review process; it is often not copy-edited to the same standards as the paper proper. Second, source code published in community repositories represents an implementation of a model, not the model itself [52]. It can thus serve only as a service to the community to facilitate code-reuse, but not to communicate the content of the model proper. Incidentally, none of the 14 papers surveyed here describes re-use of neuronal models available in repositories, such as ModelDB [50]. Nor does any paper mention that the source code for the model implemented in the paper was made available to the community, even though models from several papers are at present available from ModelDB [4], [5], [15]. In recent years, though, there appears to be a slowly growing trend to explicitly reference and re-use existing models from ModelDB; see http: //senselab. med. yale. edu/modeldb/prm. asp for an up-to-date list (Michael Hines, personal communication). Figure 2 shows that equations are mostly used to describe the dynamics of model neurons, while connections are most often presented in a combination of prose and figures, occasionally in form of pseudocode. We will review the quality of these descriptions in detail below. Table 3 shows how parameters are presented in papers. It regrettably indicates that too few authors make parameters easily accessible in tables. Network model descriptions in the literature show no consistent order of description. Among the papers surveyed here, six begin with a description of the neuron models and then proceed to network architecture, seven papers use the opposite order, while one paper mixes the description of neurons and network. We find the latter option least useful to the reader. Authors differ greatly in their efforts to anchor their models in empirical data. Destexhe et al. [4] go to great lengths to justify the design of their neuron and synapse models with respect to the neurophysiological literature. They thus provide the synthesis document proposed by Aumann [32] as the basis of any modeling effort. Unfortunately for those readers who want to investigate the resulting model, though, model description and justification are tightly intertwined in the terse methods section, making it quite demanding to extract the model description as such. Among all papers surveyed here, only Destexhe et al. [4] and Izhikevich and Edelman [8] show responses of individual synaptic conductances and individual neurons to test stimuli, while all other authors only show responses of the entire network. This means that researchers who attempt to re-implement a model and find themselves unable to reproduce the results from a paper, will not be able to find out whether problems arise from neuron model implementations or from a wrong network setup. We will now analyze in detail which difficulties arise in describing a network model, considering in turn network architecture, connectivity, and neuron models, and point out examples of good descriptions. Descriptions of network architecture become challenging as network complexity increases. Networks with a small number of populations, random connectivity and no spatial structure are easily described in a few lines of prose, as in Brunel' s paper [3]. A combination of prose and simple figures is usually sufficient to describe architecture of networks composed from a small number of one- or two-dimensional layers of individual neurons; examples are Destexhe et al. [4] and Kirkland and Gerstein [9]. Complex models spanning several brain areas with detailed spatial, layered, and functional substructure, such as Lumer et al. [10] and Izhikevich and Edelman [8], are more challenging to describe. Authors generally adopt a top-down approach, giving first an overview of the brain areas involved, before detailing the structure of the individual areas. In models of systems with clearly defined signal flow, areas are often visited in the predominant order of signal flow [6], [7], [14], while others present the more complex cortical structures before descending to subcortical structures [8], [10]. The most detailed explicit model studied here is the thalamocortical model presented by Lumer et al. [10]. The description of the cortical areas in this model (Vp and Vs), while complete, lacks in our opinion the clarity desirable of a good model description, and may thus help to identify rules for ideal model descriptions. For one, discussions on model design and properties are embedded in the model description, e. g. , the reduction of a total of 32 “combinations of response selectivities” to just two included in the model, and a comparison of the number of neurons in the model to that found in animals. We believe that design decisions and model review should be kept separate from the model description proper for the sake of clarity, since they are independent intellectual endeavours [32]. Second, Lumer et al. mix different views of their layer architecture without providing sufficient guidance to the reader. They begin by describing the Vp layer as a grid of 8×8 macro-units, with two “selectivities within a macro-unit”, each containing “a collection of 5×5 topographic elements, each of which corresponded to a contiguous location in retinal space”, before proceeding to state that “[t]opographic elements in Vp were organized in maps of 40×40 elements for each of the two modeled orientation selectivities. ” We find it difficult to interpret this description unambiguously. We are in particular in doubt about the localization of macro-units and topographic elements in retinal space. In our view, the most parsimonious interpretation is as follows: 5×5 topographic elements placed in each of 8×8 macro-units result in a grid of 40×40 topographic elements. ” This interpretation is sketched in Fig. 3. Another interesting aspect is that model composition is often described from a perspective orthogonal to the description of connections. Lumer et al. [10], e. g. , present the primary thalamus and cortex as grids of 40×40 topographical units, each containing an excitatory and an inhibitory neuron (thalamus) and a microcolumn composed of 10 neurons organized in three laminae (cortex). Connections are then described by looking at this architecture from an orthogonal perspective: Thalamus is described as two layers, one of excitatory and one of inhibitory neurons, while cortex is split into six layers, one of excitatory and one of inhibitory neurons for each of the three laminae in the model. We believe that it may be more sensible to base the model description on the perspective used in defining connections, as connectivity is the central aspect of a network model. Izhikevich and Edelman [8] present a significantly more complex model, covering the entire human cortex and thalamus. Concerning the spatial placement, they only state that “[n]euronal bodies are allocated randomly on the cortical surface, whose coordinates were obtained from anatomical MRI. ” No further information is given on how MRI measurements were converted to neuron densities in space. Thus, even if one had access to MRI data of the human brain, it would be difficult to reproduce the neuron distribution investigated by Izhikevich and Edelman. In such cases it would be advantageous to either use datasets available from community databases or to make data available to others. Figures of network architecture vary widely between papers. We will discuss them in the following section together with connections. Describing the connections well is the most challenging task in presenting a neuronal network model. For networks with random connections and no spatial structure, connectivity is easily described in a few sentences [3]. Haeusler and Maass [5] additionally represent connection strengths and probabilities in a figure; this works well for their six-population model. If yet more populations were involved, such a figure would soon become cluttered, and it becomes more useful to present connection parameters in tables, cf. supplementary material in ref. [8]. Even in these simple networks, care must be taken to specify details: Few authors are explicit on all these points, although these choices may have significant consequences for network dynamics (Tom Tetzlaff, personal communication; see also Kriener et al. [60]). Models incorporating spatial structure have more complex connection patterns, which we will call topographic connections, since they usually describe the spatial distribution of connection targets relative to the spatial location of the sending neuron, i. e. , connections are typically described as divergent connections. In most cases, connections have a random component: they are created with a certain probability. In simple cases, such as Kirkland and Gerstein [9], connections are made to neurons in a rectangular mask with equal probability. In more complex models, connection probability depends on the relative locations of the neurons that are candidates for a connection, e. g. , [10], [11]. Unfortunately, few authors provide the equations for these probability functions; Mariño et al. [11] is a laudable exception. It is somewhat paradoxical if papers present long tables of parameters for these connection probability functions, but do not provide the equation into which these parameters enter. Mariño et al. [11] are the only authors who explicitly discuss self-connections (in their supplementary material), and as far as we can see, no authors have discussed whether multiple connections between any two neurons may be created. Another neglected issue is precisely how probabilistic connections are created. The following approach seems to be implied: For each pair of neurons from the sender and target population, a connection is created if a random number is smaller than the connection probability for the pair. But one might equally well determine the total number of connections to be made first, and then distribute the connections according to the spatial probability profile [61]. Such schemes offer significant performance gains [62]. A complete specification of the connection algorithm should thus be given. Among the papers surveyed, Izhikevich and Edelman [8] has by far the most complex connectivity and the authors go to great lengths to present gray-matter connectivity in figures, tables, and prose. Alas, some information appears to be missing: It is not clear from the text exactly how connections are distributed within the axonal spans, and how they are distributed across dendritic compartments of neurons with more than one compartment in a cortical layer. We have also been unable to find specific information on how synaptic weights and delays were assigned to connections. Finally, no details are provided about the white-matter (long-range) connections, which were based on diffusion-tensor imaging (DTI) data. Without access to the DTI data it is thus impossible to re-implement the model presented. Paper authors draw network diagrams in quite different ways, both in the overall style of their diagrams and in use of symbols. Figure 4 shows network diagrams of a model loosely based on Einevoll and Plesser [63], Fig. 3, drawn in the style of three of the papers surveyed here. The diagram in the style of Hayot and Tranchina [6] (Fig. 4A) gives a reduced but clear overview of the overall architecture of the model; it provides no details. The style of Haeusler and Maass [5] (Fig. 4B) carries most information, with weights and probabilities shown next to connection lines, and line widths proportional to the product of weight and probability. Figure 4C, which imitates the style of Lumer et al. [10], is rather illustrative: it provides no quantitative information and the structure of the connectivity is less prominent than in the other two figures; on the other hand, it is the only figure hinting at the spatial structure of the network. Interestingly, all three diagram styles use different ways of marking excitatory and inhibitory connections: bars vs circles, black vs red, and arrows vs bars. Indeed, bars at the end of connection lines mark excitatory connections in Hayot and Tranchina' s style, but inhibitory connections in the style of Lumer et al, nicely illustrating the lack of standards in the field. Izhikevich and Edelman [8] have illustrated their brain model using diagrams presenting significantly more detail than in the diagrams shown in our Fig. 4. Unfortunately, we cannot reproduce Figures 2 and 8 from the supplementary material of the paper by Izhikevich and Edelman here due to copyright issues; the figures are available on the internet at http: //www. pnas. org/content/105/9/3593. figures-only and http: //www. pnas. org/content/105/9/3593/suppl/DC1, respectively. Their diagrams, though, provide so much detail of interest to the re-implementer, that the reader will have difficulty to form a clear conceptual model from the diagram. This is in many ways the curse of complex models as the following analogy may illustrate: when a physicist or electrical engineer sees a diagram of an RLC circuit, she will intuitively “see” the circuit oscillate. When presented with the complete wiring diagram for a modern analog radio receiver, though, it is hardly likely she will “hear the music”. The figure in the style of Haeusler and Maass [5] takes a middle ground. Since the individual populations are homogeneous, they can be represented by one circle each, with annotated lines providing information about connection structure and parameters. By marking connection strength through line width and differentiating excitation and inhibition by line color, the figure appeals quite directly to our intuition. It is clear, though, that any further populations would increase the complexity of the diagram to the point of illegibility. There is no established standard for the order in which connections within a network are described. Some authors proceed from local connectivity (e. g. , intracortical intralaminar) towards global connectivity [10]. Others rather follow the signal flow through the network, from retina via LGN to cortex, e. g. , Kirkland and Gerstein [9], Hayot and Tranchina [6], and Troyer et al. [14]. Neuron and synapse models are commonly described by a mixture of prose and equations, cf. Fig. 2; tables are used inconsistently to present parameters, see Table 3. Some authors do not provide complete model specifications in their paper, but rely heavily [4] or even entirely [5] on references to earlier work. While the desire to avoid repetition is understandable, we believe that authors here walk a thin line toward incomprehensibility, especially if the models used are spread over three or more publications. Even though the re-use of neuron model implementations provided in repositories such as ModelDB may save effort and contribute to a standardization in the field, none of the papers we studied made use of available model implementations—or the authors failed to point out that they did. Table 2 shows the membrane potential equations found in several papers and demonstrates that there is a reasonable amount of variation in the way this central equation is written down. There is in particular no widespread agreement on whether to include the membrane capacitance explicitly in the equation or rather to subsume it in a membrane time constant. Some authors, such as Tao et al. [13], even chose to normalize the membrane potential equation by defining. Yet greater variation is found in the representation of synaptic currents. This means that phrases such as “we use the standard equations for integrate-and-fire neurons”, which are not uncommon in the literature, are essentially meaningless, since there are no established “standard equations” for integrate-and-fire neurons. Spike generation and detection, including subsequent reset and refractory behavior, are usually described in prose, sometimes with interspersed equations. “ was reset to …, when it exceeded a threshold of … −51 mV …, at which point a spike was recorded, and relayed …, ” is a typical formulation [10]. Unfortunately, it does not state precisely how threshold crossings are detected, which times are assigned to spikes, or when exactly the reset is executed. All these issues can have significant consequences for network dynamics [64]–[66]. The previous sections have documented a wide variety of approaches to model descriptions in the literature. We believe that this variety is detrimental to the field, as it makes it difficult to communicate neuronal network models correctly and efficiently. At the same time, we believe that the field of computational neuroscience is too young to establish exacting standards for model descriptions. We will return to this problem and its various causes in the discussion. As a middle road, we propose to establish a good model description practice for the scientific literature. We will refer to it as “good practice” below for brevity. Some of our suggestions are motivated by a recent analysis of modeling techniques in ecology [32], but see also [49]. We propose a practice with the following elements: We will discuss these elements in turn below, followed by more detailed discussions about how to render specific aspects of a network model. As an illustrative example, Figures 5 and 6 provide a concise description of the Brunel [3] model following the good practice format. A similar description of the Lumer et al. [10] model is given in Figures 7–9. We would like to stress that we present the good practice here to stimulate the debate on model descriptions within the computational neuroscience community. If it is adopted widely throughout the community, it will provide numerous advantages: authors will have guidelines that will allow them to check their descriptions for completeness and unambiguousness; referees will more easily be able to assess the correctness and quality of a model; and readers will find it easier to comprehend and re-implement a model, and to compare different models.
The model survey presented here revealed a wide variety of approaches to describing the composition and connectivity of neuronal networks. We believe that this is, at least in part, due to a lack of common high-level concepts for composition and connectivity from a modeling perspective. Developing such high-level concepts describing, e. g. , certain types of randomized connectivity patterns, is thus an important task for the computational neuroscience community. The challenge at hand is perhaps best clarified when trying to draw diagrams representing neuronal network models. Such diagrams have two aims: To give the reader an intuitive understanding of model properties central to the dynamics of the model, and to unambiguously provide the necessary detail to allow a reconstruction of a model. In the absence of a mathematical formalism for model specification, diagrams often seem better suited than prose to present unambiguous detail. Simple models, such as that by Brunel [3], can be depicted in a single diagram, as illustrated in Fig. 6. The four panels in that figure, though, show that one may choose from a wide variety of styles for such diagrams, and it is not a priori clear which style is best. In panels A–C in the figure we propose three ways to differentiate between excitatory and inhibitory connections (line styles and endings) as well as to mark connectivity patterns (line endings, styles, annotations). Panel D differs from the other three in the way the external input is represented. Brunel [3] states that “[each neuron] receives connections from excitatory neurons outside the network. External synapses are activated by independent Poisson processes with rate. ” This is rendered in detail in panel D, which shows Poisson generators per modeled neuron. In all other panels, these generators have been collapsed into an external excitatory population with the implicit assumption that this population contains the correct number of Poisson generators required by the model. Presenting complex models is even more challenging. In Fig. 9, we present a set of three figures describing the model by Lumer et al. [10] at three levels of hierarchy: an overall view in panel A, details of the connectivity within the cortical populations tuned to vertical stimuli in panel B, and finally details of projection patterns into a single cortical population in panel C. All figures are simplifications of the full model, since we have left out the secondary thalamic and cortical areas. We are currently pursuing research to identify drawing styles and a hierarchy of diagrams that will be intuitive to a majority of computational neuroscientists and provide the necessary detail. Results will be presented elsewhere. Given the importance of comprehensible and precise model descriptions, it may seem surprising that no standards or good practices have emerged in computational neuroscience to date. Early proposals, such as the Neural Simulation Language [73] (see also Eliasmith and Anderson [49], Ch. 1. 5 and Kumar [74]), have not been accepted widely in the community. At present, two developments appear promising. NetworkML, which is part of the NeuroML project [28], [51], provides a simulator-agnostic XML-based declarative standard for neuron network model descriptions. Simulation code for tools such as Neuron and Genesis can be generated from models defined in NeuroML. PyNN [29], in contrast, is an imperative scripting language that can control a number of common neuronal network simulators, such as NEST, Neuron, and Brian. One reason why neither NetworkML nor PyNN has yet caught on as a means of widespread model exchange may be that neither of the two languages seems to aim at providing human-comprehensible model descriptions that might be included in publications. Another reason for the lack of model description standards may be that computational neuroscience has to a large degree been an ancillary science, an appendix of electrophysiology: The vast majority of publications in computational neuroscience compares its modeling results directly to specific sets of experimental data. And even though models have driven the development in some fields of neuroscience [75], very few authors have compared the properties of different models with each other; Erwin et al. [76] is a notable exception. De Schutter [77] even argues that there currently is a trend away from the investigation of models as such, and back to a one-to-one matching of models to experiments. As long as computational neuroscientists focus on matching their models to specific experiments, rather than either to spar their models against each other, or build their models upon each other, the motivation to use a standard notation is obviously limited. We have no doubt that model sharing will increase in computational neuroscience in years to come. This raises the question of what model sharing precisely entails. At the simplest level, models may be shared as simulator code. While this seems convenient at first, it carries significant risk, as any code is likely to contain errors, in particular errors that may surface only once an existing model is used in a different context than the one in which it was originally developed. Indeed, in at least one case, high-profile publications (outside neuroscience), had to be retracted after a subtle programming error was discovered in a widely shared scientific software [78]. Some scientists argue that everyone in a field should use the same, carefully maintained simulation software to avoid such problems, and to make computational science reliable [79]. We beg to differ: monoculture tends to create more problems than it solves. Establishing a new publication culture in computational neuroscience will require considerable effort within the community. We hope that the good model description practice that we have outlined in the previous section may be a good starting point. We believe in particular that a clear segregation of model derivation, model description, implementation, and model analysis, as proposed above, will make it easier for readers to discern the model as such, compare it to other models, and evaluate its relevance to their own research. The proposed Checklists for model descriptions will help to ensure that model descriptions themselves are reasonably complete and follow a common pattern, further improving the communication of models, while the Templates for tables invite a standardized presentation of details on various aspects of models; similarly, the Guidelines for diagrams should aid authors in illustrating their network models. Since all our proposals are informal, we hope that authors will find it straightforward to apply them when describing their network models, thus establishing a de facto standard for model descriptions. We are optimistic that we are beginning to see changes towards more cooperation within computational neuroscience, as witnessed by several collaborative reports on neuronal network simulations in the last two years [25]–[27] and the development of tools for the integration of various simulation software [29], [30], much helped by the establishment of the International Neuroinformatics Coordinating Facility (INCF) in 2005. The Connection Set Algebra proposed by Djurfeldt [72] is an encouraging step towards establishing high-level concepts for neuronal network descriptions, i. e. , giving us a concise language to talk about our models. There is also much to be learned from model sharing and curation efforts in other communities, such as the IUPS Physiome and the European Virtual physiological human projects [37], [80]. In closing, let us return to the power of notation, as exemplified by the matrix notation in the introduction. In July 1924, Werner Heisenberg gave a manuscript full of complicated mathematics to his mentor Max Born, unsure whether it was worth publishing. Born worked through Heisenberg' s ideas and realized that what Heisenberg had written down, actually amounted to the matrix mechanics of quantum theory. This insight of Born' s unleashed the full power of Heisenberg' s ideas and let Born discover the non-commutativity of quantum mechanics [81], p. 125f. We are looking forward to the day when a good formalism will give us deeper insights into the secrets of signal processing in the brain.
Scientists make precise, testable statements about their observations and models of nature. Other scientists can then evaluate these statements and attempt to reproduce or extend them. Results that cannot be reproduced will be duly criticized to arrive at better interpretations of experimental results or better models. Over time, this discourse develops our joint scientific knowledge. A crucial condition for this process is that scientists can describe their own models in a manner that is precise and comprehensible to others. We analyze in this paper how well models of neuronal networks are described in the scientific literature and conclude that the wide variety of manners in which network models are described makes it difficult to communicate models successfully. We propose a good model description practice to improve the communication of neuronal network models.
neuroscience/theoretical neuroscience
computational biology/computational neuroscience
Towards Reproducible Descriptions of Neuronal Network Models