article: Deeper exploration of the brain’s vast synaptic networks will require new tools for high-throughput structural and molecular profiling of the diverse populations of synapses that compose those networks. Fluorescence microscopy (FM) and electron microscopy (EM) offer complementary advantages and disadvantages for single-synapse analysis. FM combines exquisite molecular discrimination capacities with high speed and low cost, but rigorous discrimination between synaptic and non-synaptic fluorescence signals is challenging. In contrast, EM remains the gold standard for reliable identification of a synapse, but offers only limited molecular discrimination and is slow and costly. To develop and test single-synapse image analysis methods, we have used datasets from conjugate array tomography (cAT), which provides voxel-conjugate FM and EM (annotated) images of the same individual synapses. We report a novel unsupervised probabilistic method for detection of synapses from multiplex FM (muxFM) image data, and evaluate this method both by comparison to EM gold standard annotated data and by examining its capacity to reproduce known important features of cortical synapse distributions. The proposed probabilistic model-based synapse detector accepts molecular-morphological synapse models as user queries, and delivers a volumetric map of the probability that each voxel represents part of a synapse. Taking human annotation of cAT EM data as ground truth, we show that our algorithm detects synapses from muxFM data alone as successfully as human annotators seeing only the muxFM data, and accurately reproduces known architectural features of cortical synapse distributions. This approach opens the door to data-driven discovery of new synapse types and their density. We suggest that our probabilistic synapse detector will also be useful for analysis of standard confocal and super-resolution FM images, where EM cross-validation is not practical.
  Deeper understanding of the basic mechanisms and pathologies of the brain’s synaptic networks will require advances in our quantitative understanding of structural, molecular, and functional diversity within the vast populations of individual synapses that define those networks [1] [2] [3] [4]. Regardless of the subject of interest, synapse heterogeneity makes assay at the single-synapse level paramount. Here, we introduce and characterize a novel image analysis method for automated detection and molecular measurement of individual synapses and single-synapse molecular profiling of diverse synapse populations from multiplex fluorescence microscopy (muxFM) image data. The proposed methodology for structural identification and molecular analysis of single synapses at scale will be an enabling step toward deeper experimental analysis of the relationships between synaptic structure, molecules, and function. Reliable, high-throughput methods for large-scale synapse detection will also help to analyze volume images large enough to contain complete neural arbors, and thus to allow discernment of the relationships between detected synapses and their presynaptic and postsynaptic parent neurons [5]. The synapse detection methodology described here is not the first to grapple with the challenges of detecting synapses in immunofluorescence images [6] [7] [8] [9] [10]. The special utility and novelty of this tool partially lies in (1) producing outputs in the form of probability maps, reflecting the limited certainty with which synapses can be detected by most experimental modalities [11], and (2) the superior utility for both interactive and algorithmic exploration which is conferred by the query-based architecture resulting from the unsupervised framework. The probabilistic detection algorithm we introduce has perhaps its closest precedent in probabilistic synapse detectors that were introduced recently for the analysis of Focussed Ion Beam Scanning Electron Microscope (FIBSEM) images [12] [13]. The relationship in particular with [8] will be further discussed later in this paper. Single-synapse profiling of large and diverse synapse populations poses formidable challenges [14] [15] [16]. Electron microscopy (EM) of appropriately labeled specimens defines the current ‘gold standard’ for synapse detection: the nanometer resolution of EM is necessary for the unambiguous identification of defining synaptic features such as presynaptic vesicles, synaptic clefts, and postsynaptic densities [17] [18]. Unfortunately EM data acquisition is technically difficult, slow, burdened by large data processing and storage requirements, and offers only limited capacities to discriminate amongst the hundreds of different synaptic proteins that constitute the synaptic proteome. In contrast, fluorescence microscopy (FM) of tagged specimens is much faster and less expensive, easier to segment for analysis, and offers much greater molecular discrimination power. Unfortunately, the ability of FM to detect and discriminate individual synapses is compromised by resolution limits and the close crowding of synapses in most neural tissue specimens of interest. Robust FM detection of synapses is nonetheless potentially possible by combining measures that extend resolution limits and multiplexing for localization and co-localization of synaptic markers. In designing the algorithm and software reported here, we first relied on images acquired with conjugate array tomography (cAT), which combines the strengths of FM-AT with those of electron microscopic array tomography (EM-AT), allowing both EM and muxFM imaging of individual cortical specimens. Array tomography’s ultrathin physical sectioning provides z-axis resolution far beyond the light microscopic diffraction limit, as well as high sensitivity and high lateral resolution, while greatly simplifying voxel-conjugate registration of FM-AT and EM-AT images. FM-AT can moreover multiplex large numbers of synaptic markers by its combination of sequential and spectral label multiplexing. Thus, cAT provides an ideal platform for the development and rigorous design and testing of algorithms aimed at single-synapse molecular analysis and population molecular profiling. The remarkable structural and molecular diversity within mammalian synapse populations challenges our present biological understanding of how to define a synapse [11]. Difficulties also arise from a very broad distribution of synapse size, with the smallest synapses occurring at the highest frequencies. Thus, detection of a synapse inevitably involves setting some minimum-size criterion for any candidate cell-cell contact specialization to qualify as a synapse. For FM, the ‘size’ metric is typically the intensity of one or more fluorescent synaptic protein tags. The fact that there are clearly non-synaptic ‘backgrounds’, and that the observed size distributions are log-normal, enforces high sensitivity of synapse detection on some rather arbitrary threshold minimum size value. This sensitivity in turn makes key results of widespread interest, such as the synaptic density in a region or the presence/absence of a synapse at a given microscopic site, uncomfortably dependent on that same size threshold value. The probabilistic synapse detector proposed here may lead both to relief from such arbitrary-threshold (parameters) and to improvements in our biological understanding of what defines a synapse. The unsupervised probabilistic synapse detector reported here accepts molecular-morphological synapse models in the form of user queries, and delivers a volumetric map of the probability that each voxel represents part of a synapse. These maps can then be used directly to detect, classify, and map putative synapses, with confidence statistics for each. Taking human annotation of cAT EM data as ground truth, we show that our algorithm detects synapses from muxFM data alone as effectively as human annotators (while seeing only the muxFM data), and can reproduce known architectural features of cortical synapse distributions. The algorithm is actually validated with the most comprehensive AT datasets currently available. Though we here address only array tomography image data, our probabilistic synapse detector may also be useful for analysis of widely available confocal and super-resolution FM images.
  The proposed algorithm is inspired from biological knowledge of synapse characteristics. Synapses include two major structural components: a presynaptic terminal and a postsynaptic terminal. Detecting synapses using data from immunofluorescence imaging involves identifying such adjacent presynaptic and postsynaptic antibody markers, as shown in Fig 1, which diagrams the locations of four major excitatory synaptic proteins. Fig 2 is an example of an excitatory synapse with images of presynaptic and postsynaptic antibody markers (synapsin and PSD-95) overlaid upon an EM image. For this example, only two antibody markers are shown for visual simplicity—in practice, any number of presynaptic or postsynaptic antibody markers may be used by the proposed algorithm for synapse detection. Manual synapse identification involves determining the punctum size and brightness in one channel, and then considering adjacency to similarly-defined puncta in other channels. However, without corresponding EM data, detections using only IF data have an associated degree of uncertainty. Thus, we propose a query-based probabilistic synapse detection method that reflects the thought process underlying expert manual synapse detection. The first step is to distinguish signal from background noise. This calculation encodes the probability that the pixel value represents authentic antigen detection. The second step is to determine whether the foreground pixels correspond to a 2D punctum, since photons emanating from only a single pixel usually reflect noise. Therefore, adjacent ‘positive’ pixels, more likely to reflect a synaptic punctum, are augmented. Third, puncta that span multiple slices have a higher probability of belonging to a synapse than those that do not. To visualize this effect, the probability of a punctum belonging to a synapse is attenuated based on whether the prospective punctum spans multiple slices. The last step in computing the synapse probability map is to evaluate the presence of adjacent presynaptic and postsynaptic puncta by correlating the corresponding IF volumes. This produces a probability map, where the value at each voxel is the probability it belongs to a synapse. This algorithm provides a general framework for the evaluation of a wide variety of synapse subtypes, user-defined by setting the presynaptic and postsynaptic antibodies and puncta size. The following sections describe in detail each step in the process, as diagrammed in Fig 3. Before that, let us relate our proposed approach with the current state-of-the-art method [8], which inspired this work. The method in [8] requires manually annotating a large number of excitatory synapses using the EM data, and then using this as labeled data for training (supervised training). EM data allows the user to differentiate between symmetric and asymmetric synapses, but does not allow for subtype identification (limited labels/supervision). Thus, the support vector machine (SVM) classifier used in [8] is trained with synapses containing the marker for PSD-95, but does not take into account synapses without the PSD-95 marker (limited classes in the supervision). In contrast, the approach here proposed and detailed below is unsupervised, allowing the user to detect synapses with multiple proteometric compositions without first using other methods to identify large numbers of synapses for training. Our proposed approach does not require the user to manually inspect associated EM for training data; in fact, we do not require associated EM data with the IF data. Instead, we enable the user to ‘define’ (biologically inspired) a synapse by specifying which synaptic markers should be present (query) and what the minimum size of those markers should be, allowing a more class specific synaptic search. This is critical also for the discovery of new types of synapses, exploitation of new markers, and data-based discovery. The method in [8] will need re-training for every new class they want to find in the IF (potentially even new data acquisition protocols as well). There is a direct numerical comparison of the two methods in the experimental section, showing that the proposed algorithm is not only unsupervised and more broadly applicable than [8], but it actually outperforms it in the cases where both methods can be used. Raw immunofluorescence image data is noisy; for example, speckles of the antibody markers often bind with cellular structures not associated with synapses, such as mitochondria. In addition, fluorescence imagery contains signal from sources other than fluorescently-labelled antibodies, e. g. from background autofluorescence. Finally, all digital imagery contains inherent noise from sources such as camera read noise and photon shot noise. The noise produced by these sources is usually smaller in magnitude than that originating from authentic synaptic labeling, but it cannot simply be filtered out and dismissed from consideration, since the signal may originate from a true synaptic site, and we want to allow for the possibility that a concordance of weak evidence will lead to the detection of a synapse. Thus, the first step of the algorithm consists of differentiating the bright voxels, the foreground (potential objects of interest), from a noisy background in a probabilistic fashion. IF data volumes, when stained for synaptic markers, are also extremely sparse—approximately 2% of the voxels in the dataset belong to the foreground, as indicated in Fig 4. Therefore, the IF image volume can be used to approximate the distribution of the background noise. Let v (x, y, z) be the intensity value of a voxel at position (x, y) in slice z, for a given channel of the IF data. A probabilistic model, pB, is computed which characterizes all the pixels that belong to the background, which includes approximately 98% of the voxels. The background noise model is computed independently for each slice to account for variations in tissue and imaging properties. The background model pB is assumed to be a Gaussian distribution, whose mean and variance (μ B, σ B 2) are empirically computed from each slice z (the z index is omitted in Eq 1 for simplicity of notation). Then, the probability of a voxel belonging to the background, i. e. not being ‘bright’, is given by p B (x, y, z) = 1 σ B 2 π ∫ v (x, y, z) ∞ e - (t - μ B) 2 2 σ B 2 d t. (1) Therefore, the probability of a voxel associated with the foreground, pF, is computed as p F (x, y, z) = 1 - p B (x, y, z). (2) Fig 5 shows an example of the transformation from the raw data to the foreground probability map. Once foreground pixels have been identified in a probabilistic fashion, the next step is to determine if they form a 2D punctum. Since synapses appear as bright puncta in the IF image data, voxels which form puncta should have a higher probability of being associated with a synapse than those which do not. The probability of a voxel belonging to a 2D punctum, pP, is computed by multiplying the voxel’s foreground probability by that of its neighbors in a predefined neighborhood region, p P (x, y, z) = ∏ i = x - W x + W ∏ j = y - W y + W p F (i, j, z), (3) where W is the neighborhood size, defined by the smallest expected punctum size. These operations are analogous to applying a box filter on the logarithm of the probability map, for computational efficiency. In our experiments, W was set to be slightly larger than the size of the point spread function of the microscopes used. Fig 6 shows an example of the foreground probability map and the corresponding 2D puncta probability map. This step transforms each pixel’s value from representing the probability it belongs to the foreground to the probability it belongs to both the foreground and to a punctum. Potential synaptic puncta can span multiple slices of a given channel; puncta that span multiple slices have a higher probability of being associated with a synapse than those that do not. Therefore, we propose a factor f (x, y, z) which diminishes the probability values associated with voxels which do not maintain a similar probability value in adjacent slices, f (x, y, z) = exp - ∑ j = j s t a r t j = j e n d [ p P (x, y, z) - p P (x, y, z + j) ] 2. (4) The pixel’s 2D puncta probability is compared to that of its neighbor in slice (s) before, jstart, and slice (s) after, jend. The number of slices compared is dependent on the input size parameter for each antibody. The factor attenuates values for 2D puncta that do not span the required number of slices, as shown in Fig 7. The 3D puncta probability map is then computed by multiplying the 2D puncta probability map by this factor, p 3 D P (x, y, z) = p P (x, y, z) f (x, y, z), (5) which further improves the probability of a detection by considering the slice-to-slice spatial distribution, going from 2D to 3D. In electron microscopic images, synapses are identified by the presence of synaptic vesicles on the presynaptic side, the close adjacency of the membranes of the presynaptic axon terminal to a postsynaptic dendrite or dendritic spine, and the presence of a distinct postsynaptic specialization, as diagrammed in Fig 1. Synapses are identified in IF data by the close spatial arrangement of pre- and postsynaptic antibody markers, which correspond to proteins associated with synapses. Therefore, the next step in our approach is to look for the presence of presynaptic puncta in the neighborhood of postsynaptic puncta. More precisely, for each postsynaptic antibody voxel (i. e. , PSD-95), we search in the adjacent 3D neighborhood of the corresponding presynaptic (i. e. , synapsin) volume for a high intensity probability signal. To accomplish this, a rectangular grid is defined in the presynaptic channels around each postsynaptic voxel, as shown in Fig 8. The size of the grid is defined by the initial query parameters, which depend on both the inherent biology and microscope resolution. The logarithm of the 3D puncta probability map Eq (5) is integrated in each grid location and the maximum is taken as presynaptic signal level around the given postsynaptic location, log p p r e s = max k ∑ G k log p 3 D P (p r e s y n a p t i c), (6) where the grid G is centered at the current voxel (x, y, z) and divided into K × K × K subregions Gk. To search in a grid around a defined voxel location for the presynaptic signal, K is set to 3. When searching for the postsynaptic signal, K is set to 1 since postsynaptic signals are expected to loosely co-localize. These values can be adopted to the data resolution. The postsynaptic antibody pixel probability p p o s t = p 3 D P (p o s t s y n a p t i c), (7) is multiplied by the presynaptic probability to obtain the desired probability map: p s y n a p (x, y, z) = p p r e s (x, y, z) p p o s t (x, y, z). (8) Again, the probability information is here maintained (Fig 9), now including the morphological relationship between the channels. This ‘grid’ like approach allows the method to be robust to slight image alignment and registration issues, as well as to deconvolution artifacts.
  The proposed method was evaluated on the array tomography dataset published in [19], which contains a portion of the mouse barrel cortex extending from Layer 3 to Layer 5. Unlike the conjugate array tomography dataset, there is no associated EM imagery. This larger series of datasets includes 11 volumes representing a total of 2,306,233 μm3 of cortical volume. Since no gold standard is available for these data, the proposed method was evaluated by verifying known properties of the dataset: there is an approximately ten-to-one ratio of excitatory to inhibitory synapses [24], and there are more inhibitory synapses in Layer 4 than Layer 5 in the mouse barrel cortex [25] [26] [27]. For this dataset, the query parameters were adjusted to reflect the different synaptic markers used. Tables 7 and 8 list the query parameters used for detecting both inhibitory and excitatory synapses, similar to those in [7] [8] [19].
  The proposed synapse detection method serves the potential future needs of both basic and clinical neuroscience. Methods for large-scale synapse detection could analyze image volumes large enough to contain complete neural arbors, and thus allow the discernment of the relationships between detected synapses and their presynaptic and postsynaptic parent neurons. An understanding of the statistics of synapse variation in any given synaptic network is certain to be critical to interpreting and modeling results of mechanistic physiological study. Moreover, advances in imaging methods for tracing complete axonal and dendritic arbors [35] are likely to allow network analyses at the level of individual neurons and their synaptic connections, which might be optimally detected and measured by probabilistic means like those introduced here. When combined with complete arbor measurements [36], emerging methods for in situ measurement of single-cell transcriptomes [37] should allow single-synapse measurements to be associated with specific presynaptic and postsynaptic parent neurons of known transcriptomic profiles. Such capacities are likely to enhance our understanding of the molecular origins of synapse diversity. On the clinical side, the analysis tool we introduce here is likely to advance our abilities to detect possible abnormalities of synapse population statistics that have long been hypothesized to underlie a wide variety of mental and neurological disorders [38] [39] [40] [41] [42] [43] [44]. Quantification of synapse populations, in human postmortem and biopsy specimens, [45], and in animal models of disease, has already provided important insights into disease etiologies [39] [45] [42]. More reliable measurements based on probabilistic tools like those introduced here seem likely to facilitate future efforts to better understand disease mechanisms and to develop the quantitative assays essential to the discovery of effective therapies [46]. This work introduces a model-based unsupervised synapse detection algorithm that incorporates fundamental biological knowledge of how synapses are identified in the immunofluorescence data. We created a series of probabilistic excitatory detectors for various subtypes of synapses, and included the 3D spatial relationships typical of synaptic structures. This novel approach provides a probabilistic-based detection algorithm yielding not only detection but detection with confidence values. The implementation of synapse detection as a probability map (i. e. , probability of each pixel belonging to a synapse), as opposed to a binary detection / no-detection result may provide a powerful tool to assist experts throughout the exploratory process to gain new insights from the immunofluorescence data, including potentially discovering new subtypes of synapses. The influence of different biological and AT components on the actual probability values, from the noise of the system to the expression level of the proteins and the subclass of the synapses, an important new topic of investigation, will become possible when the proposed algorithm is applied to large new datasets, currently being generated. Creating conjugate Array Tomography datasets require specialized equipment, including the use of a Field Emission Scanning Electron Microscope (FESEM) to provide ground truth validation of synapse results. The computational work presented in this paper, together with the publicly available code and data, is a step in the direction of making this kind of analysis robust enough to no longer require expensive FESEM validation. The algorithm is computationally very simple and the only parameters are the user’s definition of a synapse subtype, rendering it ready for massive datasets. The results were validated with the best available cAT and AT data, producing state-of-the-art results without the need for supervised training. As demonstrated here, the proposed framework can be exploited for the explicit detection of synapses or their properties, the latter being critical for the discovery of new subtypes as well as the patterns of distributions of known subtypes. These, and the potential consequences of the approach here proposed to other modalities, are the subjects of our current efforts.
summary: Brain function results from communication between neurons connected by complex synaptic networks. Synapses are themselves highly complex and diverse signaling machines, containing protein products of hundreds of different genes, some in hundreds of copies, precisely arranged at each individual synapse. Synapses are fundamental not only to synaptic network function but also to network development, adaptation, and memory. In addition, abnormalities of synapse numbers or their molecular components have been implicated in a variety of mental and neurological disorders. Despite their obvious importance, mammalian synapse populations have so far resisted detailed quantitative study. In human brains and most animal nervous systems, synapses are very small and very densely packed: there are approximately 1 billion synapses per cubic millimeter of human cortex. This volumetric density poses very substantial challenges to proteometric analysis at the critical level of the individual synapse. The present work describes new probabilistic image analysis methods suitable for single-synapse analysis of synapse populations in both animal and human brains, in health and disorder.
keywords: medicine and health sciences
  diagnostic radiology
  neural networks
  nervous system
  applied mathematics
  vertebrates
  electrophysiology
  light microscopy
  neuroscience
  animals
  mammals
  simulation and modeling
  algorithms
  mathematics
  microscopy
  research and analysis methods
  computer and information sciences
  imaging techniques
  fluorescence microscopy
  tomography
  image analysis
  radiology and imaging
  diagnostic medicine
  anatomy
  synapses
  cats
  physiology
  biology and life sciences
  physical sciences
  amniotes
  neurophysiology
  organisms
title: Probabilistic fluorescence-based synapse detection