This database is associated with user-friendly tools to facilitate searching, retrieving and downloading of disease-related information.There is an urgent need for a unified resource that integrates trans-disciplinary annotations of emerging and reemerging animal infectious and zoonotic diseases. Such data integration will provide wonderful opportunity for epidemiologists, researchers and health policy makers to make data-driven decisions designed to improve animal health. Integrating emerging and reemerging animal infectious and zoonotic disease data from a large variety of sources into a unified open-access resource provides more plausible arguments to achieve better understanding of infectious and zoonotic diseases. We have developed a model for interlinking annotations of these diseases. These diseases are of particular interest because of the threats they pose to animal health, human health and global health security. We demonstrated the application of this model using brucellosis, an infectious and zoonotic disease. Preliminary annotations were deposited into VetBioBase database (http://vetbiobase.igbb.msstate.eduDatabase URL:  AnoAt the molecular level, various infectious agents have different genetic makeups that may cause similar or different signs and symptoms. The molecular diagnosis of infectious diseases using nucleic acid-based technologies such as the polymerase chain reaction, ligase chain reaction, transcription-mediated amplification and nucleic acid sequence-based amplification have become tools in understanding key molecular factors related to ERAIZD causal agents. These techniques provide highly accurate diagnosis of infections caused by bacteria, viruses, fungi, parasites and others . For exaEmerging and reemerging diseases are caused by diverse range of biological agents that exit their reservoirs, enter susceptible hosts through different routes and cause tissue damage. Provision of a resource that links findings from epidemiological and basic research investigations could help answer fundamental questions regarding factors responsible for disease development and so forth. We have developed a model for annotating multidimensional diverse ERAIZD data from a large variety of sources and integrating it into an open-access unified resource we call ERAIZDA . The model is referred to as ‘ERAIZDA model’ throughout the article. We envision that availability and utility of an ERAIZDA resource will (i) promote global data sharing beyond usual boundaries which could lead to improved interactions in multi- and interdisciplinary research teams; (ii) furnish the animal/human health and veterinary research communities with integrated disease information for developing effective joint policies and guidelines for controlling animal infectious and zoonotic diseases; (iii) lead to better planning of coordinated research strategies including setting up research priorities, developing testable data-driven hypotheses and identification of effective and efficient integrative interventions and (iv) lead to establishing database and data sharing standards used by multi- and interdisciplinary communities for collaborative data sharing, solving much of the current problems in data curation and database application programming interface and web services among related data sources. We strategically selected brucellosis to demonstrate implementation of the ERAIZDA model. Globally, brucellosis is one of the most significant zoonotic diseases  with pubThe ERAIZDA model adopts an annotation approach that takes into consideration all sources of information for an infectious disease that occurs at the animal–human–pathogen–ecosystem interface. Identification of information sources that collectively provide animal infectious, and zoonotic diseases is a first step when implementing this model. Five main sources of disease information were identified and designated as PADER , where PComprehensive annotation of animal infectious and zoonotic diseases using the PADER approach is a complex process that involves coordinated and dedicated effort of diverse expertise and skills to integrate multidisciplinary disease information into a unified resource. The complete ERAIZDA model herein described  demonstrSupplementary File S1). We acknowledge that debate may exist around the description of these parameters which could lead to better and standardized disease annotation terms.The ERAIZDA model is designed for comprehensive annotation of animal infectious and zoonotic diseases from diverse sources (PADER) to generate multidisciplinary disease-related information. We have defined parameters to be used as reference to guide the annotation process in each PADER source. These parameters are grouped into three broad categories representing (i) diseases, (ii) pathogens  and (iii) molecular information, specifically disease biomarkers. The model adopts different terminologies for describing infection, frequency and typology of an infectious or zoonotic disease . A detaiBrucellosis was chosen as a classical example of a significant endemic zoonotic disease to demonstrate the implementation of the ERAIZDA model. The annotations of brucellosis described herein are not by any means exhaustive but serve as model for annotating animal infectious and zoonotic diseases from diverse sources.Supplementary File S1) for guiding comprehensive annotation of animal infectious and zoonotic diseases using this model. In order to provide structured annotations the parameters were organized in an Excel spreadsheet where rows represented diseases and columns represented parameters.Preliminary annotation of selected PADER sources enabled us to establish important parameters (Supplementary File S2).The process of annotation started by searching important keywords [brucellosis OR (brucellosis) OR (brucellosis zoonotic disease) OR  OR (brucellosis in human)] in Google as First-Pass Analysis to get a clear picture about the expected sources of information and initial knowledge of brucellosis. The search returned 180 000 google hits. Clustering of the top 100 hits using PADER codes identified the most important sources of brucellosis information (Supplementary File S1). In the process of annotating brucellosis we found that some parameters may not apply or be relevant to some diseases and data may not be available for some parameters. In such cases, the corresponding fields are represented with triple hyphen (—) until such data become available in the subsequent updates.From the First-Pass Analysis, we were able to generate preliminary brucellosis information that enabled us establish custom terms  for describing disease parameters (Supplementary File S3). In the past 15 years, the number of brucellosis articles has been increasing progressively. A dedicated effort to annotate these articles can yield a very significant information that may provide insight of new ways for managing the brucellosis.The primary quantifiable sources of reliable brucellosis information, including epidemiology of the disease and molecular characterization of the causal agents, are a combination of articles published in peer-reviewed journals and controlled databases represented in the NCBI. The NCBI PubMed database is the most famous gateway for browsing information published in scientific journals. For example, searching the PubMed database (as of 5 May 2015) using brucellosis-related keywords [(brucellosis OR (brucellosis) OR (brucellosis zoonotic disease) OR  OR (brucellosis in human) OR  OR (Mediterranean fever) OR (undulant fever)] retrieved 16 845 articles, including nine articles  that werBrucella melitensis was the first brucellosis causal agent to be characterized. Sir David Bruce isolated Micrococcus melitensis (now B.melitensis) from spleen of a British soldier who died from Malta fever (brucellosis) in Malta in 1886 .Preliminary annotation of few selected articles indicated that  in 1886 . Subsequnopinata  (SupplemB.abortus is the most characterized species as evidenced by the number of subspecies represented in the NCBI Taxonomy Databases and has more open-access PubMed articles (Supplementary File S5).Currently, over 500 subspecies have been classified under these species and documented in the NCBI Taxonomy Database . B.aborBrucella species facilitated identification of species-specific molecular records of the causal agents of brucellosis (Supplementary File S6). These molecular data, especially the nucleotide (transcripts) and amino acid (protein) sequences, are very important in identifying unique features of indistinguishable causal agents. The ERAIZDA model includes a link for accessing most current molecular data for each annotated causal agents indexed in the Entrez databases.Entrez is the NCBI's primary text search and retrieval system that integrates diverse databases such as PubMed and Taxonomy databases with molecular databases such as DNA and protein sequences, genes, genomes, single-nucleotide polymorphs and gene expression data . TraditiSupplementary File S7. This data can also be retrieved from the VetBioBase database  as a reference to guide the disease annotation process from diverse sources referred (in this article) as PADER. Each parameter describes significant information related to a particular infectious and/or zoonotic disease. The model encourages data integration into an open-access unified resource for use by the animal/human and veterinary research communities to accelerate integrative translational interventions aimed at achieving better understanding of infectious and zoonotic diseases. Using brucellosis as an example, we demonstrated that multidisciplinary disease-related information from epidemiological investigations, clinical studies, case reports and basic science functional genomics investigations can be linked together and made available through a single resource. Preliminary annotations of brucellosis were used as a foundation to create a unified resource known as ERAIZDA. Our future focus is to use the ERAIZDA model to generate annotations of all animal infectious and zoonotic diseases of national and international priority, starting with the notable diseases listed by the World Organization for Animal Health (OIE).We present the ERAIZDA model, a comprehensive model for annotating animal infectious and zoonotic diseases. This model uses over 50 pre-defined parameters (Supplementary File S1) as reference is the most feasible way to ensure that every piece of available information is represented. It is most likely that a single source of disease information would not be able to provide all the parameters . This is why PADER becomes such an important approach. For example, when annotating molecular biomarkers using the ERAIZDA model, we recommend including supporting evidence. The best evidence is to link the biomarker with a reference such as PubMed articles and text that will enable users to have confidence in the annotated biomarker. For instance, sequences with amino acid replacement may signify useful molecular biomarkers for detecting mutations that could also be basis for detecting antimicrobial resistance and virulence determinants. Adding reference to the sequence information is particularly important, especially if the annotation involves cross referencing between related diseases. Here is an example of cross-species annotation from two articles; PubMed ID 21151656  and PubMed ID 17021120 : ‘Mycobacterium bovis, a bacterium that causes bovine tuberculosis is naturally resistant to pyrazinamide  that can be used by ontology developers to improve the existing ontologies such as pathogen Transmission Ontology and Symptom Ontology or designing new ontologies. The model also integrates some aspects of host-pathogen interactome features related to the pathogen and hosts at molecular level. For example, we understand that virulence of a pathogen is one of possible effects of host-microbe interaction. In that case, a microbial virulence could depend on host factors to be effective. This is why the model links possible molecular biomarkers including virulence factors with pathogen characteristics to ascertain the effects of interaction on, e.g., the microbial virulence or pathogenicity in relation to effects induced by host genes.Although ERAIZDA model intends to link disease names with Disease Ontology terms, we are aware that there are other ontologies that are relevant to the ERAIZDA model. Since data generated using the ERAIZDA model is intended to benefit not only the research community but also a wide range of animal and human health-based communities, we have been very careful not to link ontological terminologies that are not familiar to the audience. Indeed, the model is expected to provide very specific quantitative data guided by pre-defined parameters (as described in Supplementary File S1) to guide the annotation process in each PADER source. Having a standard annotation procedure will also facilitate data integration and updates.It should be emphasized that the integrated data by itself is not useful, unless it is freely accessible by the intended communities for further interventions. User-friendly tools are provided to offer such access to the interested communities. Generally, the ERAIZDA model encourages multidisciplinary collaboration of all key players involved in the disease management process to share research and non-research data without their usual boundaries. It is anticipated that the ERAIZDA data will continue to grow progressively and sustainably as more animal infectious disease annotations are added, and the research community will utilize and benefit from this data. Although curation is one of the most likely challenging aspects of maintaining a sustainable ERAIZDA database, having a uniform annotation procedure can lessen the challenge. To maintain accuracy and uniformity of annotations, curators are encouraged to use special forms containing the pre-defined parameters (Annotation of animal infections and zoonotic diseases from diverse sources is a central initiative to reveal important information for new interventions. Most importantly, integrating the disease annotations into an open-access unified resource and provision of user-friendly tools for searching, retrieving and downloading specific information could save users considerable time and effort. We expect to increase the depth and breadth of the ERAIZDA dataset by continuing to annotate additional animal infectious diseases of national and international priority. We believe that since zoonoses can infect both animals and humans, both the medical, public health, basic research and veterinary communities will definitely benefit from this unified resource. We encourage interested communities to use the ERAIZDA data to develop additional models needed to understand the mechanism and pathology of animal infectious and zoonotic diseases and translation of functional genomics of infectious diseases into biological value.Supplementary data are available at Database Online.


A convergence of high-throughput sequencing and computational power is transforming biology into information science. Despite these technological advances, converting bits and bytes of sequence information into meaningful insights remains a challenging enterprise. Biological systems operate on multiple hierarchical levels from genomes to biomes. Holistic understanding of biological systems requires agile software tools that permit comparative analyses across multiple information levels  to identify emergent properties, diagnose system states, or predict responses to environmental change.Here we adopt the MetaPathways annotation and analysis pipeline and Pathway Tools to construct environmental pathway/genome databases (ePGDBs) that describe microbial community metabolism using MetaCyc, a highly curated database of metabolic pathways and components covering all domains of life. We evaluate Pathway Tools’ performance on three datasets with different complexity and coding potential, including simulated metagenomes, a symbiotic system, and the Hawaii Ocean Time-series. We define accuracy and sensitivity relationships between read length, coverage and pathway recovery and evaluate the impact of taxonomic pruning on ePGDB construction and interpretation. Resulting ePGDBs provide interactive metabolic maps, predict emergent metabolic pathways associated with biosynthesis and energy production and differentiate between genomic potential and phenotypic expression across defined environmental gradients.This multi-tiered analysis provides the user community with specific operating guidelines, performance metrics and prediction hazards for more reliable ePGDB construction and interpretation. Moreover, it demonstrates the power of Pathway Tools in predicting metabolic interactions in natural and engineered ecosystems.The online version of this article (doi:10.1186/1471-2164-15-619) contains supplementary material, which is available to authorized users. Community interactions between uncultivated microorganisms give rise to dynamic metabolic networks integral to ecosystem function and global scale biogeochemical cycles . MetagenFunctional genes operate within the structure of metabolic pathways and reactions that define metabolic networks. Despite this fact, few metagenomic studies use pathway-centric approaches to predict microbial community interaction networks based on known biochemical rules. Recently, algorithms for pathway prediction and metabolic flux have been developed for environmental sequence information including the Human Microbiome Project Unified Metabolic Analysis Network (HUMAnN) and Predicted Relative Metabolic Turnover (PRMT). HUMAnN uses an integer optimization algorithm that conservatively computes a parsimonious minimum set of reactions along KEGG pathways based on pathway presence, absence or completion , 7. PRMTOne alternative to HUMAnN and PRMT is Pathway Tools, a production-quality software environment supporting metabolic inference and flux balance analysis based on the MetaCyc database of metabolic pathways and enzymes representing all domains of life –13. UnliHere we provide essential guidelines for generating and interpreting ePGDBs inspired by the multi-tiered structure of BioCyc  Figure . We begim) showed that the performance of pathway recovery based on multiple metrics  increased with sequence coverage and sample diversity nearing an asymptote at higher coverage  construction commences with the MetaPathways automated annotation pipeline using environmental sequence information as input . Resulting annotations are used by the PathoLogic algorithm implemented in Pathway Tools to predict metabolic pathways based on multiple criteria including proportion of pathways found, pathway specific enzymatic reactions, and purported taxon-specific pathway distributions. PathoLogic is known to perform well when compared to machine learning methods using the genomes of cellular organisms as input . We preve Figure a. This sEsherichia coli K12 substr. MG1655 genome (basis of the EcoCyc database), Sim1 and Sim2, and a subsampled 25 m metagenome from HOT [m coverage. Consistent with previous observations for Sim1 and Sim2, all experiments showed that pathway recovery percentage and performance sensitivity increased with sequence coverage and sample diversity nearing an asymptote at higher coverage  and between HOT, Sim1/2 and E. coli for short read (~160 bp) datasets. Additional performance metrics can be found in Additional file To better constrain pathway recovery and performance in relation to ePGDB construction we compared results of MetaSim experiments using the from HOT   algorithm. The WTD algorithm measures the taxonomic distance between predicted coding DNA sequences (CDS), e.g., BLAST hits from the RefSeq database, and expected taxonomic range for each predicted pathway using the NCBI Taxonomy Database. The NCBI Taxonomy Database is hierarchically structured, and a path between the lowest common ancestor (LCA) of observed CDS annotations and each member of the expected taxonomic range in a pathway can be charted , where eWhen the WTD algorithm was applied to HOT datasets, the taxonomic distribution of predicted pathways generally aligned with the expected taxonomic ranges of MetaCyc Pathways , bacterial endosymbionts of the mealybug Planococcus citri have been previously described by McCutcheon and colleagues to distribute biosynthetic pathways for essential amino acids in a process known as “inter-pathway complementarity.” Environmental PGDB construction using the combined Moranella and Tremblaya genomes recovered 43 out of 44 reactions and all 9 distributed amino acid biosynthesis pathways previously reported  and metatranscriptome (RNA) datasets from 25, 75, 110 m (sunlit or euphotic) and 500 m (dark) ocean depth intervals from HOT . A totalMore than 600 pathways were shared in common between the sunlit and dark ocean based on combined DNA and RNA datasets consistent with a conserved metabolic core Figure b. A totaA total of 30 pathways were identified exclusively in RNA datasets including 11 pathway variants Figure d. With rAn evaluation of the 72 most abundant pathways recovered from the combined datasets indicated that 53 were both present and expressed at 25, 75, 110, and 500 m depth intervals. Moreover, several of the most abundant pathways including ammonium transport, Rubisco shunt, NADH to cytochrome electron transfer, pyruvate fermentation, denitrification, Calvin-Benson-Bassham cycle, cysteine biosynthesis I and arginine biosynthesis III exhibited depth-dependent trends in gene expression , chlorophyll a, and retinol biosynthesis , 20  numbers representing classes with non-specific substrate activity. For example EC 3.2.1.21 represents a non-specific enzyme class for beta-D-glucosides, allowing for spurious prediction of specific carbohydrate degradation pathways. Moreover, PathoLogic has a preference for EC numbers over product descriptions that can further exacerbate false discovery associated with non-specific enzyme classes. Hazards manifesting themselves within pathway variants sharing a number of common or reversible reaction steps have previously been described by Caspi and colleagues in the context of PGDB construction for cellular organisms . For exa2 per kg) [2 per kg [The identification of dissimilatory nitrate reduction (denitrification), intra-aerobic nitrite reduction and ammonia oxidation in the combined 500 m HOT DNA and RNA datasets provides a real world example of hazard navigation. Denitrification is a distributed form of energy metabolism resulting in the production of nitrogen gas in oxygen-deficient waters (<20 μM O per kg) , 30. The per kg) –34    [Environmental PGDBs were constructed from public datsets using MetaPathways (thways/)  with defthways/) , COG [38thways/)  (versionthways/)  (DownloaE. coli K12 MG1655 complete nucleotide genome (GenBank: NC_000913) at a series of fractional levels  of the total combined length of starting component genomes (Gm). Pathways were predicted using the MetaPathways pipeline, as described above, against each of the resulting sequence sets  were pathways not predicted in the test set or gold standard. False positives (FP) were pathways found in the test set but not in the gold standard. Finally, false negatives (FN) were pathways found in the gold standard but not in the test set. Multiple summary statistics for the resulting confusion tables , Specificity, Precision, Accuracy, F-measure, and Matthew’s Correlation Coefficient (MCC)) were calculated. A summary of these performance statistics is provided in the supplement  and pathways were predicted as described above. Similarly, a classification performance analysis was performed with the set of 864 pathways predicted from the complete 454 run representing the gold standard  sample, had their pathways predicted with the above method, but with taxonomic pruning enabled using the taxonomic lineage parameter set to “Unclassified sequences”. The number of predicted pathways were tabulated and compared with the pathways previously predicted with taxonomic pruning disabled. As simple set analysis showed that within a sample the pruned pathways were a strict subset of the “no-pruning” ones, and the reduction in pathways was calculated  xexp ∈ TRMetaCyc)((p), where TRMetaCyc)((p) is the set of taxonomic range(s) for a given pathway p on the NCBI Taxonomy Database hierarchy.For each predicted pathway in the HOT dataset, a weighted taxonomic distance (WTD) distance was calculated using the WTD algorithm , asThis WTD algorithm takes as input ea,b is an edge between nodes a and b in the path and d(a) is the depth of node a. If xexp descends from the expected taxonomic range xobs, then the WTD is assigned a positive value and WTD for paths descending outside this range are assigned a negative value. After calculating the WTDs for all pairs xexp, xobs, the WTD algorithm first attempts to return the minimum non-negative distance e.g., WTD corresponding to the closest xexp where xobs is a descendant of xexp, and returns the maximum negative score e.g., closest to zero if all observed and expected taxonomies diverge. For each dataset, predicted pathways were assigned to a “Disagreement Class” based on the following criteria: (i) pathways with positive WTD were given the “None” class, (ii) pathways with distances greater than the median of negative WTDs were given the “Low” class, (iii) pathways within the 2nd quartile were given the “Medium” class, and (iv) pathways in the lower quartile were given the “High” disagreement class , Bacillus subtilis subtilis 168 (GenBank: AL009126.3), Caulobacter crescentus NA1000 (GenBank: CP001340.1), and Helicobacter pylori 26695 (GenBank: AE000511.1), abbreviated by the first character of their proper names, A, B, C, and H, respectively. The six pair-wise and four original genomes were analyzed as described above for E. coli  were downloaded from NCBI and analyzed as described above for E. coli. Resulting PGDBs for individual and combined genomes were manually inspected for amino acid biosynthetic pathways described in McCutcheon and Dohlen [Similarly, the d Dohlen  (Additiohttp://www.r-project.org), and visualized using the ggplot graphical package (http://ggplot2.org) and d3.js graphical library (http://d3js.org/).Unassembled metagenomic and transcriptomic pyrosequences from the Hawaii Ocean Time-series  were obtained from the NCBI Sequence Read Archive  and run through the MetaPathways pipeline using default settings . The Hawaii Ocean Time series datasets can be downloaded from the NCBI Sequence Read Archive under accession numbers SRX007372, SRX007369, SRX007370, SRX007371, SRX016893, SRX016897, SRX156384, SRX156385.The symbiotic Supplementary notes, figures, and tables.(PDF 12 MB)Additional file 1:Summary of candidate pathways that are potentially distributed by set-difference analysis.(XLS 30 KB)Additional file 2:Summary table of 1033 pre-QC predicted pathways and CDS counts for the Hawaii Ocean Time-series samples.(XLS 217 KB)Additional file 3:Summary table of the 840 post-QC predicted pathways and CDS counts for the Hawaii Ocean Time-series samples.(XLS 183 KB)Additional file 4:Summary table of the 840 post-QC predicted pathways and normalized CDS counts for the Hawaii Ocean Time-series samples with taxonomic disagreement class highlighted.(XLS 304 KB)Additional file 5:Summary table of the 840 post-QC predicted pathways and normalized CDS counts for the Hawaii Ocean Time-series samples with observed LCA taxonomies, expected taxonomic ranges, calculated weighted taxonomic distance, and taxonomic disagreement class.(XLS 806 KB)Additional file 6:Summary table of normalized CDS counts for the 593 DNA fraction pathways of samples from the Hawaii Ocean Time-series.(XLS 150 KB)Additional file 7:Summary table of normalized CDS counts for the 495 pathways common to DNA and RNA samples from the Hawaii Ocean Time-series.(XLS 168 KB)Additional file 8:


Enrichment analysis is a popular approach to identify pathways or sets of genes which are significantly enriched in the context of differentially expressed genes. The traditional gene set enrichment approach considers a pathway as a simple gene list disregarding any knowledge of gene or protein interactions. In contrast, the new group of so called pathway topology-based methods integrates the topological structure of a pathway into the analysis.We comparatively investigated gene set and pathway topology-based enrichment approaches, considering three gene set and four topological methods. These methods were compared in two extensive simulation studies and on a benchmark of 36 real datasets, providing the same pathway input data for all methods. In the benchmark data analysis both types of methods showed a comparable ability to detect enriched pathways. The first simulation study was conducted with KEGG pathways, which showed considerable gene overlaps between each other. In this study with original KEGG pathways, none of the topology-based methods outperformed the gene set approach. Therefore, a second simulation study was performed on non-overlapping pathways created by unique gene IDs. Here, methods accounting for pathway topology reached higher accuracy than the gene set methods, however their sensitivity was lower.We conducted one of the first comprehensive comparative works on evaluating gene set against pathway topology-based enrichment methods. The topological methods showed better performance in the simulation scenarios with non-overlapping pathways, however, they were not conclusively better in the other scenarios. This suggests that simple gene set approach might be sufficient to detect an enriched pathway under realistic circumstances. Nevertheless, more extensive studies and further benchmark data are needed to systematically evaluate these methods and to assess what gain and cost pathway topology information introduces into enrichment analysis. Both types of methods for enrichment analysis require further improvements in order to deal with the problem of pathway overlaps.The online version of this article (doi:10.1186/s12859-015-0751-5) contains supplementary material, which is available to authorized users. Analysis of gene expression experiments typically yields long lists of differentially expressed genes (DEGs) . To easeWe were interested to evaluate different methods for analysing gene expression data in the context of signalling pathways, with a special focus on comparing methods with different pathway representation strategies. There are different approaches how to handle a pathway in enrichment analysis. The traditional approach considers a pathway as a simple gene list omitting any knowledge of the gene and protein interactions. Methods belonging to this category are commonly called gene set (GS) analysis methods. Another way of integrating a pathway into enrichment analysis takes into account its graphical structure and/or topological measures. One of the first pathway topology-based (PT-based) methods was impact analysis proposed by Draghici et al. . Since tp-value calculation and (ii) the group of self-contained methods which is associated with subject sampling [p-value calculation assume independence of genes, which is known not to be true. On the other hand, self-contained methods were criticized for being too powerful and thus yielding too many significant gene sets. Also the number of replicates in the experiments is often too low for subject sampling [Enrichment methods can be divided into two groups based on the null hypothesis: (i) competitive methods which are naturally linked with gene sampling for sampling . The bassampling .Another classification of enrichment methods separates them into over-representation analysis (ORA) and functional class scoring (FCS) approaches . ORA is From the user’s point of view there are several features of the enrichment methods to be considered besides classification of the methods according to methodological and statistical framework. For instance, the implementation and availability of the software can determine how popular a method becomes. Furthermore, several pathway databases such as Kyoto Encyclopedia of Genes and Genomes (KEGG) , ReactomSeveral studies comparing different GS analysis methods were already published –20, as wTherefore, the aim of our study is to evaluate GS versus PT-based approaches using simulated gene expression data as well as benchmark datasets. For comparability of the methods we focused mainly on competitive ones, including three GS tests and four PT-based methods implemented in the R statistical computing environment , Kolmogorov-Smirnov (KS) and Fisher’s exact (FE) tests. These tests were implemented in various flavours and extensions in multiple tools. For instance, the popular GSEA  is basedwilcox.test, ks.test and fisher.test to perform WRS, KS and FE analysis, respectively within the R statistical language and environment [p-values for differential expression [p-values into ranks. In our setting, WRS tests whether the distribution of ranks of the genes in a set is shifted to the left from a distribution of ranks of the genes in corresponding complement to the gene set. KS test compares the ranks of genes in a set to a uniform distribution.We utilized the R-functions ironment , versionpression . WRS andp-values are directly obtained using hypergeometric distribution.The FE test is based on ORA. The cut-off in the list of differentially expressed genes (DEGs) was defined as false discovery rate below 0.05 (FDR <0.05). FE is testing independence of rows and columns in 2x2 contingency table (while the margins are fixed) and SPIA, CePa and Pathnet. Two variants of the CePa method were implemented in the CePa R-package, the so-called CePa ORA and CePa GSA, which we consider as two distinct methods.In this study we evaluated PT-based algorithms from three R-packages: p-values [p-value originates from a simple ORA, which assumes that the number of DEGs in a given pathway follows hypergeometric distribution. The second, so-called perturbation p-value is computed in several steps and incorporates pathway topology information. To obtain the perturbation p-value, first, for each gene in a pathway a perturbation factor is computed. It captures the logarithm of the fold-change (logFC) of a gene and the sum of perturbation factors of the direct upstream genes of a gene normalized by the number of all downstream genes. Each term of the sum is weighed by type of interaction between the genes: 1 and −1 for activation and inhibition, respectively. This results in upstream gene influencing perturbation factors of many downstream genes. In a second step the perturbation accumulation at the level of each pathway gene is calculated. It is defined as the difference between the gene perturbation factor and its observed logFC. Finally, the total pathway accumulated perturbation is computed as a sum of the accumulated perturbations of pathway’s genes. Significance is assessed in a bootstrap procedure, resulting in a perturbation p-value. The two p-values are then combined into a global p-value for each pathway using Fisher’s product test.Signaling pathway impact analysis (SPIA) is an enrichment method combining two types of evidence, which are represented by two p-values . The firp-value the null distribution of the pathway score is generated by permuting the DEGs on a given pathway topology. The centrality measure options are in- and out-degree, betweenness and in- and out-largest reach. Degree centrality measures the number of nodes directly connected to the given node, betweenness refers to the number of information streams passing through a given node and largest reach quantifies how far a node can send or receive information. Along 5 centrality options CePa also calculates an equal weight model where all weights are set up to 1. Therefore, six p-values for each pathway are calculated. We followed the authors’ recommendation to try every centrality option in the search for significant pathways. Hence, the smallest out of 6 p-values was selected to represent pathway significance.CePa  is a weiPathway data section).Furthermore, the CePa method proposed a node-based instead of a gene-based ID mapping approach. That means, if any member of a complex or a group of genes residing in one node is differentially expressed then the node is considered as differentially expressed. Also nodes representing non-gene components of a pathway such as microRNA and small molecules are retained in the pathway topology. However, these last two features of the CePa algorithm were suppressed using our pathway input data  gene symbol IDs on the graph nodes several editing steps had to be taken Fig. . First, Depending on the method, suitable pathway input has to be provided implying graphs transformations . For GS To create the SPIA pathway input, a graph of each pathway was transformed into a list of 2 adjacency matrices for activation and inhibition processes. Accordingly, a vector of weights β was set to β = {1, −1} to reflect activation and inhibition. For CePa a pathway catalogue was constructed comprising a list of pathways with the interaction IDs, and a table with columns for the interaction IDs and for the corresponding input and output interaction components. The pathway catalogue also included a mapping table. However, in our pathway data a single gene ID was always mapped onto one node. Input for PathNet consisted of an adjacency matrix of a pooled pathway, which was created by merging all database pathways, an interaction table with pathway IDs and a mapping table. The mapping table was required as PathNet algorithm transforms gene IDs into numeric objects and artificial integer IDs had to be generated.Simulations chapter). In this study pathway graph nodes were relabelled with new synthetic IDs to construct non-overlapping pathways with unique components, while the topology of the pathway remained intact.All pathway data inputs were regenerated for the second simulation study -dimensional covariance matrix employed for the multivariate normal distribution, we introduced a correlation of 0.8 between genes in the same pathway and 0.05 otherwise .When generating expression data for the original pathway simulation study the dimension of mean vector was set to p = 7697. This number represents the total number of nodes in our KEGG pathway graphs. The covariance matrix of p × p was constructed in a similar manner as in the first study, but as pathways do not overlap the matrix was positive-definite by default.To simulate expression data in the second study the mean vector dimension was In each study we performed 5 simulation types. Within a single simulation type one of the 4 parameters was explored on several levels, while the other 3 parameters were fixed at a certain level see Fig. . This remean), pathway size (size), number of pathway (N) and detection call (DC). The three levels of mean = {+/−1, +/−2, +/−6} reflect expression changes of the affected genes between control and treatment groups. The size of the deregulated pathways size = {small, medium, big} is given by the number of genes in a pathway. The number of deregulated pathways N = {12, 23, 70} represents different proportions out of all 116 pathways in the KEGG database. Four levels of the detection call DC = {10 %, 30 %, 50 %, 70 %} give the percentage of affected genes in a deregulated pathway. The levels of these parameters were adjusted to reflect ranges from small to extreme effects that can be found in real data. As the default – fixed parameter level we chose moderate levels or the most general setting, depending on a particular parameter. For a more detailed description of the variable parameters see Additional file The variable parameters are mean vector  [Communities in a given pathway were detected by an algorithm for greedy optimization of modularity implemented in the unction) . It aimsTo select affected genes in the betweenness deregulation design we considered the top highest scored betweenness nodes Fig. . BetweenIn the neighbourhood deregulation approach one node of a pathway is chosen and then all nodes within certain distance create the neighbourhood Fig. . In the Within a given parameter configuration for a run, a certain number of pathways (depending on the setting) was randomly chosen to be deregulated by assigning affected genes. Then new expression data for this particular run were drawn from the multivariate normal distribution.limma package were fitted to identify differential genes with theirs logFC, p-values and FDRs . Those were further supplied to the enrichment methods according to the specific requirements of each method.The expression matrix was directly supplied to the CePa ORA and CePa GSA algorithms. For the rest of the methods linear models from the All 7 methods were evaluated within each simulation run on the basis of the same expression as well as pathway input data.rBiopaxParser and transformed them into interaction graphs. These graphs comprised 7697 nodes in total, representing 3173 unique genes according to gene symbol IDs. On average a single gene was present in 2.4 pathways indicating the extent of pathway overlaps. The pathways comprised from 5 up to 380 genes with a median pathway size of 55.5 genes . The accuracy is then given as the proposition of true-positive and true-negative pathways detected by a method out of all 116 pathways. Within one study, for each of the 17 parameter configurations we computed the median accuracy, sensitivity and specificity of 1000 simulation runs. As in many scenarios 12 pathways was the maximum number of true-positives, the accuracy measure was mainly driven by true-negatives. Hence, it reflects the specificity measure to a bigger extent. Therefore, we focus mainly on accuracy and sensitivity as they provide more complementary view on the results and the specificity results are reported in Additional file mean = +/−1) sensitivity of ranking GS methods – 0.67 for WRS and 0.58 for KS – was the best, followed by PathNet with a sensitivity of 0.5. All methods based on ORA (both GS and PT-based) had 0 sensitivity, however, within the accuracy they reached 0.9. With a higher mean change, i.e. mean = {+/−2, +/−6}, all methods were comparably sensitive (between 0.92 and 1), while WRS reached the best accuracy scores of 0.56 and 0.54.With weak changes of the mean vector  and WRS performed best at this parameter level with a sensitivity of 0.92 and 0.83, respectively. However, within all methods the accuracy was highest for small pathways (ranged between 0.59–0.74) and decreases with larger pathways.N = 70) with WRS and CePa ORA performing the best.When exploring the effect of number of pathways which were deregulated, all methods were most accurate (0.58–0.66) when more than half of the database pathways were deregulated (DC = 10 %) coupled with betweenness topology design PT-based methods and FE test were more sensitive (0.58–0.92) than ranking GS methods (0.33–0.58). However, on 30 % and higher DC levels all methods performed comparably well in the term of sensitivity (0.83–1). At the same low level of detection call (DC = 10 %) coupled with neighbourhood topology design PathNet had the highest sensitivity of 0.58, followed by other PT-methods and FE (0.42–0.5). However, in both DC simulation types the accuracy ranged only between 0.56 and 0.31 for DC = {50 %, 70 %}.At low level of detection call . In detecting small pathways the best sensitivity (0.75) was reached by WRS and PathNet. However, the accuracy was the best for PT-based methods for most of the levels of both size and mean parameters. The sensitivity of all methods except CePa GSA decreased when many pathways were deregulated (N = 70), even to a bigger extent than in study 1. Interestingly, for N = 70 FE and KS accuracy was only 0.4 and 0.44, respectively, while the other methods reached an accuracy between 0.66 and 0.97. Both simulation types with DC parameter coupled with neighbourhood and betweenness topology designs showed similar behaviour patterns. When pathways did not overlap a DC level of 50 % was needed to achieve sensitivity over 0.75, whereas for the study with original pathways all methods reached 0.75 sensitivity at a DC = 30 % level. For the low DC (DC = 10 %) PathNet’s sensitivity was the best. In term of accuracy the PT-methods performed the best.Similarly to the results of study 1, GS methods based on ranking were the most sensitive (0.5) for the low level of Distribution of accuracy, sensitivity and specificity results over all 17 simulation configurations was visualized for each method within the both studies Fig. .Fig. 6OvIn the first simulation study with original overlapping pathways the best accuracy was achieved by WRS and SPIA. This was also reflected in the best specificity of these two methods, while the CePa GSA performed as the least specific method. However, the CePa methods reached the best sensitivity, followed by ranking tests WRS and KS.In the second simulation study with non-overlapping pathways, both overall accuracy and specificity increased, especially for the PT-based methods. Within GS methods there were prominent differences with KS performing the worst. However, KS with WRS showed as the most sensitive methods.p-values of the target pathways in 36 datasets and their ranks in the whole KEGG database, expecting the target pathways to be ranked close to the top [Besides simulated expression data, 36 microarray datasets comprising together 1127 samples were used as a benchmark to compare 7 enrichment methods. Every dataset represents a certain disease and has been linked to a defined target pathway from the KEGG database  enrichment tests with methods, which attempt to incorporate pathway topology information into analysis.Comparing GS methods and pathways topology-based (PT-based) enrichment approaches is challenging in two regards. First, the pathway input data have to be corresponding for all methods to ensure a fair comparison. Therefore, we parsed a BioPAX export of the KEGG database and transformed the pathways into gene sets as well as various interaction graph representations for the different methods. Second, it is difficult to simulate expression data for PT-based methods. The signal in a pathway is mediated via multiple biological mechanisms. Capturing a pathway deregulation in a graph and reflecting this alteration in the synthetic expression data is an intriguing task. To our knowledge there is no coherent work published on this topic. We made one of the first attempts to generate expression data mirroring the topology of the deregulated pathways. However, there are several limitations in our simulation scheme. For instance, multivariate approaches usually require more complex covariance matrices for a fair and unbiased evaluation, therefore, we have not included multivariate self-contained methods.We decided to use 3 approaches to allocate affected genes to a pathway: gene community, betweenness and neighbourhood. However, in a certain setting some methods might be favoured due to their inherent algorithms, e.g. both CePa methods by betweenness and PathNet by neighbourhood designs. Further, it could be questioned how well these network measures reflect the real perturbation signals in a pathway.In the first simulation study we could not identify any PT-based method as outstanding neither for any parameter configuration, nor for the overall accuracy measure. The pathway input for this study was represented by original KEGG pathways, which exhibit considerable gene overlap among each other. When a pathway was called deregulated it received a certain number of genes assigned as affected according to the detection call level. However, a single gene was on average present in 2.4 pathways. This resulted in a number of pathways, which were not considered as deregulated but also contained affected genes. These pathways then showed up as false positives and consequently led to a very low specificity and accuracy of all methods in study 1. However, one could assume that these ‘accidental’ affected genes were allocated randomly in the non-deregulated pathways, whereas in the deregulated pathway they were placed in a topological context. Therefore, it could be expected that the PT-based methods would overcome the problem of these false positives, however, this seems not to be the case.mean = +/− 1). The CePa GSA method exhibited a clearly different behaviour than the others, which was expected from the single self-contained method included in this work. Prominently different performance of this self-contained method could be further enhance by the fact that CePa computes for each pathway five p-values corresponding to different centrality measures and the authors of the method recommend to use the lowest value. Therefore, its overall sensitivity in the first study was higher than all the other competitive methods [N = 70). Similarly, in the diseased benchmark datasets CePa GSA was the best in identifying target pathways but it also identified more than half of all KEGG pathways as significant. Depending on a disease, multiple pathways could be altered. However, it could be questioned whether such a number of pathways is realistic or it reflects lack of specificity of this method.It was already demonstrated that ranking methods are able to detect modest signals of deregulation better than the ORA approach . We furt methods . On the In the second simulation study with non-overlapping pathways the overall specificity markedly increased, mainly for the PT-based methods. This was also reflected in the best accuracy for these four PT-based methods.DC = 70 %, N = 70, size = big). That could stem from its null hypothesis, as it actually tests whether a gene set or its complement is significant [Interestingly, the overall specificity of KS was the lowest among all methods and it was also always decreasing when more genes were affected  and the levels of the pathways with which they are associated. Therefore, interpretability of results obtained within such an approach is limited, even if the gene expression data does approximate the proteome to a certain extent. Furthermore, if a pathway gene set, used as prior knowledge for the enrichment analysis, does not properly reflect the experimental data, we cannot expect that adding topological information into such a gene set would increase the method’s performance.A straightforward solution to address this discrepancy would be to test enrichment of signalling pathways in protein abundance data as they reflect the protein character of the pathways much more closely. Another solution might be to move from testing enrichment of pathways to testing sets of target genes of transcription factors, as already proposed by Naeem et al. . HoweverUsually, new PT-based methods are evaluated on a small number of real datasets and not in all cases their performance is compared to any other PT-based method –11, 24. Enrichment analysis is a useful and popular approach to ease interpretation of high throughput data. We comparatively investigated 3 gene set (GS) and 4 pathway topology-based (PT-based) methods in extensive simulation studies and on real benchmark data.Our results are consistent with some already published findings: As expected, the self-contained method behaved differently than the competitive ones. The ranking methods were more sensitive than over-representation analysis methods when expression changes were moderate. Further, we demonstrated that on original KEGG pathways none of the PT-based methods was clearly outperforming the GS approach, neither in simulations nor in benchmark testing. This changed in the simulation study with non-overlapping pathways, where the PT-based approach exceeded simple GS tests. This suggests that further conceptual work has to be done to deal with the problem of pathway overlaps.From the gene set methods Wilcoxon rank sum test performed comparably as topological methods in most settings, and it is much easier to use and re-implement with different input data than PT-based approaches.Including topology to test for enrichment of a pathway is an interesting approach. It introduces additional information into the analysis and allows attractive visualizations of the pathway graphs. However, it needs to be further explored whether the additional biological insights justify the increased complexity of the PT-based enrichment analyses.The data sets supporting the results of this article are included within the article and its additional files.


This study aimed to develop a pathway to bring together current UK legislation, good clinical practice and appropriate management strategies that could be applied across a range of healthcare settings.The pathway was constructed by a multidisciplinary clinical team based in a busy Memory Assessment Service. A process of successive iteration was used to develop the pathway, with input and refinement provided via survey and small group meetings with individuals from a wide range of regional clinical networks and diverse clinical backgrounds as well as discussion with mobility centres and Forum of Mobility Centres, UK.We present a succinct clinical pathway for patients with dementia, which provides a decision-making framework for how health professionals across a range of disciplines deal with patients with dementia who drive.International Journal of Geriatric Psychiatry published by John Wiley & Sons, Ltd.By integrating the latest guidance from diverse roles within older people's health services and key experts in the field, the resulting pathway reflects up-to-date policy and encompasses differing perspectives and good practice. It is potentially a generalisable pathway that can be easily adaptable for use internationally, by replacing UK legislation for local regulations. A limitation of this pathway is that it does not address the concern of mild cognitive impairment and how this condition relates to driving safety. © 2014 The Authors.  Australia requires the patient to inform the licensing authority of any permanent long-term injury or illness that affects safe driving ability, but again no specific mention of dementia is made , these have failed to satisfactorily address the issue of driving, with these guidelines only briefly mentioning that medicolegal issues including driving need to be addressed but offering no supporting guidance for the clinician.The lack of guidance and ambiguity in how to deal with the issue of driving and the patient with dementia may lead to a clinician's reluctance to tackle the issue. Although key guidelines have been issued for dementia in the UK and Europe . A process of successive iteration, and consensual discussion within the group, developed a draft pathway. Once the initial pathway had been drafted, a survey  of individuals, with 29 respondents, from a wide range of regional networks  and diverse clinical backgrounds (medicine and psychiatry (15), nursing ( 4), psychology (8) and occupational therapy (2)) helped refine the pathway. Participants were asked to provide ratings for the utility of the pathway, how likely they would be to use it in practice and how much it clarified the issue for them, as well as provide suggestions for alterations or specific inclusions. Finally, the pathway was further shaped following discussion with key external stakeholders, that is, North East Drive Mobility (the Accredited Mobility Centre in the North East of England), the UK Forum of Mobility Centres and the Driver and Vehicle Licensing Agency, into its final format.The completed pathway Figure  consistshttp://research.ncl.ac.uk/driving_and_dementia).It should be recognised, however, that this pathway is not presented as a stand-alone item, but it is the central aspect of an overall support package that was developed by the aforementioned team. The full ‘driving pack’ contents are presented and described in Table We present a clinical pathway for patients with dementia, which was developed following a robust process with input from key experts in the field. The overall aim was to address the uncertainty that exists on how patients with a dementia diagnosis, who wish to continue to drive, should be managed. By this process, we have drawn together current UK legislation, good clinical practice and appropriate management aspects into a simple care pathway that could be rapidly and easily applied across a range of healthcare settings and also be utilised by individuals with limited experience of managing this issue.A range of clinical pathways are available through the National Institute for Health and Care Excellence and bring together clinical guidelines, interventional procedures, public health guidance and quality standards into a logical flow diagram for users NICE, . PathwayA strength of this pathway is the multidisciplinary approach utilised in its development. The developers were able to integrate the latest guidance from diverse roles within older people's service and key experts in the field, resulting in a pathway that reflects up-to-date policy and encompasses differing perspectives and good practice. This procedure enhances the efficacy of the pathway as a general tool that can be utilised across all disciplines within service.et al., et al., The pathway also provides the framework for a uniform approach across services, as the pathway is not a stand-alone tool. It forms part of a driving pack, that contains further information and scaffolds the pathway, for example, information on local mobility centres; and alternative forms of transport, and is supported by relevant patient information on pre and post driving cessation advice. It is hoped that this can facilitate discussion with patients and families and enhance a collaborative approach O'Neill, , betweenThis pathway was developed in the UK, taking into account UK practice and legislation; however, it is potentially a generalised pathway that can be easily adaptable for use worldwide, by substituting UK-based (DVLA) legislation for local regulations. The multidisciplinary method utilised in development enables it to be integrated into differing service approaches, internationally. This may also facilitate in addressing the lack of guidance and clarification that emerged, when investigating European, American, Australian and New Zealand policy.Particular limitations emerged during the development of this pathway. The first is that in the area of driving and dementia, there is a lack of coherency in the field in terms of theory and empirical evidence, and thus, our pathway is informed on the basis of expert clinical judgement and consensus opinion. And although we recognise this is not as strong as empirical evidence, it was ratified by a range of respondents as clinically useful. However, where possible, the advice given is based upon previous research  (Budson and Solomon, None declared.Individuals with a diagnosis of dementia are at increased risk when driving.Currently, no consensus guidance exists.The pathway is designed to address this disparity.This paper reports a clinical service development and clinical opinion document. In this context, it is not a formal research study and therefore did not require specific institutional or ethical approval for its completion.


Elucidating the complexities of cell signaling pathways is of immense importance to gain understanding about various biological phenomenon, such as dynamics of gene/protein expression regulation, cell fate determination, embryogenesis and disease progression. The successful completion of human genome project has also helped experimental and theoretical biologists to analyze various important pathways. To advance this study, during the past two decades, systematic collections of pathway data from experimental studies have been compiled and distributed freely by several databases, which also integrate various computational tools for further analysis. Despite significant advancements, there exist several drawbacks and challenges, such as pathway data heterogeneity, annotation, regular update and automated image reconstructions, which motivated us to perform a thorough review on popular and actively functioning 24 cell signaling databases. Based on two major characteristics, pathway information and technical details, freely accessible data from commercial and academic databases are examined to understand their evolution and enrichment. This review not only helps to identify some novel and useful features, which are not yet included in any of the databases but also highlights their current limitations and subsequently propose the reasonable solutions for future database development, which could be useful to the whole scientific community. Biochemical pathways, the molecular mechanism through which the cellular components are governed in the extra and intra cellular reaction networks, are involved in various physiological and cellular developmental processes and sensitive to the external or internal fluctuations of the cells . These pHowever, initially the construction and analysis of such large and complex reaction networks were not so easy, as the unavailability of advanced tools and techniques for the functional annotations of various unknown or newly identified genes/proteins in the pathway diagram were the major problems to the pathway curators. Fortunately, the advancements in cellular and molecular biology experiments, high-throughput genomics or proteomics studies and the successful completion of human genome project flourished this field by generating plethora of genomics and proteomics data . Moreoveinsilico model developers for mathematically interpreting the emerging properties of different cellular networks upon its exposure in various external cues  the database should solely or partially provide the human cell signaling-related data and (ii) the HTTP links of the databases should be in active or working state and can be readily accessible to the users without any accessing charge. At the time of communicating this review article, there were altogether 24 such active, open source, well known and broadly used human cell signaling databases available, which fulfill these two criteria. In the Supplementary Table S1, the name and the HTTP links of these databases are provided for the interested readers. It should be noted that there are few databases included in this review article for comparison, such as Pathway Commons and GOLD.db, which do not satisfy the quality of a primary database as these databases do not posses any self-curated pathway data  to acquire the knowledge of various resources from which they can collate different types of signal transduction information. To elaborately discuss this section, detailed comparisons of different types of available pathway data, which are present in different databases, are discussed for the interest of database end-users. It should be noted that the comparison discussed in this review are completely based on the open access data, which are provided by all the databases included in this review. The quality of data can be varied between open access and the purchased or licensed versions, but comparing those aspects is beyond the scope of this review. Moreover, to compare the data heterogeneity across different databases, a case study on human Hedgehog and Notch signaling pathways are also discussed. Simultaneously, the facilities, their limitations and challenges regarding various technical features involved in these databases are highlighted and the possible way out from those constrains are also proposed for future database development. Hence, the current review is intended to help both the database end-users and the databases developers by providing the comprehensive comparisons of the data contents and the technical features available in the selected databases.Escherichiacoli  and their analysis through various commercial software, whereas the academic databases are more inclined to the inclusion of open source pathway analysis tools, sharing of computer readable pathway data for ibraries . Althougibraries . To perfibraries , are alsibraries , 45–47. However, one can also think to compare the databases by grouping them in different categories on the basis of other criteria. For example, the signaling databases can be classified according to their mode of collating pathway data and can be grouped as ‘Primary or Self-curated’ database, ‘Secondary or Aggregator’ database and 'Hybrid' database, which possess both the self-curated and aggregated pathway data. Major academic databases KEGG, SPAD, DOQCS, NetPath, REACTOME, SignaLink, SPIKE, BioModels, INOH and PANTHER, and almost all the commercial databases BIOCARTA, GENE GO/METACORE, Cell Signaling TECHNOLOGY, PROTEIN LOUNGE, MILLIPORE, Applied Biosystems and INVITROGEN fall into the first category. All these databases contain manually curated data, which are mostly curated by the experts of this field. On the other hand, the academic databases Pathway Commons and hiPathDB can be classified in the second group. Pathway Commons integrate the pathway data from the signaling databases, REACTOME and NCI-PID. It also curates PPI data from BioGRID, MINT and HPRD, etc. Moreover, hiPathDB collates pathway data from KEGG, BIOCARTA, NCI-PID and REACTOME databases and subsequently removes the redundancy to provide a unified and reformatted pathway models. Other useful databases, such as WikiPathways, NCI-PID, GOLD.db, CPDB and InnateDB are dependent on self-curated as well as aggregated data from the primary databases and hence fall into the 'Hybrid’ database category. WikiPathways contains both manually curated data (provided by the pathway curators) and imported pathway data from NetPath, KEGG, etc. In NCI-PID, 137 human pathways (or 9248 Interactions) are manually curated by NCI-PID administrators (up to September 2012), whereas 322 human pathways (or 7575 Interactions) are directly imported from BIOCARTA and REACTOME databases. There are only three self-annotated pathways found in GOLD.db, whereas the information of other pathways is mostly collated from KEGG and BIOCARTA by the GOLD.db database developers. Similarly, InnateDB contains its own self-annotated pathway data and the imported pathway information from other resources, such as KEGG, NetPath, INOH, REACTOME and NCI-PID. It is also worth to mention that several other primary databases, such as Uniprot, Genbank, GO database, gene expression databases  also annotate the pathway-specific information in their annotated data sets, which sometimes prove to be beneficial for the researchers in this field.The purpose of the comparison of all the selected 24 databases, on the basis of individual features, is to review the current scenarios of the present cell signaling databases by considering the current requirements of the pathway analysis as reference point. It is previously mentioned that this comparison is not intended to compare the databases at particular direction or to a specific community of signaling databases, rather it addresses the available facilities, challenges, limitations and their reasonable solutions for these databases, which are faced constantly by both the database end-users and the database developers nowadays. However, to organize this broad topic, the entire comparison is divided into two parts: ‘Data contents’ and ‘Technical features’. It is a well known fact that the developers of any database mainly focus on two different aspects, firstly, the 'primary or raw data’ and/or the ‘secondary or processed data’ and secondly, the ‘technical operations’ through which a database can be smoothly accessed, queried and several applications can be run. Hence, to compare the databases, it is chosen to focus on these two major characteristics: (i) Pathway Information or data and (ii) Technical Details. In the subsequent sections, all the 24 databases under study are compared according to the various features within these two major categories. Moreover, to see the pros and cons of these features in real time scenario, simultaneously a case study is also performed on Hedgehog , 48 and viceversa. The different data types and the number of pathways calculated to construct this matrix representation are taken from the open access data presented in all the databases . However, it should be taken into the consideration that the pathway data provided by these commercial databases may vary in both quantitatively and qualitatively between the open access and purchased modes. For example, the pathway information provided in the database PROTEIN LOUNGE is the reduced version of the main pathway information, which they provide only to its subscribers. According to this figure, NCI-PID has the highest number  of different types of signaling pathways present in its database, whereas BioModels , disease specific , immunological  and apoptotic, function in human body throughout the embryonic to adult stages. In total, 19 such broad classifications of pathway data types are found across all the databases. In ioModels  and DOQCioModels  contain ioModels , KEGG, RioModels , PROTEINioModels  are the insilico. BioModels provides the model files in various computer readable file formats such as SBML, Octave (m-file), Scilab and BioPAX, whereas DOQCS provides the model files in GENESIS file format  Pathway’ and NCI-PID as ‘Signaling events mediated by the Hedgehog family’). One of the reasons behind the heterogeneity of the pathway nomenclature is that there is no standard naming convention exists for the biochemical pathways. Moreover, assigning an appropriate name for a particular pathway is another problem for the database developers; as an example, in case of Hedgehog pathway one can assign its general name ‘Hedgehog Pathway’, but the other can assign its name on the basis of the main proteins of this pathway, such as ‘GLI mediated Hedgehog signaling pathway ’ or ‘SHH‘Ontology ’-based pathway annotation and nomenclature can be a reasonable solution for this problem. By forming the ontology-based pathway annotation tree, one can easily assign a specific name and the functional annotations for a particular pathway in the database. There are two such types of ontology database exist: INOH  and BIOPThe variations in data curation by the developers make the distribution of pathway data in different databases more heterogeneous in nature. To examine the extent of this data heterogeneity, a case study on Hedgehog and Notch pathway is performed by manually counting the number of molecular species  and interactions  available across different databases A and B.This comparison shows that in case of both of these pathways, NCI-PID (an academic database) contains the highest number of molecular species and interactions in its database, whereas GENEGO and MILLIPORE provide more information when compared with the other commercial databases. Besides, the other databases, such as NetPath, CPDB, Pathway Commons, PANTHER and BioModels, also serve as the major resources for the curation of these two pathways A and B. It is also observed that the heterogeneity of total number of signaling pathways (human) in different databases is seemed to be widely varied in different databases C. As thiTo find out the reason of this data heterogeneity, it is required to know the sources from where most of the databases extract the data and the procedures through which these data are extracted from those sources. It is observed that despite the presence of several data sources, till now the most reliable and widely used resource is the experimental data available in the published literatures. Most of the primary databases, such as KEGG, REACTOME and PANTHER, are developed on the basis of manually curated pathway data from the experimental data published in literature or any other related resources. Hence, it is observed that to curate the pathway-related information, manual curation is the most accepted procedure till now. Simultaneously, it is also important to note that the accuracy and the reliability of this procedure depend on the intelligence of human resources, who are involved in the manual curation and extraction of pathway information from the large pool of experimental data. As a result, this manual curation procedure has both advantages and disadvantages. The advantage is that the prior knowledge of manual data curation, such as biological interpretation, reasoning and the syntax, which are highly required to curate the pathway information and reconstruction of new pathway map, can only be achieved through this process. On the other hand, the major drawback of this method is that the pathway data is not available from a single experimental resource as it is highly dispersed in multiple published articles, and hence, to extract the valuable information from this fragmented data source, a large number of human resources are required to be involved throughout the entire process. Significantly, it is obvious that the strength of human resources of different databases is not homogenous, and thus, the depth of the data curation varies significantly across different databases. Moreover, because human intervention is widely involved in this method, therefore the data contamination due to human errors can be another disadvantage and can lead to the serious error propagation to other resources and experimental findings.However, because of its high level of authenticity and acceptability, till now manual curation is regarded as the best method for the pathway database curation when compared with the other methods like computational data curation, homology-based pathway reconstruction, genome mapping, comparative genome analysis, etc. . One of The data heterogeneity sometimes creates confusion to choose the appropriate and comprehensive data from an exact database. It is therefore developer's responsibility to minimize all the ambiguities and create the database with high level of authenticity and reliability. It can be performed by regular updating of the database with current research outcomes and providing the appropriate cross references to the uploaded reactions data for easy verifications. It is observed that almost all the databases provide the literature references for the uploaded pathway data. Although, except few databases (e.g. BioModels and NETAPTH), the reference(s) for each included interactions of a particular pathway are not much observed in other popular databases and hence the reaction annotations, such as literature references, experimental details, stoichiometry and rate parameters, are not easily available for a specific reaction from these databases. As a result, a large amount of molecular reactions data is still dispersed in many signaling pathway databases without proper annotation. To annotate that data properly, a dedicated database of biomolecular reactions and assignment of a unique identification number of each interaction is required. However, it is already reported that a large number of interaction data available in the PPI databases, such as IntAct , BiogridUncontrolled regulation of several oncoproteins in signaling pathways are found to be involved in various cancers and other diseases . Hence, It is experimentally proven that the same signaling pathway can be activated differentially in different tissues or cell lines and thus can exhibit tissue-specific inhomogeneous expression patterns of the target genes or proteins , 71. Apainsilico models, the signaling pathway databases have the scopes to provide the major contributions.Several computational models on different cell signaling pathways using various mathematical methods are proposed till now, out of which the mathematical models using the Ordinary Differential Equations, Graph theory, Boolean model, Fuzzy logic, etc. are most popular and widely used , 79–83. insilico model development. BioModels is such kind of database that serves as the repository of the insilico computational models of the signaling pathways or the biological processes mined from the published literatures. Using Path2Model , it is very much difficult to follow certain rules or standard for extensive data curation, storage and sharing process by the database developers. To accelerate the data sharing process and to ease the tremendous efforts, which database developers put on the data curation process, this study claims for the necessity of a general computational and technical platform, through which vast amount of pathway data can be easily accessed from all the available databases and simultaneously can be processed for further pathway data sharing and analysis purposes. To develop such general computational benchmark, it is utmost necessary to understand various technical details, such as the architecture, database schema, file formats and search options, of the existing databases. Hence, this article also evaluates the technical or computational facilities, which are provided in different databases to the general users at free of cost or without any license or agreement.To compare the tools and techniques available in the databases, it is first required to compare the database structure and schema as the smoother access of different computational operations is based on these two factors , 93. A dinsilico or mathematical models, disease-specific information, etc. It is found that databases like CPDB, REACTOME, NCI-PID, Pathway Commons, hiPathDB and BioModels are using object-oriented database schema, and thus, these databases are dynamic in nature. On the other hand, the other databases are mostly focused on the pathway diagrams and provide the data in xml, text and/or image formats. In these databases, the data uploading or updating is the major problem compared with the RDBMS systems. Hence, it is clear that to make an advanced, easily searchable, updateable and accessible database, RDBMS is the best option compared with the other presently use database management system. Using this schema, all the interactions, proteins, their associated information, pathway pictures, etc. can be stored in a tabular format by assigning specific database identification number and can be used to link and parse the data dynamically. Parsing accurate data and to perform complex searches with less time and effort is also a major concern of any database, which can be easily performed by using RDBMS.While reviewing the database management systems used by the signaling pathway databases, it is observed that most of the databases are based on flat file schema. Initially, the signaling pathway databases were created to host only the pathway diagrams and the description of the pathway. However, as time proceeds, the database becomes more complex as it is started to provide not only the pathway diagram but also the annotation of the pathway objects , protein expressions, Supplementary Table S2, different types of search options used by different databases are provided.The search options provided in the signaling pathway databases also vary according to the database schema used by the database administrators. There are different types of search options available in different databases; however, it is broadly categorized into two sections: Simple text search and Advance or complex search. Simple text search can be performed by entering simple text or phrase in the search field. It also takes pathway name, pathway ID, biological process, gene or protein name, disease name and sometime multiple terms. This type of simple text-based search option can be included in any types of data structures, although it takes more time if the data is not well structured and indexed in the database. On the other hand, advance or complex search is more computationally rich wherein users can filter or restrict the search terms and run multiple queries at a time to get more specific information from the database. For example, to get the drug names against a specific pathway or protein and have specific side effects, user can run a complex query in the database, through which it will get all the drugs or drug like molecules that target the specific pathway or a protein molecule including their side effects. In Supplementary Table S2).Simultaneously, browsing the pathway names is also an important feature, which is required for searching or selecting a particular pathway from the list of available pathways. It is observed that there are multiple ways adopted by the database developers through which a database can be browsed and out of all browsing by alphabetical or by pathway accession number is most popular, followed by the species/organism-specific browsing. There are other types of browsing options, such as Cellular or Biological Process, Disease names, extracellular signaling molecule/ligand or Category of Pathway-specific browsing, available, through which the databases can also be browsed. In BioModels and WikiPathways, the pathway data are categorized according to their curation level, i.e. the featured pathways and curated pathways, and thus it also provides an extra dimension to browse and select the pathways  and various reaction processes. On the other hand, the standard graphical notations provided by SBGN are successfully implemented by REACTOME and PANTHER database to show the complex and large pathway diagram in the database web interface. Using the various notations available in SBGN format, these two databases have not only been able to show a large pathway but also been able to show various complex reaction processes such as complex formation, conformational modification of proteins, phosphorylation and compartmentalization of chemical reactions of a signal transduction network. However, it is observed that the pathway image in PDF format and the pathway data in BioPAX, followed by SBML are the mostly used file formats. There are other specific file formats, such as KGML and GENESIS . In case of online or web-based tools , databases, such as WikiPathways, BIOCARTA, REACTOME, CPDB, KEGG and PANTHER, provide the best user-friendly tools in their web sites. Hence, through this comparison based on the use cases of technical features, it can be concluded that the databases, such as REACTOME, PANTHER, WikiPathways, NCI-PID, BIOCARTA and BioModels, have created a great impact in this field of study  are easily searchable while searching Hedgehog and Notch pathway information by using simple text or key words search . The advanced search option is also seemed to be useful in this context. To view the pathway images, SVG or PDF format  is more suitable than the other file formats, whereas for pathway data sharing, SBML or BioPAX  is found to be more useful than the other pathway data sharing file formats, as these formats are easily readable by most of the third party software. In case of downloading the pathway data in local machine, direct static download link provided by most of the databases are useful. However, to perform large-scale pathway analysis, automated pathway annotations or to extract the non-redundant pathway data from various resources, dynamic web-based API service  is useful  is named in multiple ways in different databases. Hence, the necessity of a simple but more specific guideline which would be used to name and annotate the signaling pathways in the database is required. Like the other naming conventions , there should also be a proper way to name the pathways with unique identification number. As most of the signaling pathways are initiated by some specific receptors, therefore receptor-based nomenclature or the ontology-based pathway nomenclature would be a reasonable solution. However, the implication of such nomenclature system is always a debatable matter.It is clearly mentioned in the previous sections that the heterogeneity of the number of molecular species and reactions present in the pathway databases is mostly caused due to the lack of specific boundary conditions for pathway reconstruction. Imposing a boundary to the depth of data curation can restrict the data heterogeneity. It is observed that the inclusion of various cross talking reactions and molecules with the core signaling pathway in different databases brings this heterogeneity. A case study has been performed on Hedgehog signaling network to show how various cross talking molecules and reactions with the core pathway bring such types of data heterogeneity across different databases. For example , and the users will get the module specific or whole pathway information. REACTOME has such kind of hierarchical modules to allocate the pathway in the database. Notch pathway is stored in this database in four parts or modules: Pre-NOTCH Expression and Processing, Signaling by NOTCH1, Signaling by NOTCH2, Signaling by NOTCH3, Signaling by NOTCH4 and Signaling by NOTCH5. Each part is connected with each other, and upon users' preference, it can show the entire pathway or a part of the pathway.It is a known fact that the signal propagation in signaling pathway through various signaling proteins is not a static process rather it is very dynamic in nature. Various types of protein complexes (transient or stable) are formed during the course of signal transduction, which are found missing in most of the signal transduction information provided by the databases, except REACTOME, InnateDB and NCI-PID. Moreover, the databases are also unable to show the conformational changes of various proteins occurred at the time of pathway activation, except REACTOME, which displays the conformational changes of various proteins using SBGN notations. It is obvious that the static sequential diagrams of signaling pathways provided in the signaling databases are unable to show such complex dynamics of a cell and hence pathway animation would be one of the best solutions in this regard. Besides, by using SBML notations in CellDesigner, one can also incorporate such protein complexes in the pathway diagram, which may cause the diagram look messy, but it would be more information rich when compared with the ordinary wiring or flow diagram of signaling pathways.The major problem which most of the signaling pathway databases face till this date is the pathway data curation and its subsequent annotation, which are mostly dependent on active human interaction and intelligence. Right now, there is no tool or computer program available, which can fully automate this process. Several attempts are being taken in the pathway reconstruction procedure, such as various text mining algorithms are used to extract the relevant information from the biomedical journals and patents , pathwayHowever, the authenticity and accuracy of automatic pathway data curation and annotation is still under the scrutiny of various database developers and end-users. The percentage of accuracy achieved by the fully automated process to reconstruct a pathway diagram is still very low, and hence it is ignored by the most of the database developers. However, to keep the database most up to date with the current research outcomes, it is also required to reduce the entire manual work load and subsequently speed up the whole process by including manual checking process to further verify the automatically curated data. In this context, a semi-automated computer program, which will help to draw, annotate, check, validate and update the pathway resources, could be used by the developers. The software developers should also be encouraged to work on this major issue as there is a huge scope of research still left in this area.Regular update of the pathway diagram by the newly identified species or reactions is a major concern to the database developers. Usually, most of the database provides the pathway diagram in a raster file formats  and further modification of such file is a repetitive job. Annotating the pathway molecules in the diagram by its cellular locations, providing the appropriate legend for different molecular entities and biochemical reactions, etc. are also a major concern and require intense human interaction. Therefore, to automate the reconstruction of pathway diagrams, one should use the XML-based dynamic image file, such as SVG , which is easily updateable, easy to host in any modern web browser and easy to convert in any of the raster file format by proper converter tool. A dedicated database using RDBMS can be created where all such type of pathway drawing-related information and relevant updates  can be stored and depending on user's request the information can be fetched from the database to generate a new SVG file to be served to the user's browsers. Hence, the users will always get up-to-date pathway image, and the developers need not have to redraw the pathway images repeatedly. The quality of such vector image is also good when compared with the raster image, as it can be zoomed or magnified up to many folds without distorting the figure's objects.insilico analysis using the pathway data available in the databases is the major focus nowadays in systems biology research. Performing such analysis successfully requires the integration of various working modules  by the theoretical biologists of this field. Moreover, the operation of such insilico analysis requires the involvement of various interdisciplinary scientific communities and hence a common computational platform, which will provide and integrate all such working modules, altogether is absolutely required. Therefore, inclusion of such common computational platform in the database web interface is now one of the major challenges to the current database developers. There are few databases, which started to integrate different types of third party computational tools  in the database interface. However, the operation of such applications slows the regular functions of the database as most of them are based on JAVA applet and sometimes demands the knowledge of high-tech computational ability from the users. On the other hand, simple PHP or other dynamic language-based computer applications to perform various computational operations with step wise manual can resolve this problem, which is also easy to implement in the database web interface. These types of applications should be made in such a way that it can act as an integrated part of the database, and various computational tasks can be performed simultaneously with less effort and time. For example, Pathway Commons have nicely built a common platform to perform various pathway analysis tasks in the database web interface. More details are provided in the ‘Online Analysis Tools’ section and Developing, testing and validating hypothesis by insilico simulation), annotation or indexing of pathways, pathway data curation process, wide data heterogeneity across different databases, etc. Similarly, from technical point of view, the previous reviews did not elaborately reviewed the technical features such as database schema or management systems that are used by the databases, file formats of the pathway data, searching or browsing options, data download and FTP/API services, etc.. Although there are various studies on cell signaling network available, which report some limitations regarding large-scale data integration from various pathway resources, and simultaneously compare their current constraints , which are available in different databases. It will help the users to select the appropriate database and to curate the data easily and accurately.Supplementary data are available at Database Online.