article: The profusion of high-throughput instruments and the explosion of new results in the scientific literature, particularly in molecular biomedicine, is both a blessing and a curse to the bench researcher. Even knowledgeable and experienced scientists can benefit from computational tools that help navigate this vast and rapidly evolving terrain. In this paper, we describe a novel computational approach to this challenge, a knowledge-based system that combines reading, reasoning, and reporting methods to facilitate analysis of experimental data. Reading methods extract information from external resources, either by parsing structured data or using biomedical language processing to extract information from unstructured data, and track knowledge provenance. Reasoning methods enrich the knowledge that results from reading by, for example, noting two genes that are annotated to the same ontology term or database entry. Reasoning is also used to combine all sources into a knowledge network that represents the integration of all sorts of relationships between a pair of genes, and to calculate a combined reliability score. Reporting methods combine the knowledge network with a congruent network constructed from experimental data and visualize the combined network in a tool that facilitates the knowledge-based analysis of that data. An implementation of this approach, called the Hanalyzer, is demonstrated on a large-scale gene expression array dataset relevant to craniofacial development. The use of the tool was critical in the creation of hypotheses regarding the roles of four genes never previously characterized as involved in craniofacial development; each of these hypotheses was validated by further experimental work.
  Human knowledge relevant to biomedical research is expanding at an exponential pace. Over the last twenty years, more than 10 million publications have been indexed by the National Library of Medicine (NLM) and made available through PubMed, reflecting a compounded annual growth rate of more than 4. 8% [1], [2]. Structured knowledge, in the form of molecular biology relevant databases, has also been growing at an impressive rate. The journal Nucleic Acids Research publishes an annual compendium of peer-reviewed databases relevant to molecular biology; the 2008 issue reported on 1,078 such databases [3]. While intense specialization has in many cases made it possible for biomedical researchers to know everything practically relevant in a very narrow domain, a breakdown of disciplinary boundaries and the fundamental interconnectedness of biological systems have rendered specialization an increasingly impractical strategy for keeping up with biomedical knowledge. Information about fundamental molecular structures and functions, such as mutations or protein-protein interactions, are spread across the entire literature. For example, [4; figure 1] demonstrated that nearly 40% of the more than 5,000 journals indexed in PubMed in a typical year contained at least one assertion regarding protein transport, interaction or expression that could be found by a text mining system. One approach to dealing with this overwhelming amount of information is to organize human experts to curate key aspects of it, resulting in databases of formally represented assertions with pointers to the evidence in the literature. Over the last 6 years, the U. S. National Institutes of Health has invested more than $52 million to support ontology development and use (Personal communication from Peter Good), including the Gene Ontology Consortium and the National Center for Biomedical Ontology. However, even this large investment has been simply unable to keep up with the volume of relevant publications; [1] showed that even under extremely optimistic assumptions it will be decades before annotation will be complete and up to date. Furthermore, not all human knowledge of biomolecular function is explicitly stated in any database or publication. Molecular biologists often make inferences regarding the likely function of a molecule based on factors such as homology, interaction partners, or other methods; this approach has been called the “post-genomic approach to protein function” [5]. Protein interactions are reasonably well characterized experimentally in yeast, but much less so in other organisms. As of this writing, the Database of Interacting Proteins (DIP) [6] contains records regarding 18,331 interactions among 4,923 yeast proteins derived from 23,344 experiments, likely a close to complete inventory. However, there are only 415 curated interactions among 307 mouse proteins derived from 595 experiments in the database—likely fewer than 1% of the true protein-protein interactions. Recently, computational approaches to protein function inference such as [7], [8], [9] and others have extended interaction predictions to generate functional categorization of dramatically larger numbers of proteins. As these inferences of function are less reliable than experimental observations, most computational approaches associate a likelihood or reliability with each prediction. Advances in instrumentation are also generating molecular data at ever increasing rates. High-throughput (also known as genome-scale) assays for detection and analysis of gene expression, genetic polymorphisms, macromolecular interactions and other fundamental processes are generating datasets that contain information about the structures or activities of on the order of 106 different genes or gene products at a time. More than 200,000 such assays from more than 8,300 different experiments are publicly available from the National Library of Medicine' s Gene Expression Omnibus catalog (http: //www. ncbi. nlm. nih. gov/geo/ viewed on April 7,2008), and far more results of high-throughput experiments are available in more restricted settings. Experiments that exploit these genome scale assays often generate results that implicate dozens to hundreds of genes or gene products related to a phenomenon under study. The amount of information regarding even just these significant results (and relevant homologs) in gene-centric databases and in the research literature is often overwhelming, yet the proper interpretation of the results requires taking stock of all of that knowledge. Furthermore, as the revolution in systems biology has made clear, it is critical to analyze the specific interactions among the genes, not just the genes in isolation. In a set of hundreds of relevant genes, there are tens of thousands of potential interactions to consider. Analyzing all of the relevant genes and interactions in genome-scale data, while important to advancing human understanding of biomedical phenomena, is a truly daunting task.
  Knowledge in our system is constrained to be compatible with the World Wide Web Consortium' s Web Ontology Language (OWL). While program internals represent the knowledge graphs more directly, it is always possible for the Hanalyzer to export an OWL version of its knowledge, and to import knowledge in OWL format. The OWL syntax for representing properties is a binary relation, linking two individuals or an individual and a value; however, for a great deal of knowledge in molecular biology it is natural and convenient to link an individual to more than one other individual or value – for example, the process of transporting a protein from one subcellular location to another would naturally involve a relation between the protein and two locations. For this reason, we adopt the practice recommended in the W3C working group note Defining N-ary Relations on the Semantic Web of 12 April 2006 (http: //www. w3. org/TR/2006/NOTE-swbp-n-aryRelations-20060412/) pattern 1, primarily following use case 3. The quantification links (described in more detail below) follow use case 1. Provenance information is stored as an annotation property. Entities in our knowledge network that are clearly and unambiguously interpretable by the biomedical community are termed “fiducials. ” A fiducial is either a specific element of a community-curated ontology (such as available through the National Center for Biomedical Ontology' s BioPortal, http: //www. bioontology. org/bioportal. html) or derived from a specific entry in a publicly available database, such as a particular identifier from the Entrez Gene database (http: //www. ncbi. nlm. nih. gov/sites/entrez? db=gene). All representations of genes, gene products, macromolecular sequence features, molecular functions, biological processes, metabolic pathways, subcellular locations, cell types, organisms, diseases and drugs in our system are fiducials. Not all relationships between entities in our knowledge network can be mapped to elements of a community curated ontology. Arcs between fiducials are used to represent non-fiducial elements. For example, in addition to relations defined in the OBO Relation Ontology (http: //www. obofoundry. org/ro/) such as ‘part-of, ’ we use at least two additional relationships: One non-fiducial link represents the very abstract relationship that specifies a connection of any kind between a pair of proteins. Ultimately, the user interface displays this very abstract relationship, which, borrowing terminology from [9], we will refer to as a semantic relationship. The other non-fiducial link quantifies the overall inferred reliability of the semantic relation (see below for how this is calculated). Also, as described in detail below, some knowledge sources may assert links among fiducials that do not correspond to relations from the OBO Relation Ontology. The totality of all entities and relationships in this knowledge representation in the system at any given time is called the knowledge network. The fiducials and semantic relationships in the knowledge network are initially populated by a series of processes that extract information from databases and from the literature. Again following the terminology of [9], we call these processes “experts. ” The size of the graph produced can be limited by seeding the knowledge-base with a target set of fiducials (usually a set of genes of interest from a particular experiment), and requiring any addition to the knowledge-base to have a relationship involving one of these target fiducials. Unless otherwise noted, the knowledge networks discussed below begin from a target list of 8923 Mus musculus genes that were differentially expressed among at least one pair of conditions in the craniofacial dataset described below. The genes were specified by identifiers from the Mouse Genome Informatics (MGI) database [18] (or Entrez Gene or Uniprot IDs, which can be readily translated), and no distinction is made between genes and gene products. Relationships describing protein-protein interactions are extracted from the Biomolecular Interaction Network Database (BIND) [19], Database of Interacting Proteins (DIP) [20], Molecular Interaction database (MINT) [21], the IntAct database [22] and the RIKEN protein interaction table [23]. Additional relationships are taken from the list compiled by [24] which relates a protein annotated to the Gene Ontology [25] Molecular Function term “protein binding” (GO: 0005515) with evidence code IPI (inferred from physical interaction) to the protein identified in the “with” field of the term annotation. Interactions from all databases are combined and divided into experimental assay groups by canonicalizing spelling variants among text strings describing the assay (e. g. , TAP and tandem affinity precipitation) and grouping like assays (e. g. , CLASSICAL-TWO-HYBRID, MATRIX-TWO-HYBRID, TWO-HYBRID, TWO-HYBRID-ARRAY, TWO-HYBRID-TEST all represented by the single label TWO-HYBRID). Any relationship without an assay description is labeled UNKNOWN. This process results in 4,544 relationships among 1,693 targeted MGI identifiers with 25 assay type labels. Each assay type becomes an “expert, ” and can therefore be assigned a reliability score independent of the other assays. Relationships describing protein-DNA interactions are extracted from the TRANSFAC 10. 2 database [26] by relating a protein to the transcription factor recognizing a given sequence motif found in the regulatory region of the protein. The expert derived from this information (Transfac) contains 580 relationships among 434 MGI identifiers. Additional putative protein-DNA interactions are extracted from the PReMod database of genome-wide mammalian cis-regulatory module predictions [27] which catalogs phylogenetically conserved regulatory modules between human and mouse. The resource lists the TRANSFAC motif identifiers of elements in a conserved module, together with the upstream and downstream genes. Two different experts are derived from this information, one which relates a transcription factor recognizing any motif in the module to both the upstream and downstream genes (PReMod) and one which relates two transcription factors if they recognize motifs in the same identified conserved module (PReModM). The PReMod expert asserts 345,814 relationships among 13,852 targeted MGI identifiers while the PReModM expert asserts 17,317 relationships among 189 targeted MGI identifiers. The large number of relationships from these experts suggests the potential of a high level of noise, which is expected for computational predictions. The OpenDMAP system [4] was used to extract information from all abstracts in Medline regarding protein transport events, protein-protein interaction assertions, and what proteins are expressed in which cell types. OpenDMAP is particularly well suited to this task, since its information extraction patterns are explicitly associated with a knowledge-base, and all of its outputs are in terms of the representation scheme of the knowledge-base. Although discussed in detail in [4] a brief example describing the extraction of protein transport assertions from the literature here is illustrative. Protein transport is a 4-place relationship between two proteins (a transporter and a transportee, represented by MGI IDs) and two subcellular locations (fiducials from the Gene Ontology cellular component subtree). Most assertions do not mention all aspects of that relationship, although to be extracted at least one protein and one compartment had to be recognized. To map this extracted information into the network, up to five pairwise relationships are created. An expert (Transloc) derived from this information asserts a relationship between the transporter and the transportee, using the Entrez gene ID to MGI identifier mapping available at the MGI website, for a total of 157,764 interactions among 1108 targeted MGI identifiers. Protein-protein interactions extracted from the literature can be translated into network arcs straightforwardly. Extracted assertions regarding the type of cell that a protein was expressed in were mapped to a relation between a gene and an element of the cell type ontology. A total of 265,795 interaction instances and 176,153 expression instances were extracted from all Medline abstracts. Of these, 8292 interaction instances and 7035 expression in cell type instances could be mapped to a targeted MGI mouse gene, resulting in the assertion of 4525 relations among 3157 genes based on literature assertions of protein-protein interactions, and 127,283 relations among 1677 genes being expressed in the same cell type (fiducials from the Cell Type Ontology). Even when the previously described information extraction system is unable to extract a direct relationship from biological literature, systematic overlap between publications that merely mention two genes can be taken as indirect evidence of a semantic relationship between them. Several systems have used the existence of an article that mentions a pair of genes as evidence of an interaction between them (e. g. [28]). Others use a probabilistic measure based on mutual information [29] or the hypergeometric distribution [30], [31] and extract relationships exceeding a probability threshold. However, [32] demonstrates that a related measure (thresholded asymmetric co-occurrence fraction or ACF) provides more robust performance in network-based protein function prediction. Since the reliability of finding and normalizing gene mentions in free text is substantially higher than that of more general information extraction [33], we apply this technique in addition to the OpenDMAP approach described above. The ACF measure [32] calculates the proportion of the number of shared mentions relative to the number of mentions the less frequently-mentioned gene in a given pair, incorporating a bias toward relationships involving less well-studied genes. The result is a set of inferred relationships between a pair of genes whenever their ACF>0. 5; this expert is called co-Lit. Each expert also records the support for each assertion it makes, including at least a pointer to the source of the data and, when possible, a publication (as a PubMed identifier) and the date when the assertion was created. The reporting component can show this provenance information and link to the original database entry or document passage during analysis. Note that finding multiple relationships between a single pair of entities is entirely possible. For example, a pair of proteins may be related via an expert that extracts knowledge from a protein-protein interaction database, and by another that does text-mining searches for protein transport statements in Medline abstracts. When multiple relationships are found between a single pair of entities, the reliability of the semantic relationship increases. Once the initial knowledge-base is created, it is enhanced by reasoning processes that add additional relationships. These processes are also called experts. When necessary for clarity, experts that obtain knowledge by reading an external source are called reading experts, and those that infer additional knowledge are called reasoning experts. An important method for adding semantic relations between genes is to infer that such a relationship exists when two genes have certain properties in common. A series of experts asserts semantic relationships between pairs of genes based on: shared membership in a signaling or metabolic pathway (co-KEGG) [34], shared annotation to a particular biological process (co-BP), molecular function (co-MF) or cellular component (co-CC) [25], shared gene knockout phenotype (co-Pheno) [18] or shared protein domain assignment (co-Interpro) [18]. For resources involving a nested hierarchy of ontology terms, such as the Gene Ontology (GO) or the Mammalian Phenotype (MP) ontology, relationships exist at a number of levels. For MP, a relationship is added among proteins annotated to their most specific term while for GO, certain terms are first merged when the information content score by the Jiang measure between the terms exceeds 19. 0 (see [35], [36] for details). These experts assert between 7,873 (co-KEGG) and 267,317 (co-BP) relationships covering a combined total of 22,922 MGI identifiers. Another set of inferred relationships links sets of ontology terms using the ontology enrichment process described in [37] to link molecular functions and biological processes from the Gene Ontology to small molecule participants from the Chemical Entities of Biomedical Interest (ChEBI) ontology. For example, this process creates relationships between the GO molecular function terms “Calcium Signaling” and “Calcium Transport” and the ChEBI term “Calcium (2+). ” Additional semantic relationships between genes are inferred if such enrichment results in two genes sharing a small molecule participant in a molecular function or biological process (co-ChEBI). For example, this inference adds a semantic relationship between pairs of genes that have functions each of which in turn has calcium as a participant. Similar inference is made over the GO cross-products (see [38] and http: //wiki. geneontology. org/index. php/Cross_Product_Guide). A critical aspect of the reasoning component is the ability to assimilate the information from all experts and estimate the confidence that a relationship exists between any given pair of proteins. The collection of assertions from both reading and reasoning experts contains a large number of false positives due to uncertainty in a computational prediction, experimental noise in an assay, or even the intentionally noisy nature of inferred relationships. For example, it is not likely that all cytoplasmic proteins interact as the co-CC expert suggests, yet co-localization information can usefully contribute to estimating the likelihood of a semantic relationship when integrated with the other evidence types. Biological data integration techniques have been widely studied in the literature, ranging from simple measures which assign higher confidence to assertions shared by multiple experts [39], [40] or based on certain relationship network topology characteristics [41], [42], [43], [44], to more sophisticated integration strategies which use machine learning techniques to estimate interaction likelihoods, such as probabilistic graphical models [45], [46], [47], [48], [49], [50], [51], [52], [53], [54], [55] or kernel methods [56], [57], [58]. Many of the techniques attempt to estimate error rates of the individual expert types before integration using either a gold standard [59], [60], [61], [62], [63], [64], [65], [66], [67], [68], or the set of data sources themselves to determine relative reliabilities [36], [69]. In mouse, there are already too few available sources for determining relationships to justify withholding one as the gold standard. Moreover, since our system attempts to capture a variety of semantics for what type of relationship might exist between two entities, determining the appropriate gold standard is difficult. The consensus reliability estimate [36] used in our system avoids the use of an explicit gold standard by computing the consensus number of assertions for a given relationship among all experts and assigning a higher reliability to a given expert if many other experts agree with its assertions on average (see [36] for details). Since many of the reasoning experts assert a large percentage of all possible relationship pairs, the consensus numbers used in the averaging are computed only over experts which are derived from sources explicitly naming both proteins (protein-protein interactions, 25 experts), protein-DNA interactions (Transfac, PReMod, PReModM), translocation events (Transloc), and literature co-occurrence (co-Lit) ). All assertions from a given expert are assigned the reliability of that expert. One of the most popular methods to combine individual reliabilities is to assume independence of experts (naive Bayes assumption) and compute the integrated likelihood P for each relationship using the Noisy-OR function P = 1−Πi (1−ri) where ri is the reliability of an expert i (scaled if necessary into the range 0 to 1 to allow interpretation as probabilities) [53], [65], [66], [67], see also the useful exposition [70]. The Noisy-OR function has the useful property that the probability of a relationship is high with at least one reliable assertion yet increases with additional support. This property is especially relevant in biology, where it is often difficult to identify false negatives; a given assertion is strengthened by additional information but unlike the case for estimating the reliability of an expert on the whole, an individual assertion is not penalized for lack of additional evidence. Moreover, since the experts are assumed to be independent, experts can be removed from or added to the analysis without excessive re-computation. The purpose of building this large, integrated network is to facilitate the exploration of high-throughput data in light of what is already known, with the goal of generating explanations of the observations and also generating novel biological hypotheses. In order to report the aspects of the knowledge network that are relevant to understanding a dataset, it is necessary to both select appropriate sub-networks for presentation and to present them in a comprehensible and useful way. A simple approach involves visualizing the knowledge network that includes particular fiducials. A gene list can be used to generate the knowledge sub-network that includes them, similar to the approaches presented in [47], [71], [72], [73]. Such a sub-network can also be extended to include other genes linked to the query set by sharing highly interconnected subcomponents that might represent protein complexes (see for example [44], [74]. Sub-networks, along with attributes of the fiducials and linkages among them are visualized using Cytoscape [75], an open source network visualization platform. A Cytoscape plugin, CommonAttributes, was written that allows the user to trace the provenance of links in the knowledge network and directly access the underlying data sources and publications (Figure 2). This approach was used to identify functional explanations of a gene list by exploiting inferences not available in any of the individual data sources [76]. A more effective method of exploiting the knowledge network is to create another quantitative network based on the experimental data (called a data network) and combine the two networks in various ways. In the application reported here, a data network is constructed from the results of a gene expression array experiment. Nodes in this data network are genes that exhibited differential expression in the experiment, arcs connect genes whose expression levels are correlated at an above threshold level, and arc weights are the absolute value of the correlation coefficient (see below for details). Data networks can be generated from any sort of data that can be represented as a weighted graph among fiducials, not just expression arrays. Methods for combining the knowledge network with the data network can highlight linkages in the data that are well supported by existing knowledge, thereby facilitating explanation, or can highlight linkages in the data that are not well supported by existing knowledge, facilitating the generation of novel hypotheses. Both approaches can be exploited together, as demonstrated below. Other studies have used protein interaction networks together with p-values from tests of differential expression gene expression to identify ‘active’ sub-networks in an input graph [17], [57], [77], [78] and to improve expression profile clustering using a combined distance metric computed from profile correlation and network distance [79], [80]. Our system integrates these two prior approaches, using a combined distance metric to identify active (and explanatorily interesting) sub-networks. The distinction between the knowledge network and the data network allows comparison at the level of networks, in the spirit of multiple graph approaches such as [81], [82]. Our approach builds on those in two ways: by exploiting the inferences in a dynamically generated and extremely broad knowledge network, and by offering multiple combination functions that support both explanation and hypothesis generation applications. Creation of combined networks that integrate the knowledge and data networks in different ways is a key step. Due to the use of fiducials in both the data and knowledge networks, aligning the nodes of these networks is trivial. In contrast, there are many alternatives for combining the arc weights from the knowledge and data networks. The semantic integration combination functions (e. g. noisy-OR) could also be used to combine corresponding arcs in the data and knowledge networks, but many other alternatives are also available, and some are superior. Approaches based on likelihood ratios for individual sources [60], [64], [83], [84] typically assume independence (naive Bayes) and simply multiply likelihoods. When the assertion probabilities can be interpreted as p-values, [69] review three techniques from statistical mechanics for integration: Fisher' s F, Mudholkar-George' s T, and Liptak-Stouffer' s Z (see [85]). Averaging the probabilities or averaging logistic functions of the probabilities, as used in [80] are also possibilities. The effect of combining probabilities from two sources using various techniques is illustrated in Figure 3. All functions except those denoted Average and Hanisch Logit exhibit the behavior described earlier about Noisy-OR, where the value of the combined probability is 1. 0 if at least one of the sources assigns a probability of 1. 0 (observed as the red area touching the z = 1. 0 plane). Mudholkar-George' s T and Liptak-Stouffer' s Z have the additional property that the combined probability is 0. 0 if at least one of the sources assigns a probability of 0. 0 (observed as the dark blue area touching the z = 0. 0 plane). In this context, these two functions are less applicable since negative relationships (probability of 0. 0) are difficult to observe. The remaining functions differ on how they treat intermediate probability values. Fisher' s F shows a rapid decline in combined probability compared to Noisy-OR which maintains a higher combined value when at least one is high. In contrast, Averaging and Hanisch Logit methods require agreement among sources to achieve a high combined value, allowing a value of 1. 0 only when both source probabilities are 1. 0. The sinusoidal curve of Hanisch Logit implements a thresholding effect where the combination is given more weight than in Averaging when both source probabilities are at least greater than 0. 5. Since the purpose of the combination network is to emphasize concurrence among the knowledge and the data networks, the Averaging and Logit methods are more appropriate than the others. In the application described below, the distribution of weights in the knowledge and data graphs are such that the Averaging combination method gives high scores to arcs that are supported in both the data and knowledge networks (behaving somewhat like a Boolean AND), while the Logit method privileges the high scoring arcs in the data network over those in the knowledge network (since the distribution of correlations is weighted more towards 1 than the distribution of knowledge confidence scores). Figure 4 shows in detail how an example link is created in each of these graphs.
  The use of two different combination functions to investigate the network enables the development of an investigative methodology that supports hypothesis generation through systematic network exploration. The top 1000 edges as scored by either function generate a network comprised of 945 genes and 1,743 total edges. This collection of high scoring edges is organized as 92 pairs, 15 triplets, seven small clusters (<10 nodes), one large ‘yarnball’ (551 nodes), and three medium-sized clusters (comprising 27 to 51 nodes) (Figure 6). One of the medium-sized sub-networks (total 45 nodes, 107 edges, is analyzed in detail here (circled in Figure 6), illustrating a typical use of the Hanalyzer. The above analysis generated hypotheses regarding the role of four genes (Apobec2, E430002G05Rik, Hoxa2, Zim1) in the development of the murine tongue. These hypotheses were tested by whole-mount in situ hybridizations to E11. 5 and E12. 5 mouse embryos, collected, prepared and hybridized as described in [115], stained with Hoxa2 [116], Apobec2, E430002G05Rik and Zim3 RNA probes, as described in [117]. The mouse Apobec2 probe was derived by PCR from E10. 5 FVB mouse head cDNA using the primers Apobec2F (5′-CCA GCC AGG CTT AGC TGC TGA CAG-3′) and Apobec2R (5′-GCT CAC CAG AAT GAG CAG ACG AAG-3′); the mouse E430002G05Rik probe was derived using the primers E43F (5′-GGT TTA TCA TCC AGT TGA GGT TTG G-3′) and E43R (5′-GCA GAC AGG TTG CTT TCC TGA-3′); the mouse Zim3 probe was derived using the primers Zim3F (5′-CGT ACA AGT GTG ACA AGT GC) -3′ and Zim3R (5′-GCA CAA ATG CTC CAA GTA GG-3′). As shown in Figure 12 all four genes are expressed in the developing tongue at E12. 5. Examination of the first arch tissue at E11. 5 indicates that neither Apobec2 nor E430002G05Rik are expressed at this time-point, although the former gene is clearly expressed in the developing cardiac region (A, D). However, by E12. 5 both genes are expressed in discrete regions of the developing tongue (B, C, E, F). At E11. 5 Hoxa2 expression is prominent in the second arch tissue and there is clearly a sharp boundary of expression with the first arch (G). Nevertheless, weaker expression is apparent in the core of the first arch, and expression is again visible in the tongue at E12. 5, presenting as bilateral stripes aligned with the anterioposterior axis (H, I). Zim1 expression is visible in the core mesenchyme of the first branchial arch at E11. 5 and by E12. 5 almost the entire tongue, with the exception of the ectoderm, is strongly stained. This data confirms all four hypotheses; each of these genes is expressed in the developing tongue. The expression patterns for the four genes are different though, indicating that their function may not be directly related. The almost uniform expression of Zim1 at E12. 5 suggests that it is marking the neural crest derived mesenchyme of the tongue that will give rise to the smooth muscle and connective tissue. Alternatively, it may identify the intrinsic skeletal musculature of the tongue. In contrast, the expression of the other three genes is consistent with their expression in different extrinsic tongue muscles that project out of the tongue and attach to surrounding skeletal components to allow tongue movement during swallowing and chewing.
  The version of the Hanalyzer described here built a background network for only mouse genes, and the data network was constructed from a particularly well-powered time and tissue gene expression array series. One important question is how well this methodology will generalize to other organisms and data types. [36] demonstrated many of the experts that compose this system can be used to build knowledge networks for other model organisms, including yeast, worm, and fly. Ongoing work involves building knowledge networks for human and rat as well. As many of the same types of experts are available for each of these organisms, expansion of the knowledge networks to other organisms is a straightforward software engineering task. One important issue in generalizing to other organisms is the question of how to build homology-based experts. As observed in [122], [123], [124], there are many factors that are involved in the successful application of homology-based protein-protein interaction networks. Such predictions are even more difficult and uncertain in humans and other eukaryotes, although promising methods that could form the basis for such experts have been published recently, e. g. [7] and [125]. The natural language processing aspects of the system increase its performance over database integration systems alone [32]. The OpenDMAP approach used is state of the art [126], but there is much room for improvement. One particularly important area for future work is in multi-organism gene normalization. While there is a great deal of data in the form of expression arrays over which gene correlations can be made, there are many other sources of high-throughput information that could be profitably analyzed using 3R methodology. Construction of quantitative data networks with genes or gene products as fiducials could be based on data produced by many high throughput experimental techniques, including proteomics, miRNA assays, genotyping, and others. What are the best methods for generating such data networks, and are there differences in the types of knowledge networks (experts) that are best suited to analyzing them? Another issue regards the inherently changing nature of biomedical knowledge. Experts can be re-run periodically to keep the knowledge networks up to date, but a variety of open research questions about handling time remain: Is it valuable to highlight more recent results for annotators? How should temporal considerations factor into the reliability calculations? Should reasoning experts take temporal considerations into account? How? While the experts used in the Hanalyzer proved to be useful for analysts, there are a large number of potential experts, both reading (external) and reasoning that could be included in a 3R system. What is the optimal set of experts to use for building knowledge networks? Does that differ for different applications? The Noisy-OR combination method (and most others) assumes that the experts are independent of each other, yet many potentially useful sources of knowledge exhibit complex dependencies; should selection of experts be made in light of this constraint? Many sorts of inference, ranging from logical entailment to information theoretic, statistical or heuristic might be productively included in a 3R system; what is the optimal set of reasoning experts to use? Finally, while the example of biological validation of several hypotheses generated through the use of the system provides some evidence that the system is of genuine value to biological data analysts, the question of how best to evaluate 3R systems remains open. Perhaps the “insight-based” evaluation methodology previously described for scientific visualization systems [127], [128] could be modified to evaluate 3R systems as well. The Hanalyzer, including the experts and the Cytoscape plugin for visualization is available as open source software via SourceForge at hanalyzer. sourceforge. net. The extracted assertions from the OpenDMAP text mining experts are available as supplementary materials associated with [4]; the links from the ACF expert are available as supplementary materials associated with [32].
summary: Recent technology has made it possible to do experiments that show hundreds or even thousands of genes that play a role in a disease or other biological phenomena. Interpreting these experimental results in the light of everything that has ever been published about any of those genes is often overwhelming, and the failure to take advantage of all prior knowledge may impede biomedical research. The computer program described in this paper “reads” the biomedical literature and molecular biology databases, “reasons” about what all that information means to this experiment, and “reports” on its findings in a way that makes digesting all of this information far more efficient than ever before possible. Analysis of a large, complex dataset with this tool led rapidly to the creation of a novel hypothesis about the role of several genes in the development of the tongue, which was then confirmed experimentally.
keywords: computational biology/literature analysis
  genetics and genomics/gene discovery
  developmental biology/morphogenesis and cell biology
  genetics and genomics/bioinformatics
  genetics and genomics/gene function
  computational biology/genomics
  developmental biology/molecular development
  developmental biology/developmental molecular mechanisms
  computational biology/systems biology
title: Biomedical Discovery Acceleration, with Applications to Craniofacial
                      Development