A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> A representation and interpretation of the area under a receiver operating characteristic (ROC) curve obtained by the "rating" method, or by mathematical predictions based on patient characteristics, is presented. It is shown that in such a setting the area represents the probability that a randomly chosen diseased subject is (correctly) rated or ranked with greater suspicion than a randomly chosen non-diseased subject. Moreover, this probability of a correct ranking is the same quantity that is estimated by the already well-studied nonparametric Wilcoxon statistic. These two relationships are exploited to (a) provide rapid closed-form expressions for the approximate magnitude of the sampling variability, i.e., standard error that one uses to accompany the area under a smoothed ROC curve, (b) guide in determining the size of the sample required to provide a sufficiently reliable estimate of this area, and (c) determine how large sample sizes should be to ensure that one can statistically detect difference... <s> BIB001 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> Receiver operating characteristic (ROC) curves are frequently used in biomedical informatics research to evaluate classification and prediction models for decision support, diagnosis, and prognosis. ROC analysis investigates the accuracy of a model's ability to separate positive from negative cases (such as predicting the presence or absence of disease), and the results are independent of the prevalence of positive cases in the study population. It is especially useful in evaluating predictive models or other tests that produce output values over a continuous range, since it captures the trade-off between sensitivity and specificity over that range. There are many ways to conduct an ROC analysis. The best approach depends on the experiment; an inappropriate approach can easily lead to incorrect conclusions. In this article, we review the basic concepts of ROC analysis, illustrate their use with sample calculations, make recommendations drawn from the literature, and list readily available software. <s> BIB002 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> Bardet-Biedl syndrome (BBS: OMIM 209900) is a rare developmental disorder that exhibits significant clinical and genetic heterogeneity. Although modeled initially as a purely recessive trait, recent data have unmasked an oligogenic mode of disease transmission, in which mutations at different BBS loci can interact genetically in some families to cause and/or modify the phenotype. Here, I will review and discuss recent advances in elucidating both genetic and cellular aspects of this phenotype and their potential application in understanding the genetic basis of phenotypic variability and oligogenic inheritance. <s> BIB003 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> Motivation: Gene prioritization aims at identifying the most promising candidate genes among a large pool of candidates—so as to maximize the yield and biological relevance of further downstream validation experiments and functional studies. During the past few years, several gene prioritization tools have been defined, and some of them have been implemented and made available through freely available web tools. In this study, we aim at comparing the predictive performance of eight publicly available prioritization tools on novel data. We have performed an analysis in which 42 recently reported disease-gene associations from literature are used to benchmark these tools before the underlying databases are updated. ::: ::: Results: Cross-validation on retrospective data provides performance estimate likely to be overoptimistic because some of the data sources are contaminated with knowledge from disease-gene association. Our approach mimics a novel discovery more closely and thus provides more realistic performance estimates. There are, however, marked differences, and tools that rely on more advanced data integration schemes appear more powerful. ::: ::: Contact: yves.moreau@esat.kuleuven.be ::: ::: Supplementary information: Supplementary data are available at Bioinformatics online. <s> BIB004 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> Many efforts are still devoted to the discovery of genes involved with specific phenotypes, in particular, diseases. High-throughput techniques are thus applied frequently to detect dozens or even hundreds of candidate genes. However, the experimental validation of many candidates is often an expensive and time-consuming task. Therefore, a great variety of computational approaches has been developed to support the identification of the most promising candidates for follow-up studies. The biomedical knowledge already available about the disease of interest and related genes is commonly exploited to find new gene-disease associations and to prioritize candidates. In this review, we highlight recent methodological advances in this research field of candidate gene prioritization. We focus on approaches that use network information and integrate heterogeneous data sources. Furthermore, we discuss current benchmarking procedures for evaluating and comparing different prioritization methods. <s> BIB005 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> Bioinformatics aids in the understanding of the biological processes of living beings and the genetic architecture of human diseases. The discovery of disease-related genes improves the diagnosis and therapy design for the disease. To save the cost and time involved in the experimental verification of the candidate genes, computational methods are employed for ranking the genes according to their likelihood of being associated with the disease. Only top-ranked genes are then verified experimentally. A variety of methods have been conceived by the researchers for the prioritization of the disease candidate genes, which differ in the data source being used or the scoring function used for ranking the genes. A review of various aspects of computational disease gene prioritization and its research issues is presented in this article. The aspects covered are gene prioritization process, data sources used, types of prioritization methods, and performance assessment methods. This article provides a brief overview and acts as a quick guide for disease gene prioritization. <s> BIB006 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> In order to maximize the use of results from high-throughput experimental studies, e.g. GWAS, for identification and diagnostics of new disease-associated genes, it is important to have properly an ... <s> BIB007 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> Motivation: Genome‐wide association studies have identified thousands of loci associated with human disease, but identifying the causal genes at these loci is often difficult. Several methods prioritize genes most likely to be disease causing through the integration of biological data, including protein‐protein interaction and phenotypic data. Data availability is not the same for all genes however, potentially influencing the performance of these methods. Results: We demonstrate that whilst disease genes tend to be associated with greater numbers of data, this may be at least partially a result of them being better studied. With this observation we develop PhenoRank, which prioritizes disease genes whilst avoiding being biased towards genes with more available data. Bias is avoided by comparing gene scores generated for the query disease against gene scores generated using simulated sets of phenotype terms, which ensures that differences in data availability do not affect the ranking of genes. We demonstrate that whilst existing prioritization methods are biased by data availability, PhenoRank is not similarly biased. Avoiding this bias allows PhenoRank to effectively prioritize genes with fewer available data and improves its overall performance. PhenoRank outperforms three available prioritization methods in cross‐validation (PhenoRank area under receiver operating characteristic curve [AUC]=0.89, DADA AUC = 0.87, EXOMISER AUC = 0.71, PRINCE AUC = 0.83, P < 2.2 × 10−16). Availability and implementation: PhenoRank is freely available for download at https://github.com/alexjcornish/PhenoRank. Supplementary information: Supplementary data are available at Bioinformatics online. <s> BIB008 </s> A Survey of Gene Prioritization Tools for Mendelian and Complex Human Diseases <s> Measures of Performance <s> Motivation Computational gene prioritization can aid in disease gene identification. Here, we propose pBRIT (prioritization using Bayesian Ridge regression and Information Theoretic model), a novel adaptive and scalable prioritization tool, integrating Pubmed abstracts, Gene Ontology, Sequence similarities, Mammalian and Human Phenotype Ontology, Pathway, Interactions, Disease Ontology, Gene Association database and Human Genome Epidemiology database, into the prediction model. We explore and address effects of sparsity and inter‐feature dependencies within annotation sources, and the impact of bias towards specific annotations. Results pBRIT models feature dependencies and sparsity by an Information‐Theoretic (data driven) approach and applies intermediate integration based data fusion. Following the hypothesis that genes underlying similar diseases will share functional and phenotype characteristics, it incorporates Bayesian Ridge regression to learn a linear mapping between functional and phenotype annotations. Genes are prioritized on phenotypic concordance to the training genes. We evaluated pBRIT against nine existing methods, and on over 2000 HPO‐gene associations retrieved after construction of pBRIT data sources. We achieve maximum AUC scores ranging from 0.92 to 0.96 against benchmark datasets and of 0.80 against the time‐stamped HPO entries, indicating good performance with high sensitivity and specificity. Our model shows stable performance with regard to changes in the underlying annotation data, is fast and scalable for implementation in routine pipelines. Availability and implementation http://biomina.be/apps/pbrit/; https://bitbucket.org/medgenua/pbrit. <s> BIB009
Although many evaluations of gene prioritization tools are described in literature BIB004 ,  , BIB007 , no unified approach for evaluation of gene prioritization tool exists. Doncheva et al. BIB005 , and Gill et al. BIB006 provide a good overview of performance measures applicable for evaluation of gene prioritization tools. Optimal validation approach and measure of performance depend on the nature of the disease (monogenic or polygenic) and tolerance to false positives. For monogenic or oligogenic BIB003 diseases by definition just one or few causative genes exist and their experimental validation is time-consuming and laborious. Therefore, in the case when only a few candidates can be tested, suitable performance measures are the averaged number of false candidates appeared above the right one, or success rate, when the correctly identified disease gene was ranked within n top genes. In contrast with monogenic diseases, complex disorders are associated with multiple genes, and the expected number of true associations may vary depending on the disease. Although the mean rank ratio of true positive findings is applicable in this case, it provides little information about the overall distribution of the true positive ranks. For complex disorders, potentially associated genes are not necessarily tested individually but may be analyzed as a group. For some tasks, such as gene set or pathway enrichment analysis, individual gene ranks are not so crucial as the enrichment of true positives in the top of the list. Averaged fold-enrichment BIB005 ,  shows the increase of true positive rate (TPR) in the top of the ranked list compared to the background TPR. The method achieves n/m-fold enrichment on average, if in n% of cases correctly identified disease genes are ranked among the top m% of all candidates  . Besides the TPR, also known as sensitivity, another essential characteristic of performance is specificity, which shows the ability of the method to correctly classify negative exemplars (true negative rate). The relationship between sensitivity and false positive rate (FPR) plotted at variable threshold levels gives a receiver operating characteristic (ROC) curve BIB001 . The cross-validated area under the ROC curve (AUC) BIB002 provides an estimate of overall tool performance BIB007 , BIB008 ,  , BIB009 .