We introduce an approach to discovery informatics that uses so called knowledge graphs as the essential representation structure. Knowledge graph is an umbrella term that subsumes various approaches to tractable representation of large volumes of loosely structured knowledge in a graph form. It has been used primarily in the Web and Linked Open Data contexts, but is applicable to any other area dealing with knowledge representation. In the perspective of our approach motivated by the challenges of discovery informatics, knowledge graphs correspond to hypotheses. We present a framework for formalising so called hypothesis virtues within knowledge graphs. The framework is based on a classic work in philosophy of science, and naturally progresses from mostly informative foundational notions to actionable specifications of measures corresponding to particular virtues. These measures can consequently be used to determine refined sub-sets of knowledge graphs that have large relative potential for making discoveries. We validate the proposed framework by experiments in literature-based discovery. The experiments have demonstrated the utility of our work and its superiority w.r.t. related approaches.
Research of ways how discoveries can be automated or facilitated by machines dates back to the dawn of the digital computer era. The work @cite_36 provides a comprehensive analysis of the discovery process operationalised as creative problem solving. It reviews several classic machine discovery systems and the heuristics used by them, and also mentions several properties of worthy discoveries like novelty and value. A more recent related work @cite_33 reviews the major approaches to studying the process of scientific discovery, provides another survey of automated discovery systems and analyses additional features of relevant discoveries, such as surprise. The works @cite_43 @cite_30 review still more machine discovery systems and heuristics, and identify features like refutability and simplicity as essential to discoveries. One of the most recent and relevant works from this area is @cite_48 . It builds on @cite_36 @cite_33 and introduces formalisations of several discovery features. In particular, it models novelty and value using metric spaces, and surprise using Bayesian probabilities.
@cite_30 @cite_33 @cite_36 @cite_48 @cite_43
2005197686 2093367122 2059362837 2109154616
Abstract In this paper we track the development of research in empirical discovery. We focus on four machine discovery systems that share a number of features: the use of data-driven heuristics to constrain the search for numeric laws; a reliance on theoretical terms; and the recursive application of a few general discovery methods. We examine each system in light of the innovations it introduced over its predecessors, providing some insight into the conceptual progress that has occurred in machine discovery. Finally, we reexamine this research from the perspectives of the history and philosophy of science. Abstract In this paper, we review AI research on computational discovery and its recent application to the discovery of new scientific knowledge. We characterize five historical stages of the scientific discovery process, which we use as an organizational framework in describing applications. We also identify five distinct steps during which developers or users can influence the behavior of a computational discovery system. Rather than criticizing such intervention, as done in the past, we recommend it as the preferred approach to using discovery software. As evidence for the advantages of such human–computer cooperation, we report seven examples of novel, computer-aided discoveries that have appeared in the scientific literature. We consider briefly the role that humans played in each case, then examine one such interaction in more detail. We close by recommending that future systems provide more explicit support for human intervention in the discovery process. We present techniques for gathering data that expose errors of automatic predictive models. In certain common settings, traditional methods for evaluating predictive models tend to miss rare but important errors—most importantly, cases for which the model is confident of its prediction (but wrong). In this article, we present a system that, in a game-like setting, asks humans to identify cases that will cause the predictive model-based system to fail. Such techniques are valuable in discovering problematic cases that may not reveal themselves during the normal operation of the system and may include cases that are rare but catastrophic. We describe the design of the system, including design iterations that did not quite work. In particular, the system incentivizes humans to provide examples that are difficult for the model to handle by providing a reward proportional to the magnitude of the predictive model's error. The humans are asked to “Beat the Machine” and find cases where the automatic model (“the Machine”) is wrong. Experiments show that the humans using Beat the Machine identify more errors than do traditional techniques for discovering errors in predictive models, and, indeed, they identify many more errors where the machine is (wrongly) confident it is correct. Furthermore, those cases the humans identify seem to be not simply outliers, but coherent areas missed completely by the model. Beat the Machine identifies the “unknown unknowns.” Beat the Machine has been deployed at an industrial scale by several companies. The main impact has been that firms are changing their perspective on and practice of evaluating predictive models. “There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.” --Donald Rumsfeld We propose a new unsupervised learning technique for extracting information from large text collections. We model documents as if they were generated by a two-stage stochastic process. Each author is represented by a probability distribution over topics, and each topic is represented as a probability distribution over words for that topic. The words in a multi-author paper are assumed to be the result of a mixture of each authors' topic mixture. The topic-word and author-topic distributions are learned from data in an unsupervised manner using a Markov chain Monte Carlo algorithm. We apply the methodology to a large corpus of 160,000 abstracts and 85,000 authors from the well-known CiteSeer digital library, and learn a model with 300 topics. We discuss in detail the interpretation of the results discovered by the system including specific topic and author models, ranking of authors by topic and topics by author, significant trends in the computer science literature between 1990 and 2002, parsing of abstracts by topics and authors and detection of unusual papers by specific authors. An online query interface to the model is also discussed that allows interactive exploration of author-topic models for corpora such as CiteSeer.