Bioinformatics software plays a very important role in making critical decisions within many areas including medicine and health care. However, most of the research is directed towards developing tools, and little time and effort is spent on testing the software to assure its quality. In testing, a test oracle is used to determine whether a test is passed or failed during testing, and unfortunately, for much of bioinformatics software, the exact expected outcomes are not well defined. Thus, the main challenge associated with conducting systematic testing on bioinformatics software is the oracle problem. Metamorphic testing (MT) is a technique used to test programs that face the oracle problem. MT uses metamorphic relations (MRs) to determine whether a test has passed or failed and specifies how the output should change according to a specific change made to the input. In this work, we use MT to test LingPipe, a tool for processing text using computational linguistics, often used in bioinformatics for bio-entity recognition from biomedical literature. First, we identify a set of MRs for testing any bio-entity recognition program. Then we develop a set of test cases that can be used to test LingPipe's bio-entity recognition functionality using these MRs. To evaluate the effectiveness of this testing process, we automatically generate a set of faulty versions of LingPipe. According to our analysis of the experimental results, we observe that our MRs can detect the majority of these faulty versions, which shows the utility of this testing technique for quality assurance of bioinformatics software.
One of the biggest challenges for conducting automated systematic testing on scientific programs is the oracle problem. This challenge is especially prevalent in the field of bioinformatics due to the inherent complexity of these programs. In this paper, we explore two approaches: pseudo-oracles and metamorphic testing for conducting automated systematic testing of bioinformatics programs. We use BBMap: an open source genome alignment tool as the system under test to evaluate the effectiveness of the above two approaches for identifying subtle faults. Our results show that the pseudo-oracle testing does not provide a consistent basis for fault-detection. This is mainly due to not having a consistent rate of agreement between the outputs of the system under test and the pseudo-oracle. On the other hand, metamorphic testing detects a majority of subtle faults, suggesting that it is more effective in identifying these types of faults. The research reported in this paper is motivated by the need to validate models of disease spread. To date, reasoned confidence in the results of these models is complicated by the increasing complexity of the models and by their desired predictive use. As part of an overall approach to verification and validation of disease spread models, we investigate metamorphic testing. In this paper, we present early results of initial metamorphic testing on agent-based epidemiological models. Compartmental models in epidemiology are widely used as a means to model disease spread mechanisms and understand how one can best control the disease in case an outbreak of a widespread epidemic occurs. However, a significant challenge within the community is in the development of approaches that can be used to rigorously verify and validate these models. In this paper, we present an approach to quantify and verify the behavioral properties of compartmental epidemiological models under several common modeling scenarios including: birth death rates and multi-host pathogen species. We build a workflow that uses metamorphic testing, novel visualization tools and model checking to gain insights into the functionality of compartmental epidemiological models. Our initial results indicate that metamorphic testing can be used to verify the implementation of these models and provide insights into special conditions where these mathematical models may fail. The visualization front-end allows the end-user to scan through a variety of parameters commonly used in these models to elucidate the conditions under which an epidemic can occur. Furthermore, specifying these models using a process algebra allows one to automatically construct behavioral properties that can be rigorously verified using model checking. Together, our approach allows for detecting implementation errors as well as handling conditions under which compartmental epidemiological models may fail to provide insights into disease spread dynamics.