The act of sharing scientific knowledge is rapidly evolving away from traditional articles and presentations to the delivery of executable objects that integrate the data and computational details (e.g., scripts and workflows) upon which the findings rely. This envisioned coupling of data and process is essential to advancing science but faces technical and institutional barriers. The Whole Tale project aims to address these barriers by connecting computational, data-intensive research efforts with the larger research process—transforming the knowledge discovery and dissemination process into one where data products are united with research articles to create “living publications” or tales. The Whole Tale focuses on the full spectrum of science, empowering users in the long tail of science, and power users with demands for access to big data and compute resources. We report here on the design, architecture, and implementation of the Whole Tale environment.
Science gateways allow users to conduct (generally domain-specific) analyses that exploit advanced computing infrastructure. They provide intuitive user interfaces that abstract the complexities of submitting jobs via queue submission systems or instantiating virtual computing environments for executing GUI-based tools @cite_93 . Given the gateway's position at the center of all analysis, it is possible to capture the steps performed by users (see e.g., @cite_4 ). Often these steps are recorded in the form of workflows @cite_45 or in other standard formats @cite_53 . Science gateways are generally focused on a specific domain, and on the analysis of data. Unlike the they do not provide a general model for capturing and sharing computational processes on arbitrary datasets and linking these artifacts with publications.
@cite_93 @cite_45 @cite_4 @cite_53
2131159644 2136482257 2512030163
The HUBzero cyberinfrastructure lets scientific researchers work together online to develop simulation and modeling tools. Other researchers can then access the resulting tools using an ordinary Web browser and launch simulation runs on the national Grid infrastructure, without having to download or compile any code. Increased reliance on computational approaches in the life sciences has revealed grave concerns about how accessible and reproducible computation-reliant results truly are. Galaxy http: usegalaxy.org, an open web-based platform for genomic research, addresses these problems. Galaxy automatically tracks and manages data provenance and provides support for capturing the context and intent of computational methods. Galaxy Pages are interactive, web-based documents that provide users with a medium to communicate a complete computational analysis. This report documents the program and the outcomes of Dagstuhl Seminar 16041 "Reproducibility of Data-Oriented Experiments in e-Science". In many subfields of computer science, experiments play an important role. Besides theoretic properties of algorithms or methods, their effectiveness and performance often can only be validated via experimentation. In most of these cases, the experimental results depend on the input data, settings for input parameters, and potentially on characteristics of the computational environment where the experiments were designed and run. Unfortunately, most computational experiments are specified only informally in papers, where experimental results are briefly described in figure captions; the code that produced the results is seldom available. This has serious implications. Scientific discoveries do not happen in isolation. Important advances are often the result of sequences of smaller, less significant steps. In the absence of results that are fully documented, reproducible, and generalizable, it becomes hard to re-use and extend these results. Besides hindering the ability of others to leverage our work, and consequently limiting the impact of our field, the absence of reproducibility experiments also puts our reputation at stake, since reliability and validity of empiric results are basic scientific principles. This seminar brought together experts from various sub-fields of computer science to create a joint understanding of the problems of reproducibility of experiments, discussing existing solutions and impediments, and proposing ways to overcome current limitations.