A Survey of Data-Intensive Scientific Workflow Management <s> Scientific Workflow Management Systems <s> The problem of systematically capturing and managing provenance for computational tasks has recently received significant attention because of its relevance to a wide range of domains and applications. The authors give an overview of important concepts related to provenance management, so that potential users can make informed decisions when selecting or designing a provenance solution. <s> BIB001 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Scientific Workflow Management Systems <s> In scientific workflows, provenance data helps scientists in understanding, evaluating and reproducing their results. Provenance data generated at runtime can also support workflow steering mechanisms. Steering facilities for workflows is considered a challenge due to its dynamic demands during execution. To steer, for example, scientists should be able to suspend (or stop) a workflow execution when the approximate solution meets (or deviates) preset criteria. These criteria are commonly evaluated based on provenance data (execution data) and domain-specific data. We claim that the final decision on whether to interfere on the workflow execution may only become feasible when workflows can be steered by scientists using provenance data enriched with domain-specific data. In this paper we propose an approach based on specialized software components, named Data Extractor (DE), to acquire domain-specific data from data files produced during a scientific workflow execution. DE gathers domain-specific data from produced data files and associates it to existing provenance data on the provenance repository. We have evaluated the proposed approach using a real bioinformatics workflow for comparative genomics executed in SciCumulus cloud workflow parallel engine. <s> BIB002 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Scientific Workflow Management Systems <s> Large-scale scientific computations are often organized as a composition of many computational tasks linked through data flow. The data that flows along this manytask computing often moves from a desktop to a high-performance environment and to a visualization environment. Keeping track of this data flow is a challenge to provenance support in high-performance Scientific Workflow Management Systems. After the completion of a computational scientific experiment, a scientist has to manually select and analyze its staged-out data, for instance, by checking inputs and outputs along computational tasks that were part of the experiment. In this paper, we present a provenance management system that describes the production and consumption relationships between data artifacts, such as files, and the computational tasks that compose the experiment. We propose a query interface that allows for scientists to browse provenance data and select the output they want to visualize using browsers or a high-resolution tiled display. <s> BIB003 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Scientific Workflow Management Systems <s> Scientific workflows are commonly used to model and execute large-scale scientific experiments. They represent key resources for scientists and are enacted and managed by Scientific Workflow Management Systems (SWfMS). Each SWfMS has its particular approach to execute workflows and to capture and manage their provenance data. Due to the large scale of experiments, it may be unviable to analyze provenance data only after the end of the execution. A single experiment may demand weeks to run, even in high performance computing environments. Thus scientists need to monitor the experiment during its execution, and this can be done through provenance data. Runtime provenance analysis allows for scientists to monitor workflow execution and to take actions before the end of it (i.e. workflow steering). This provenance data can also be used to fine-tune the parallel execution of the workflow dynamically. We use the PROV data model as a basic framework for modeling and providing runtime provenance as a database that can be queried even during the execution. This database is agnostic of SWfMS and workflow engine. We show the benefits of representing and sharing runtime provenance data for improving the experiment management as well as the analysis of the scientific data. <s> BIB004 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Scientific Workflow Management Systems <s> This chapter introduces the data-driven workflow concept supported by the WS-PGRADE/gUSE system. Workflow management systems were investigated by Workflow Management Coalition, among others in aspects of implemented data flow structures, and several workflow patterns are identified as commonly used and meaningful workflow structures. The workflow concept of gUSE is shown by introducing the supported data patterns and illustrating their creation in the system. Moreover, the possibilities of utilizing parallelization techniques are described, and then the different views of a workflow design and management are described covering the whole lifecycle of a workflow development. Finally, more complex composition of patterns and their creation techniques are shown. <s> BIB005
A Workflow Management System (WfMS) is a system that defines, creates, and manages the execution of workflows. A WfMS is able to interpret the workflow process definition typically in the context of business applications. A SWfMS is a WfMS that handles and manages scientific workflow execution. It is powerful tool to execute workflows in a scientific workflow engine, which is a software service that provides the runtime environment for workflow execution BIB005 . In order to execute a scientific workflow in a given environment, a SWfMS typically generates a Workflow Execution Plan (WEP), which is a program that captures optimization decisions and execution directives, typically the result of compiling and optimizing a workflow, before execution. To support scientific workflow analysis, SWfMS should support additional functionality such as workflow provenance. Workflow provenance may be as (or more) important as the scientific experiment itself BIB001 . Provenance is the metadata that captures the derivation history of a dataset, including the original data sources, intermediate datasets, and the workflow computational steps that were applied to produce this dataset BIB004  BIB002 BIB003 . Provenance data is used for workflow analysis and workflow reproducibility.