A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> From its start using supercomputers, scientific computing constantly evolved to the next levels such as cluster computing, meta-computing, or computational Grids. Today, Cloud Computing is emerging as the paradigm for the next generation of large-scale scientific computing, eliminating the need of hosting expensive computing hardware. Scientists still have their Grid environments in place and can benefit from extending them by leased Cloud resources whenever needed. This paradigm shift opens new problems that need to be analyzed, such as integration of this new resource class into existing environments, applications on the resources and security. The virtualization overheads for deployment and starting of a virtual machine image are new factors which will need to be considered when choosing scheduling mechanisms. In this paper we investigate the usability of compute Clouds to extend a Grid workflow middleware and show on a real implementation that this can speed up executions of scientific workflows. <s> BIB001 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> BackgroundWidespread adoption of high-throughput sequencing has greatly increased the scale and sophistication of computational infrastructure needed to perform genomic research. An alternative to building and maintaining local infrastructure is “cloud computing”, which, in principle, offers on demand access to flexible computational infrastructure. However, cloud computing resources are not yet suitable for immediate “as is” use by experimental biologists.ResultsWe present a cloud resource management system that makes it possible for individual researchers to compose and control an arbitrarily sized compute cluster on Amazon’s EC2 cloud infrastructure without any informatics requirements. Within this system, an entire suite of biological tools packaged by the NERC Bio-Linux team (http://nebc.nerc.ac.uk/tools/bio-linux) is available for immediate consumption. The provided solution makes it possible, using only a web browser, to create a completely configured compute cluster ready to perform analysis in less than five minutes. Moreover, we provide an automated method for building custom deployments of cloud resources. This approach promotes reproducibility of results and, if desired, allows individuals and labs to add or customize an otherwise available cloud system to better meet their needs.ConclusionsThe expected knowledge and associated effort with deploying a compute cluster in the Amazon EC2 cloud is not trivial. The solution presented in this paper eliminates these barriers, making it possible for researchers to deploy exactly the amount of computing power they need, combined with a wealth of existing analysis software, to handle the ongoing data deluge. <s> BIB002 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> We present GroudSim, a Grid and Cloud simulation toolkit for scientific applications based on a scalable simulation-independent discrete-event core. GroudSim provides a comprehensive set of features for complex simulation scenarios from simple job executions on leased computing resources to calculation of costs, and background load on resources. Simulations can be parameterised and are easily extendable by probability distribution packages for failures which normally occur in complex environments. Experimental results demonstrate the improved scalability of GroudSim compared to a related process-based approach. <s> BIB003 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> In this paper we present the Coaster System. It is an automatically-deployed node provisioning (Pilot Job) system for grids, clouds, and ad-hoc desktop-computer networks supporting file staging, on-demand opportunistic multi-node allocation, remote logging, and remote monitoring. The Coaster System has been previously shown to work at scales of thousands of cores. It has been used since 2009 for applications in fields that include biochemistry, earth systems science, energy modeling, and neuroscience. The system has been used successfully on the Open Science Grid, the Tera Grid, supercomputers (IBM Blue Gene/P, Cray XT and XE systems, and Sun Constellation), a number of smaller clusters, and three cloud infrastructures (BioNimbus, Future Grid and Amazon EC2). <s> BIB004 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> Cloud computing systems are becoming an important platform for science applications. Infrastructure as a Service (IaaS) clouds provide the capability to provision virtual machines (VMs) on demand with a specific configuration of hardware resources, but they do not provide functionality for managing those resources once provisioned. In order for such clouds to be used effectively for parallel and distributed scientific applications, tools need to be developed that can help users to deploy their applications in the cloud. This paper describes a system we have developed to provision, configure, and manage clusters of virtual machines. <s> BIB005 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> There is a growing interest in the use of cloud computing for scientific applications, including scientific workflows. Key attractions of the cloud include the pay-as-you-go model and elasticity. While the elasticity offered by clouds can be beneficial for many applications and use-scenarios, it also imposes significant challenges in the development of applications or services. For example, no general framework exists that can enable a scientific workflow to execute in a dynamic fashion, i.e. exploiting elasticity of clouds and automatically allocating and deal locating resources to meet time and/or cost constraints. This paper presents a case-study in creating a dynamic cloud workflow implementation of a scientific application. We work with Mass Matrix, an application which searches proteins and peptides from tandem mass spectrometry data. In order to use cloud resources, we first parallelize the search method used in this algorithm. Next, we create a flexible workflow using the Pegasus Workflow Management System. Finally, we add a new dynamic resource allocation module, which can use fewer or a larger number of resources based on a time constraint specified by the user. We evaluate our implementation using several different datasets, and show that the application scales quite well, and that our dynamic framework is effective in meeting time constraints. <s> BIB006 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> Abstract With the increasing popularity of the Cloud computing, there are more and more requirements for scientiﬁc work–ﬂows to utilize Cloud resources. In this paper, we present our preliminary work and experiences on enabling the interaction between the Kepler scientiﬁc workﬂow system and the Amazon Elastic Compute Cloud (EC2). A set of EC2 actors and Kepler Amazon Machine Images are introduced with the discussion on their different usage modes. Through two bioinformatics usecases, we demonstrate the capability of our work for both Cloud resource coordination and workﬂow execution on virtual Cloud resources. <s> BIB007 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> Large-scale applications expressed as scientific workflows are often grouped into ensembles of inter-related workflows. In this paper, we address a new and important problem concerning the efficient management of such ensembles under budget and deadline constraints on Infrastructure- as-a-Service (IaaS) clouds. We discuss, develop, and assess algorithms based on static and dynamic strategies for both task scheduling and resource provisioning. We perform the evaluation via simulation using a set of scientific workflow ensembles with a broad range of budget and deadline parameters, taking into account uncertainties in task runtime estimations, provisioning delays, and failures. We find that the key factor determining the performance of an algorithm is its ability to decide which workflows in an ensemble to admit or reject for execution. Our results show that an admission procedure based on workflow structure and estimates of task runtimes can significantly improve the quality of solutions. <s> BIB008 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> The Taverna workflow tool suite (http://www.taverna.org.uk) is designed to combine distributed Web Services and/or local tools into complex analysis pipelines. These pipelines can be executed on local desktop machines or through larger infrastructure (such as supercomputers, Grids or cloud environments), using the Taverna Server. In bioinformatics, Taverna workflows are typically used in the areas of high-throughput omics analyses (for example, proteomics or transcriptomics), or for evidence gathering methods involving text mining or data mining. Through Taverna, scientists have access to several thousand different tools and resources that are freely available from a large range of life science institutions. Once constructed, the workflows are reusable, executable bioinformatics protocols that can be shared, reused and repurposed. A repository of public workflows is available at http://www.myexperiment.org. This article provides an update to the Taverna tool suite, highlighting new features and developments in the workbench and the Taverna Server. <s> BIB009 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Parallelization in Single Site Cloud <s> Public commercial clouds emerged as new and attractive resource provisioning option for scientific computing. This new alternative raises new challenges for users of such clouds, since optimizing the completion time of scientific applications might substantially increase the monetary cost of leasing cloud resources. In this paper, we first propose a set of basic rescheduling operations covering a broad set of scenarios for reducing the costs of running scientific workflows in clouds. Based on them, we design two heuristic scheduling algorithms. The first algorithm aims at reducing the cost of resource provisioning while still attaining the optimal make span. The second algorithm further reduces the costs to meet a budget constraint with a small increase in the make span. The experiments conducted using real-world and synthetic workflow applications demonstrate important benefits compared to related state-of-the-art approaches. <s> BIB010
In a single site cloud environment, SWfMSs can be directly installed in the VMs and exploit services deployed in the cloud  BIB009 . Existing parallelization techniques, e.g. parallelism techniques (see Section 3.1), scheduling techniques (see Section 3.2), existing execution models in grid  , can be used to execute a data-intensive scientific workflow in the environment. SWfMSs can exploit some middleware to create or remove VMs and enable the communication between VMs in order to execute data-intensive scientific workflows in the cloud BIB002 BIB004 BIB007 . Since we can create VMs dynamically in the cloud environment, some methods  BIB010 BIB005 BIB008 BIB006 BIB003 BIB001 are proposed to make a trade-off between parallelization and other constraints, e.g. budget limit for data-intensive scientific workflow execution. Since high parallelization degree can lead to less execution time, SWfMSs can dynamically create new VMs in order to reduce execution time under financial cost constraint. But if the estimated financial cost of workflow execution with current number of VMs exceeds the financial cost constraint, SWfMSs can remove some VMs to reduce financial cost.