A Survey of Data-Intensive Scientific Workflow Management <s> Galaxy <s> BackgroundWidespread adoption of high-throughput sequencing has greatly increased the scale and sophistication of computational infrastructure needed to perform genomic research. An alternative to building and maintaining local infrastructure is “cloud computing”, which, in principle, offers on demand access to flexible computational infrastructure. However, cloud computing resources are not yet suitable for immediate “as is” use by experimental biologists.ResultsWe present a cloud resource management system that makes it possible for individual researchers to compose and control an arbitrarily sized compute cluster on Amazon’s EC2 cloud infrastructure without any informatics requirements. Within this system, an entire suite of biological tools packaged by the NERC Bio-Linux team (http://nebc.nerc.ac.uk/tools/bio-linux) is available for immediate consumption. The provided solution makes it possible, using only a web browser, to create a completely configured compute cluster ready to perform analysis in less than five minutes. Moreover, we provide an automated method for building custom deployments of cloud resources. This approach promotes reproducibility of results and, if desired, allows individuals and labs to add or customize an otherwise available cloud system to better meet their needs.ConclusionsThe expected knowledge and associated effort with deploying a compute cluster in the Amazon EC2 cloud is not trivial. The solution presented in this paper eliminates these barriers, making it possible for researchers to deploy exactly the amount of computing power they need, combined with a wealth of existing analysis software, to handle the ongoing data deluge. <s> BIB001 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Galaxy <s> High throughput sequencing assays have given rise to the field of genomics and transformed biomedical research into a computational science. Due to the large size of genomics datasets, high-performance computing is essential for analysis. Galaxy (http://galaxyproject.org) is a popular Web-based platform that can be used for all facets of genomic analyses, including data retrieval and integration, multi-step analysis, repeated analyses via workflows, visualization, collaboration, and publication. This paper describes Galaxy and discusses four lessons learned from the development of Galaxy. First, Galaxy uses open, extensible frameworks so that it can be adapted to new technologies as they become available. Second, by leveraging Web technologies, Galaxy makes genomics tools accessible to everyone and provides a common platform for collaboration. Third, Galaxy fosters community amongst both developers and users and encourages each community to adapt and extend Galaxy to meet their needs. Finally, Galaxy software development and genomic research are closely coupled, and challenges encountered during genomic research drive Galaxy development. <s> BIB002 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Galaxy <s> Galaxy is an open source web based framework for data intensive biomedical research. It is an environment for interactive analysis that transparently tracks the details of analyses, a workflow system for convenient reuse, data management, sharing and publishing of the results. In this paper we describe extending the Galaxy framework to the GARUDA Grid computing infrastructure for running bioinformatics applications on various grid. This involves developing a grid job adaptor for the Galaxy workflow. GARUDA grid is an aggregation of heterogeneous resources and advanced capabilities for scientific applications. Grid way is the met scheduler for running the jobs on the Garuda grid. Here we present the integration of Galaxy workflow with the Grid way middleware by developing a grid job runner to interface. This grid enabled galaxy facilitates computational biologists to perform complex problems on the grid environment through a web browser. <s> BIB003 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Galaxy <s> Cloud computing is attracting increasing attention as a means of providing users with fast provisioning of computational and storage resources, elastic scaling, and payas-you-go pricing. The integration of scientific workflows and Cloud computing has the potential to significantly improve resource utilization, processing speed, and user experience. This paper proposes a novel approach for deploying bioinformatics workflows in Cloud environments using Galaxy, a platform for scientific workflows, and Globus Provision, a tool for deploying distributed computing clusters on Amazon EC2. Collectively this combination of tools provides an easy to use, high performance and scalable workflow environment that addresses the needs of data-intensive applications through dynamic cluster configuration, automatic user-defined node provisioning, high speed data transfer, and automated deployment and configuration of domain-specific software. To demonstrate how this approach can be used in practice we present a domain-specific workflow use case and performance evaluation. <s> BIB004 </s> A Survey of Data-Intensive Scientific Workflow Management <s> Galaxy <s> The Taverna workflow tool suite (http://www.taverna.org.uk) is designed to combine distributed Web Services and/or local tools into complex analysis pipelines. These pipelines can be executed on local desktop machines or through larger infrastructure (such as supercomputers, Grids or cloud environments), using the Taverna Server. In bioinformatics, Taverna workflows are typically used in the areas of high-throughput omics analyses (for example, proteomics or transcriptomics), or for evidence gathering methods involving text mining or data mining. Through Taverna, scientists have access to several thousand different tools and resources that are freely available from a large range of life science institutions. Once constructed, the workflows are reusable, executable bioinformatics protocols that can be shared, reused and repurposed. A repository of public workflows is available at http://www.myexperiment.org. This article provides an update to the Taverna tool suite, highlighting new features and developments in the workbench and the Taverna Server. <s> BIB005
Galaxy is a web-based SWfMS for genomic research. In the presentation layer, Galaxy provides a GUI for designing scientific workflows through browsers. It can be installed in a public web server (https://usegalaxy.org/) or a private server to address specific needs. In the user services layer, users can upload data from a user's computer or online resources and share workflow information including workflows, workflow description information, workflow input data and workflow provenance data in a public web site. Moreover, users can import workflows from "myExperiment" BIB005 social network BIB002 . In the WEP generation layer, Galaxy manages the dependencies between each activity for workflow parallelization. In the WEP execution layer, Galaxy generates concrete tasks for each activity, puts the tasks in a queue to be submitted, and monitors the task status (in queue, running or completion) BIB003 . Through this mechanism, Galaxy exploits dynamic scheduling to dispatch executable tasks. Galaxy uses Gridway to execute tasks in the Grid. Gridway manages a task queue and the tasks in a queue are executed in an available computing node that is selected according to a greedy approach, i.e. requests are sent to all the available computing nodes while the node that has minimum response time is selected  . In the infrastructure layer, Galaxy can exploit Globus BIB004 and CloudMan BIB001 to achieve dynamic computing and storage provisioning such as dynamic VM inserting and removing and shared-disk file system construction across computing nodes. Galaxy is adapted to cloud environment by CloudMan BIB001 middlewear, which can create Amazon EC2 clusters based on a Bio-Linux machine image, dynamically change cluster size and attach S3 storage to the clusters.