Increasing adoption of scientific workflows in the community has urged for the development of multi-tenant platforms that provides these workflows execution as a service. Workflow as a Service (WaaS) concept has been brought up by researchers to address the future design of Workflow Management Systems (WMS) that can serve a large number of users from a single point of service. This platform differs from a traditional WMS in handling the workload of workflows at runtime. A traditional WMS is usually designed to execute a single workflow in a dedicated process while the WaaS platforms enhance the process by exploiting multiple workflows execution in a resource-sharing environment model. In this paper, we explore a novel resource-sharing policy to improve system utilization and to fulfill various Quality of Service (QoS) requirements from multiple users. We propose an Elastic Budget-constrained resource Provisioning and Scheduling algorithm for Multiple workflows designed for WaaS platforms that is able to reduce the computational overhead by encouraging resource-sharing policy to minimize workflows' makespan while meeting user-defined budget. Our experiments show that the EBPSM algorithm is able to utilize the resource-sharing policy to achieve higher performance in terms of minimizing the makespan compared to the state-of-the-art budget-constraint scheduling algorithm.
One of the challenges of executing scientific workflows in the clouds is related to the data-locality. Scientific workflows are known as data-intensive applications that involve a vast amount of data to be processed. Therefore, the communication overhead for transferring the data between tasks' execution may take a considerable amount of time that might impact the overall makespan. A work by Stavrinides and Karatza @cite_28 shows that the use of distributed in-memory storage to store the datasets locally for tasks' execution can reduce the communication overhead. Our work is similar regarding the data-locality policy to minimize the data transfer between tasks' execution. However, we use the cached datasets in VMs local storage from tasks' execution to endorse the locality of data. We enhance this policy so that the algorithm can intelligently decide which task to be scheduled in particular VMs that can provide the minimum execution time given the available cached datasets.
@cite_28
2546495709
Abstract As large-scale distributed systems gain momentum, the scheduling of workflow applications with multiple requirements in such computing platforms has become a crucial area of research. In this paper, we investigate the workflow scheduling problem in large-scale distributed systems, from the Quality of Service (QoS) and data locality perspectives. We present a scheduling approach, considering two models of synchronization for the tasks in a workflow application: (a) communication through the network and (b) communication through temporary files. Specifically, we investigate via simulation the performance of a heterogeneous distributed system, where multiple soft real-time workflow applications arrive dynamically. The applications are scheduled under various tardiness bounds, taking into account the communication cost in the first case study and the I O cost and data locality in the second. The simulation results provide useful insights into the impact of tardiness bound and data locality on the system performance.