Several scientific and industry applications require High Performance Computing (HPC) resources to process and or simulate complex models. Not long ago, companies, research institutes, and universities used to acquire and maintain on-premise computer clusters; but, recently, cloud computing has emerged as an alternative for a subset of HPC applications. This poses a challenge to end-users, who have to decide where to run their jobs: on local clusters or burst to a remote cloud service provider. While current research on HPC cloud has focused on comparing performance of on-premise clusters against cloud resources, we build on top of existing efforts and introduce an advisory service to help users make this decision considering the trade-offs of resource costs, performance, and availability on hybrid clouds. We evaluated our service using a real test-bed with a seismic processing application based on Full Waveform Inversion; a technique used by geophysicists in the oil & gas industry and earthquake prediction. We also discuss how the advisor can be used for other applications and highlight the main lessons learned constructing this service to reduce costs and turnaround times.
Following a new phase of HPC cloud studies, Mateescu @cite_5 evaluated a set of benchmarks and complex HPC applications on a range of platforms, both in-house and in the cloud. The studies showed cloud effectiveness for such applications mainly in the case of complementing supercomputers. Gupta and Milojicic @cite_18 highlighted that cloud can be suitable for some HPC applications, but not all. The same group of authors @cite_17 performed a set of experiments using benchmarks and complex HPC applications on various platforms, including supercomputers and clouds, to answer the question why and who should choose cloud for HPC, for what applications, and how should cloud be used for HPC?''.
@cite_5 @cite_18 @cite_17
2082245129 2088445131 2624365150
We introduce a hybrid High Performance Computing (HPC) infrastructure architecture that provides predictable execution of scientific applications, and scales from a single resource to multiple resources, with different ownership, policy, and geographic locations. We identify three paradigms in the evolution of HPC and high-throughput computing: owner-centric HPC (traditional), Grid computing, and Cloud computing. After analyzing the synergies among HPC, Grid and Cloud computing, we argue for an architecture that combines the benefits of these technologies. We call the building block of this architecture, Elastic Cluster. We describe the concept of Elastic Cluster and show how it can be used to achieve effective and predictable execution of HPC workloads. Then we discuss implementation aspects, and propose a new distributed information system design that combines features of distributed hash tables and relational databases. HPC applications are increasingly being used in academia and laboratories for scientific research and in industries for business and analytics. Cloud computing offers the benefits of virtualization, elasticity of resources and elimination of cluster setup cost and time to HPC applications users. However, poor network performance, performance variation and OS noise are some of the challenges for execution of HPC applications on Cloud. In this paper, we propose that Cloud can be viable platform for some HPC applications depending upon application characteristics such as communication volume and pattern and sensitivity to OS noise and scale. We present an evaluation of the performance and cost tradeoffs of HPC applications on a range of platforms varying from Cloud (with and without virtualization) to HPC-optimized cluster. Our results show that Cloud is viable platform for some applications, specifically, non communicationintensive applications such as embarrassingly parallel and tree-structured computations up to high processor count and for communication-intensive applications up to low processor count.