Cloud Computing has emerged as a successful computing paradigm for efficiently utilizing managed compute infrastructure such as high speed rack-mounted servers, connected with high speed networking, and reliable storage. Usually such infrastructure is dedicated, physically secured and has reliable power and networking infrastructure. However, much of our idle compute capacity is present in unmanaged infrastructure like idle desktops, lab machines, physically distant server machines, and laptops. We present a scheme to utilize this idle compute capacity on a best-effort basis and provide high availability even in face of failure of individual components or facilities. We run virtual machines on the commodity infrastructure and present a cloud interface to our end users. The primary challenge is to maintain availability in the presence of node failures, network failures, and power failures. We run multiple copies of a Virtual Machine (VM) redundantly on geographically dispersed physical machines to achieve availability. If one of the running copies of a VM fails, we seamlessly switchover to another running copy. We use Virtual Machine Record Replay capability to implement this redundancy and switchover. In current progress, we have implemented VM Record Replay for uniprocessor machines over Linux KVM and are currently working on VM Record Replay on shared-memory multiprocessor machines. We report initial experimental results based on our implementation.
Previous efforts on utilizing idle compute capacity include SETI@Home @cite_1 , Folding@Home @cite_7 , BOINC project @cite_5 , etc. Our work has a similar philosophy. The difference is in the level of abstraction. These previous efforts require that the programs be written to a specific programming model, and then provide a middleware which needs to be installed in all participating host machines. The middleware then coordinates and schedules the client programs. In contrast, our abstraction is more general (and often more powerful) than the middleware approach. Our computation units are VMs, allowing a client full freedom to run her favourite OS and applications on the participating hosts, without compromising security and reliability. To provide reliability, the middleware-based approaches usually constrain the programming model for easy restartability. In contrast, we allow a completely flexible programming model and provide reliability through efficient recording and replaying.
@cite_5 @cite_1 @cite_7
2142863519 2103363198 1673352269
BOINC (Berkeley Open Infrastructure for Network Computing) is a software system that makes it easy for scientists to create and operate public-resource computing projects. It supports diverse applications, including those with large storage or communication requirements. PC owners can participate in multiple BOINC projects, and can specify how their resources are allocated among these projects. We describe the goals of BOINC, the design issues that we confronted, and our solutions to these problems. Millions of computer owners worldwide contribute computer time to the search for extraterrestrial intelligence, performing the largest computation ever. For decades, researchers have been applying computer simulation to address problems in biology. However, many of these "grand challenges" in computational biology, such as simulating how proteins fold, remained unsolved due to their great complexity. Indeed, even to simulate the fastest folding protein would require decades on the fastest modern CPUs. Here, we review novel methods to fundamentally speed such previously intractable problems using a new computational paradigm: distributed computing. By efficiently harnessing tens of thousands of computers throughout the world, we have been able to break previous computational barriers. However, distributed computing brings new challenges, such as how to efficiently divide a complex calculation of many PCs that are connected by relatively slow networking. Moreover, even if the challenge of accurately reproducing reality can be conquered, a new challenge emerges: how can we take the results of these simulations (typically tens to hundreds of gigabytes of raw data) and gain some insight into the questions at hand. This challenge of the analysis of the sea of data resulting from large-scale simulation will likely remain for decades to come.