Architecture and Design of Cray DataWarp
This paper describes the architecture, design, use, and performance of Cray DataWarp. DataWarp is an infrastructure that uses direct attached solid state disk (SSD) storage to provide more cost effective bandwidth than an external parallel file system (PFS) allowing DataWarp to be provisioned for bandwidth and the PFS to be provisioned for capacity and resiliency. Placing this storage between the application and the PFS allows application I/O to be decoupled from PFS I/O and in some cases avoiding it altogether. This reduces the time required for the application to do I/O, while also increasing the overlap of computation with PFS I/O, typically reducing application elapsed time. DataWarp allocates and configures SSD backed storage for jobs and users on demand, providing many of the benefits of both software defined storage and storage virtualization. Keywords Filesystems I/O, DataWarp, SSD