With the ever-growing need of data in HPC applications, the congestion at the I O level becomes critical in super-computers. Architectural enhancement such as burst-buffers and pre-fetching are added to machines, but are not sufficient to prevent congestion. Recent online I O scheduling strategies have been put in place, but they add an additional congestion point and overheads in the computation of applications. In this work, we show how to take advantage of the periodic nature of HPC applications in order to develop efficient periodic scheduling strategies for their I O transfers. Our strategy computes once during the job scheduling phase a pattern where it defines the I O behavior for each application, after which the applications run independently, transferring their I O at the specified times. Our strategy limits the amount of I O congestion at the I O node level and can be easily integrated into current job schedulers. We validate this model through extensive simulations and experiments by comparing it to state-of-the-art online solutions, showing that not only our scheduler has the advantage of being de-centralized and thus overcoming the overhead of online schedulers, but also that it performs better than these solutions, improving the application dilation up to 13 and the maximum system efficiency up to 18 .
The study from @cite_8 offers ways of isolating the performance experienced by applications of one operating system from variations in the I O request stream characteristics of applications of other operating systems. While their solution cannot be applied to HPC systems, the study offers a way of controlling the coarse grain allocation of disk time to the different operating system instances as well as determining the fine-grain interleaving of requests from the corresponding operating systems to the storage system.
@cite_8
2018997957
Virtualized storage systems are required to service concurrently executing workloads, with potentially diverse data delivery requirements, that are running under multiple operating systems. Although a number of algorithms have been developed for I O performance virtualization among operating system (OS) instances and their applications, none results in absolute performance virtualization. By absolute performance virtualization we mean that the performance experienced by applications of one operating system does not suffer due to variations in the I O request stream characteristics of applications of other operating systems. Key requirements of I O performance virtualization are fairness and performance isolation. In this paper, we present a novel virtual I O scheduler (VIOS) that provides absolute performance virtualization by being fair in sharing I O system resources among operating systems and their applications, and provides performance isolation in the face of variations in the characteristics of I O streams. The VIOS controls the coarse grain allocation of disk time to the different operating system instances and is OS independent; optionally, a set of OS-dependent schedulers may determine the fine-grain interleaving of requests from the corresponding operating systems to the storage system.