Abstract In the coming era of exascale supercomputing, in-situ visualization will be a crucial approach for reducing the output data size. A problem of in-situ visualization is that it loses interactivity if a steering method is not adopted. In this paper, we propose a new method for the interactive analysis of in-situ visualization images produced by a batch simulation job. A key idea is to apply numerous (thousands to millions) in-situ visualizations simultaneously. The viewer then analyzes the image database interactively during postprocessing. If each movie can be compressed to 100 MB, one million movies will only require 100 TB, which is smaller than the size of the raw numerical data in exascale supercomputing. We performed a feasibility study using the proposed method. Multiple movie files were produced by a simulation and they were analyzed using a specially designed movie player. The user could change the viewing angle, the visualization method, and the parameters interactively by retrieving an appropriate sequence of images from the movie dataset.
In-situ visualization has a long history. @cite_8 conducted a general review of steering simulation and visualization. Examples of in-situ visualization in peta-scale simulations and their technical challenges (especially those caused by massively parallel processing) are summarized in @cite_3 @cite_2 . They show that in-situ visualization is a promising solution for peta and exascale simulations. A natural extension of the online dynamical control of simulation is the end-to-end approach @cite_12 @cite_10 , where even mesh generation, which is usually conducted in the preprocessing stage, is performed on supercomputers. They developed a steering simulation for seismic wave propagation where the visualization images were shown in real time.
@cite_8 @cite_3 @cite_2 @cite_10 @cite_12
2096945381 1980853116 2141005805 1988100325
The growing power of parallel supercomputers gives scientists the ability to simulate more complex problems at higher fidelity, leading to many high-impact scientific advances. To maximize the utilization of the vast amount of data generated by these simulations, scientists also need scalable solutions for studying their data to different extents and at different abstraction levels. As we move into peta- and exa-scale computing, simply dumping as much raw simulation data as the storage capacity allows for post-processing analysis and visualization is no longer a viable approach. A common practice is to use a separate parallel computer to prepare data for subsequent analysis and visualization. A naive realization of this strategy not only limits the amount of data that can be saved, but also turns I O into a performance bottleneck when using a large parallel system. We conjecture that the most plausible solution for the peta- and exa-scale data problem is to reduce or transform the data in-situ as it is being generated, so the amount of data that must be transferred over the network is kept to a minimum. In this paper, we discuss different approaches to in-situ processing and visualization as well as the results of our preliminary study using large-scale simulation codes on massively parallel supercomputers. Computer simulations are growing in sophistication and producing results of ever greater fidelity. This trend has been enabled by advances in numerical methods and increasing computing power. Yet these advances come with several costs including massive increases in data size, difficulties examining output data, challenges in configuring simulation runs, and difficulty debugging running codes. Interactive visualization tools, like ParaView, have been used for post-processing of simulation results. However, the increasing data sizes, and limited storage and bandwidth make high fidelity post-processing impractical. In situ analysis is recognized as one of the ways to address these challenges. In situ analysis moves some of the post-processing tasks in line with the simulation code thus short circuiting the need to communicate the data between the simulation and analysis via storage. ParaView Catalyst is a data processing and visualization library that enables in situ analysis and visualization. Built on and designed to interoperate with the standard visualization toolkit VTK and the ParaView application, Catalyst enables simulations to intelligently perform analysis, generate relevant output data, and visualize results concurrent with a running simulation. In this paper, we provide an overview of the Catalyst framework and some of the success stories. In situ visualization is clearly a promising solution for ultrascale simulations. We've seen some success in realizing this solution and in ongoing efforts to add support for in situ visualization to open source visualization toolkits such as ParaView and Visit. However, for others to adopt this approach, we need further research and experimental studies to derive a set of guidelines and usable visualization software components. If this research is successful, it will lead to a new visualization and data-understanding infrastructure, potentially change how scientists work, and accelerate scientific discovery. This paper discusses critical issues in realizing in situ visualization and suggest important research directions. As high performance computing approaches exascale, CPU capability far outpaces disk write speed, and in situ visualization becomes an essential part of an analyst's workflow. In this paper, we describe the ParaView Coprocessing Library, a framework for in situ visualization and analysis coprocessing. We describe how coprocessing algorithms (building on many from VTK) can be linked and executed directly from within a scientific simulation or other applications that need visualization and analysis. We also describe how the ParaView Coprocessing Library can write out partially processed, compressed, or extracted data readable by a traditional visualization application for interactive post-processing. Finally, we will demonstrate the library's scalability in a number of real-world scenarios.