Starting from a high-level problem description in terms of partial differential equations using abstract tensor notation, the Chemora framework discretizes, optimizes, and generates complete high performance codes for a wide range of compute architectures. Chemora extends the capabilities of Cactus, facilitating the usage of large-scale CPU GPU systems in an efficient manner for complex applications, without low-level code tuning. Chemora achieves parallelism through MPI and multi-threading, combining OpenMP and CUDA. Optimizations include high-level code transformations, efficient loop traversal strategies, dynamically selected data and instruction cache usage strategies, and JIT compilation of GPU code tailored to the problem characteristics. The discretization is based on higher-order finite differences on multi-block domains. Chemora's capabilities are demonstrated by simulations of black hole collisions. This problem provides an acid test of the framework, as the Einstein equations contain hundreds of variables and thousands of terms.
Jacobsen @cite_43 extended this model by adding inter-node communication via MPI @. They followed the approach described in Micik @cite_42 and overlapped the communication with computations as well as GPU-host with host-host data exchange. However, they did not take advantage of the full-duplex nature of the PCI-Express bus, which would have decreased the time spent for communication. Their computational model also divides the domain along the slowest varying dimension only, and this approach is not suitable for all numerical problems. For example, for large computational domains, the size of the ghost zone becomes noticeable in comparison to the computed part of the domain, and the communication cost becomes larger than the computational cost, which can be observed in the non-linear scaling of their model.
@cite_43 @cite_42
2108266785 2620842051
Modern graphics processing units (GPUs) with many-core architectures have emerged as general-purpose parallel computing platforms that can accelerate simulation science applications tremendously. While multiGPU workstations with several TeraFLOPS of peak computing power are available to accelerate computational problems, larger problems require even more resources. Conventional clusters of central processing units (CPU) are now being augmented with multiple GPUs in each compute-node to tackle large problems. The heterogeneous architecture of a multi-GPU cluster with a deep memory hierarchy creates unique challenges in developing scalable and efficient simulation codes. In this study, we pursue mixed MPI-CUDA implementations and investigate three strategies to probe the efficiency and scalability of incompressible flow computations on the Lincoln Tesla cluster at the National Center for Supercomputing Applications (NCSA). We exploit some of the advanced features of MPI and CUDA programming to overlap both GPU data transfer and MPI communications with computations on the GPU. We sustain approximately 2.4 TeraFLOPS on the 64 nodes of the NCSA Lincoln Tesla cluster using 128 GPUs with a total of 30,720 processing elements. Our results demonstrate that multi-GPU clusters can substantially accelerate computational fluid dynamics (CFD) simulations. A connector for mounting on a vertical main frame has an elongate body portion containing a protector field and an elongate jumper field extending along one side. The connector is mounted on the frame member by a bracket which mounts the connector so that the body portion extends at an angle to the plane of the main frame. With an appropriate angle, a useful space is provided between connectors, for access, while removal and replacement of protector modules, and entire connectors, is readily available. One particularly appropriate angle is about 53 DEG , while another is about 70 DEG , although these can vary. A test field can also extend along one side, adjacent to the jumper field.