The Partitioned Global Address Space (PGAS) programming model strikes a balance between the locality-aware, but explicit, message-passing model and the easy-to-use, but locality-agnostic, shared memory model. However, the PGAS rich memory model comes at a performance cost which can hinder its potential for scalability and performance. To contain this overhead and achieve full performance, compiler optimizations may not be sufficient and manual optimizations are typically added. This, however, can severely limit the productivity advantage. Such optimizations are usually targeted at reducing address translation overheads for shared data structures. This paper proposes a hardware architectural support for PGAS, which allows the processor to efficiently handle shared addresses. This eliminates the need for such hand-tuning, while maintaining the performance and productivity of PGAS languages. We propose to avail this hardware support to compilers by introducing new instructions to efficiently access and traverse the PGAS memory space. A prototype compiler is realized by extending the Berkeley Unified Parallel C (UPC) compiler. It allows unmodified code to use the new instructions without the user intervention, thereby creating a real productive programming environment. Two implementations are realized: the first is implemented using the full system simulator Gem5, which allows the evaluation of the performance gain. The second is implemented using a softcore processor Leon3 on an FPGA to verify the implementability and to parameterize the cost of the new hardware and its instructions. The new instructions show promising results for the NAS Parallel Benchmarks implemented in UPC. A speedup of up to 5.5x is demonstrated for unmodified and unoptimized codes. Unoptimized code performance using this hardware was shown to also surpass the performance of manually optimized code by up to 10 .
Multiple systems have implemented Hardware support for shared memory across a system. For example, the T3D supercomputer used a 'Support Circuitry' chip located between the processor and the local memory @cite_1 ; this chip, on top of providing functionality like message passing and synchronization, allowed the processor to access any memory location across the machine. In @cite_18 is proposed a network engine especially designed for PGAS languages: it allows network communication between nodes by mapping other nodes memory space accross the network and providing a relaxed memory consistency model best suited for PGAS. Results were only presented in terms of read write throughput and transaction rates as no PGAS applications or benchmarks were tested. This approach is complementary to our work, as it focused on the network interface for PGAS languages and this paper focuses on the shared space addressing. Combining both an efficient addressing a an efficient network interface would provide a very efficient support for PGAS; this is noted as future work.
@cite_18 @cite_1
185800698 2035932528 2119148081 2169240096
Whereas most applications in the realm of the partitioned global address space make use of PGAS languages we here demonstrate an implementation on top of a PGAS-API. In order to improve the scalability of the unstructured CFD solver TAU we have implemented an asynchronous communication strategy on top of the PGAS-API of GPI. We have replaced the bulk-synchronous two-sided MPI exchange with an asynchronous, RDMA-driven, one-sided communication pattern. We also have developed an asynchronous shared memory strategy for the TAU solver. We demonstrate that the corresponding implementation not only scales one order of magnitude higher than the original MPI implementation, but that it also outperforms the hybrid OpenMP MPI programming model. Intel Many Integrated Core (MIC) architectures are becoming an integral part of modern supercomputer architectures due to their high compute density and performance per watt. Partitioned Global Address Space (PGAS) programming models, such as OpenSHMEM, provide an attractive approach for developing scientific applications with irregular communication characteristics, by abstracting shared memory address space, along with one-sided communication semantics. However, the current OpenSHMEM standard does not efficiently support heterogeneous memory architectures such as Xeon Phi. Host and Xeon Phi cores have different memory capacities and compute characteristics. But, the global symmetric memory allocation in the current OpenSHMEM standard mandates that same amount of memory be allocated on every process. In this paper, we propose extensions to overcome this restriction and propose high performance runtime-level designs for efficient communication involving Xeon Phi processors. Further, we re-design applications to demonstrate the effectiveness of the proposed designs and extensions. Experimental evaluations indicate 4X to 7X reduction in OpenSHMEM data movement operation latencies, and 6X to 11X improvement in performance for collective operations. Application evaluations in symmetric mode indicate performance improvements of 28 at 1,024 processes. Further, application redesigns using the proposed extensions provide several magnitudes of performance improvement, as compared to the symmetric mode. To the best of our knowledge, this is the first research work that proposes high performance runtime designs for OpenSHMEM on Intel Xeon Phi clusters. Partitioned Global Address Space (PGAS) languages combine the programming convenience of shared memory with the locality and performance control of message passing. One such language, Unified Parallel C (UPC) is an extension of ISO C defined by a consortium that boasts multiple proprietary and open source compilers. Another PGAS language, Titanium, is a dialect of JavaTM designed for high performance scientific computation. In this paper we describe some of the highlights of two related projects, the Titanium project centered at U.C. Berkeley and the UPC project centered at Lawrence Berkeley National Laboratory. Both compilers use a source-to-source strategy that trans-lates the parallel languages to C with calls to a communication layer called GASNet. The result is portable high-performance compilers that run on a large variety of shared and distributed memory multiprocessors. Both projects combine compiler, runtime, and application efforts to demonstrate some of the performance and productivity advantages to these languages. The looming challenges of exascale computing are generating renewed interest in the Partitioned Global Address Space (PGAS) parallel programming model. From a high-performance interconnect perspective, the one-sided communication model in PGAS addresses some of the challenges that could inhibit the viability of traditional two-sided MPI message passing on millions of cores. The OpenSHMEM project is a recent effort at standardizing a lightweight one-sided communication interface to enable the development of highly scalable PGAS applications. The Portals data movement layer is a low-level one-sided interface that has successfully supported several higher-level one-sided and two-sided interfaces, including PGAS network transport layers. This paper describes the design and implementation of OpenSHMEM on the latest generation of the Portals interface, including a description of how Portals has evolved to maximize performance for PGAS-style communication operations.