Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. GLOBAL ADDRESS SPACE <s> There is growing evidence that current architectures do not well handle cache-unfriendly applications such as sparse math operations, data analytics, and graph algorithms. This is due, in part, to the irregular memory access patterns demonstrated by these applications, and in how remote memory accesses are handled. This paper introduces a new, highly-scalable PGAS memory-centric system architecture where migrating threads travel to the data they access. Scaling both memory capacities and the number of cores can be largely invisible to the programmer.The first implementation of this architecture, implemented with FPGAs, is discussed in detail. A comparison of key parameters with a variety of today's systems, of differing architectures, indicates the potential advantages. Early projections of performance against several well-documented kernels translate these advantages into comparative numbers. Future implementations of this architecture may expand the performance advantages by the application of current state of the art silicon technology. <s> BIB001 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. GLOBAL ADDRESS SPACE <s> Maintaining a scalable high-performance virtual global address space using distributed memory hardware has proven to be challenging. In this paper we evaluate a new approach for such an active global address space that leverages the capabilities of the network fabric to manage addressing, rather than software at the endpoint hosts. We describe our overall approach, design alternatives, and present initial experimental results that demonstrate the effectiveness and limitations of existing network hardware. <s> BIB002 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> G. GLOBAL ADDRESS SPACE <s> Non-volatile memory (NVM) provides a scalable and power-efficient solution to replace DRAM as main memory. However, because of relatively high latency and low bandwidth of NVM, NVM is often paired with DRAM to build a heterogeneous memory system (HMS). As a result, data objects of the application must be carefully placed to NVM and DRAM for best performance. In this paper, we introduce a lightweight runtime solution that automatically and transparently manage data placement on HMS without the requirement of hardware modifications and disruptive change to applications. Leveraging online profiling and performance models, the runtime characterizes memory access patterns associated with data objects, and minimizes unnecessary data movement. Our runtime solution effectively bridges the performance gap between NVM and DRAM. We demonstrate that using NVM to replace the majority of DRAM can be a feasible solution for future HPC systems with the assistance of a software-based data management. <s> BIB003
Collaborating European projects united by a common goal: ExaNoDe, ExaNeSt and ECOSCALE, propose the memory model that enabled a real global address space (GAS), where data are actually shared between multiple partitions without the need to perform memory transfers( [40]- BIB003 ). The ExaNoDe operating system is a NUMA-aware Linux that supports dynamic distributed memory management and userlevel socket-based communication across servers to offer the benefits of UNIMEM system architecture. UNIMEM provides data movement mechanisms and integration with peripheral devices, allowing to use memory hierarchy for the OS by accessing the light RDMA operations. A general scheme of the GAS is shown in Figure 8 . A partitioned GAS (PGAS) introduced in [44] is a non-uniform global address space where the representation of a global address statically encodes the physical address of the memory. PGAS approaches can be seen as a compromise between exposing locality and presenting global access to data. A general scheme of the PGAS is shown in Figure 9 . Address translation in PGAS uses static address mappings that are efficient for RDMA, but that limit the ability for the computation and communication load balancing. Highly-scalable PGAS memory-centric system architecture where threads migrate to the data they access is presented in BIB001 . A portion of the address space marked as ''Replicated'' is used to ensure data locality and it is accessible via a programming model. The instruction code is always maintained in replicated memory so that a thread can always find its code locally without knowing where the thread is executing. Active GAS (AGAS) was designed by BIB002 for a message-driven runtime system to dynamically balance load and data during execution. A general scheme of the AGAS is shown in Figure 10 . AGAS allows performing hardware-controlled dynamic mapping, thanks to the fact that distributed shared memory systems create logical shared address space using distributed memory hardware at the OS level. Additionally, a network-managed global virtual memory where the AGAS runtime set-ups the network fabric to adaptively route messages based on memory locations rather than host identifiers was proposed. For the memory accessed via GAS the need for cache coherency is eliminated since the data does not move. However, GAS complicates memory access because all memory accesses are remote, even if memory is located on a local node. On the other hand, PGAS provide a more-scalable solution by tightly coupling threads to memory locations. However, application classes that can utilise PGAS efficiently are limited due to difficulty in designing algorithms around the concept of stationary-threads.