Survey of Memory Management Techniques for HPC and Cloud Computing <s> F. HIGH BANDWIDTH MEMORY <s> As the number of cores installed in a single computing node drastically increases, the intra-node communication between parallel processes becomes more important. The parallel programming models, such as Message Passing Interface (MPI), internally perform memory-intensive operations for intra-node communication. Thus, to address the scalability issue on many-core processors, it is critical to exploit emerging memory features provided by the contemporary computer systems. For example, the latest many-core processors are equipped with a high-bandwidth on-package memory Modern 64-bit processors also support a large page size (e.g., 2MB), which can significantly reduce the number of TLB misses. The on-package memory and the huge pages have considerable potential for improving the performance of intra-node communication. However, such features are not thoroughly investigated in terms of intra-node communication in the literature. In this paper, we propose enhanced memory management schemes to efficiently utilize the on-package memory and provide support for huge pages. The proposed schemes can significantly reduce the data copy and memory mapping overheads in MPI intra-node communication. Our experimental results show that our implementation on MVAPICH2 can improve the bandwidth of point-to-point communication up to 373%, and can reduce the latency of collective communication by 79% on an Intel Xeon Phi Knights Landing (KNL) processor. <s> BIB001 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> F. HIGH BANDWIDTH MEMORY <s> Today's supercomputers are moving towards deployment of many-core processors like Intel Xeon Phi Knights Landing (KNL), to deliver high compute and memory capacity. Applications executing on such many-core platforms with improved vectorization require high memory bandwidth. To improve performance, architectures like Knights Landing include a high bandwidth and low capacity in-package high bandwidth memory (HBM) in addition to the high capacity but low bandwidth DDR4. Other architectures like Nvidia's Pascal GPU also expose similar stacked DRAM. In architectures with heterogeneity in memory types within a node, efficient allocation and data movement can result in improved performance and energy savings in future systems if all the data requests are served from the high bandwidth memory. In this paper, we propose a memory-heterogeneity aware runtime system which guides data prefetch and eviction such that data can be accessed at high bandwidth for applications whose entire working set does not fit within the high bandwidth memory and data needs to be moved among different memory types. We implement a data movement mechanism managed by the runtime system which allows applications to run efficiently on architectures with heterogeneous memory hierarchy, with trivial code changes. We show upto 2X improvement in execution time for Stencil3D and Matrix Multiplication which are important HPC kernels. <s> BIB002 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> F. HIGH BANDWIDTH MEMORY <s> To feed the high degrees of parallelism in modern graphics processors and manycore CPU designs, DRAM manufacturers have created new DRAM architectures that deliver high bandwidth. This paper presents a simulation-based study of the most common forms of DRAM today: DDR3, DDR4, and LPDDR4 SDRAM; GDDR5 SGRAM; and two recent 3D-stacked architectures: High Bandwidth Memory (HBM1, HBM2), and Hybrid Memory Cube (HMC1, HMC2). Our simulations give both time and power/energy results and reveal several things: (a) current multi-channel DRAM technologies have succeeded in translating bandwidth into better execution time for all applications, turning memory-bound applications into compute-bound; (b) the inherent parallelism in the memory system is the critical enabling factor (high bandwidth alone is insufficient); (c) while all current DRAM architectures have addressed the memory-bandwidth problem, the memory-latency problem does still remain, dominated by queuing delays arising from lack of parallelism; and (d) the majority of power and energy is spent in the I/O interface, driving bits across the bus; DRAM-specific overhead beyond bandwidth has been reduced significantly, which is great news (an ideal memory technology would dissipate power only in bandwidth, all else would be free). <s> BIB003 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> F. HIGH BANDWIDTH MEMORY <s> Abstract Heterogenous memory systems that consist of multiple memory technologies are becoming common in high-performance computing environments. Modern processors and accelerators, such as the Intel Knights Landing (KNL) CPU and NVIDIA Volta GPU, feature small-size high-bandwidth memory near the compute cores and large-size normal-bandwidth memory that is connected off-chip. Theoretically, HBM can provide about four times higher bandwidth than conventional DRAM. However, many factors impact the actual performance improvement that an application can achieve on such system. In this paper, we focus on the Intel KNL system and identify the most important factors on the application performance, including the application memory access pattern, the problem size, the threading level and the actual memory configuration. We use a set of representative applications from both scientific and data-analytics domains. Our results show that applications with regular memory access benefit from MCDRAM, achieving up to three times performance when compared to the performance obtained using only DRAM. On the contrary, applications with irregular memory access pattern are latency-bound and may suffer from performance degradation when using only MCDRAM. Also, we provide memory-centric analysis of four applications, identify their major data objects, correlate their characteristics to the performance improvement on the testbed. <s> BIB004 </s> Survey of Memory Management Techniques for HPC and Cloud Computing <s> F. HIGH BANDWIDTH MEMORY <s> With the advent of Intels second-generation many-core processor (Knights Landing: KNL), high-bandwidth memory (HBM) with potentially five times more bandwidth than existing dynamic random-access memory has become available as a valuable computing resource for high-performance computing (HPC) applications. Therefore, resource management schemes should now be able to consider existing central processing unit cores, conventional main memory, and this newly available HBM to improve the overall system throughput and user response time. In this paper, we present our profiling mechanism and related scheduling policy that analyzes the resource usage patterns of various HPC workloads. By carefully allocating memory-intensive workloads to HBM in KNL, we show that the overall performance of multiple message passing interface workloads can be improved in terms of the execution time and system utilization. We evaluate and verify the effectiveness of our scheme for optimizing the use of HBM by using NAS Parallel Benchmarks. <s> BIB005
High bandwidth memory (HBM) is a new memory technology that allows a significant performance improvement on bandwidth-bound applications. However, more bandwidth does not always bring a better execution time, especially when it is allocated without enough concurrency in the memory system to support it BIB003 . In BIB004 , the HBM-DRAM memory system was analysed in terms of performance benefits of representative HPC applications executions. Such memory architecture consists in the HBM implemented with 3D-stacked Multi-Channel DRAM (MCDRAM) and conventional DRAM. These are significantly different in terms of capacity, bandwidth and latency. The HBM-DRAM system was configured in flat (as an additional NUMA node), cache and hybrid mode. If the intra-node communicator includes a sufficient number of processes or if the buffer size requested is large enough, Enhanced memory management schemes for HBM based on MCDRAM presented in BIB001 allocate user and shared buffers to the high-bandwidth on-package memory. In addition to this, they reduce the overheads of memory mapping. A runtime-aware prefetching mechanism presented in BIB002 allows within the CHARM++ runtime system to prefetch and to evict data heterogeneous memory nodes when the working set does not fit within HBM. The proposed mechanism was investigated for bandwidth-sensitive HPC applications on Intel Xeon Phi KNL. Allocating memoryintensive workloads among MCDRAM and DDR memory resources in Intel's manycore processor KNL using different heterogeneous memory modes is described in BIB005 . An intelligent scheduling approach based on profiled resource usage patterns showed that the actual performance of multiple message passing interface workloads could be improved in terms of the execution time and system utilisation. Knowing the application and the architecture's topology is needed to use different memory modes (shown in Figure 7 ). The configuration which are better fine-tuned, have a significant improvement in performance. This, however, comes at the cost of non-portable optimisations. At the same time, the basic configurations that do not require application or runtime changes can offer less performance. In addition to the configuration, HBM is challenging in terms of using in programming models. For instance, communication libraries such as MPI can use HBM to allocate and manage internal memory objects for achieving efficient inter-node communication support. However, this affects the available memory budget, reducing the amount of memory accessible to the user data and, potentially, reducing the overall performance. In some studies, HBM has been used as a cache for the slower memory. One of the critical advantages of a caching approach is that existing applications can run unmodified. Accessing and storing a large number of tags are challenging in the multi-gigabyte cache managing. 