A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> Data mapping <s> Main memory bandwidth is a critical bottleneck for modern GPU systems due to limited off-chip pin bandwidth. 3D-stacked memory architectures provide a promising opportunity to significantly alleviate this bottleneck by directly connecting a logic layer to the DRAM layers with high bandwidth connections. Recent work has shown promising potential performance benefits from an architecture that connects multiple such 3D-stacked memories and offloads bandwidth-intensive computations to a GPU in each of the logic layers. An unsolved key challenge in such a system is how to enable computation offloading and data mapping to multiple 3D-stacked memories without burdening the programmer such that any application can transparently benefit from near-data processing capabilities in the logic layer. Our paper develops two new mechanisms to address this key challenge. First, a compiler-based technique that automatically identifies code to offload to a logic-layer GPU based on a simple cost-benefit analysis. Second, a software/hardware cooperative mechanism that predicts which memory pages will be accessed by offloaded code, and places those pages in the memory stack closest to the offloaded code, to minimize off-chip bandwidth consumption. We call the combination of these two programmer-transparent mechanisms TOM: Transparent Offloading and Mapping. Our extensive evaluations across a variety of modern memory-intensive GPU workloads show that, without requiring any program modification, TOM significantly improves performance (by 30% on average, and up to 76%) compared to a baseline GPU system that cannot offload computation to 3D-stacked memories. <s> BIB001 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> Data mapping <s> Emerging 3D stacked memory systems provide significantly more bandwidth than current DDR modules. However, general purpose processors do not take full advantage of these resources offered by the memory modules. Taking advantage of the increased bandwidth requires the use of specialized processing units. In this paper, we evaluate the benefits of placing hardware accelerators at the bottom layer of a 3D stacked memory system compared to accelerators that are placed external to the memory stack. Our evaluation of the design using cycle-accurate simulation and RTL synthesis shows that, for important data intensive kernels, near-memory accelerators inside a single 3D memory package provide 3x-13x speedup over a Quad-core Xeon processor. Most of the benefits are from the application of accelerators, as the near-memory configurations provide marginal benefits compared to the same number of accelerators placed on a die external to the memory package. This comparable performance for external accelerators is due to the high bandwidth afforded by the high-speed off-chip links. On the other hand, near-memory accelerators consume 7%–39% less energy than the external accelerators. <s> BIB002
The absence of effective data mapping mechanisms can severely hamper the benefits of processing close to memory. The data should be mapped in such a way that data that will be used by the near-memory processing units should be available in the vicinity. Hence, it is crucial to look into effective data mapping schemes (static and dynamic) In TOM BIB001 , they proposed a software/hardware cooperative method to predict which pages of the memory will be used by the offloaded code, and they tried to minimize the bandwidth consumption by placing those pages in the memory stack closest to the offloaded code. Yitbarek et al. BIB002 proposed a data-mapping scheme which they integrated in a The number of distinct memory accesses since the last access to a given memory address Data Stream Stride (spatial locality) The difference in memory addresses between two consecutive memory accesses Application bandwidth The amount of I/O bandwidth utilized by a particular application Instruction mix Provides a general overview of the application Parallelism Inherent application parallelism such as instruction level parallelism, thread level parallelism, data level parallelism Branch Entropy Measures the randomness of branch behavior which is essential to provide speculation support Table 3 : Several metrics for micro-architecture independent analysis of workloads Fig. 3 : Design space exploration for system evaluation platform that used some accelerators near 3D-stacked memory. Near-memory workloads often access data from contiguous addresses for this reason they re-mapped data at page granularity in order to have contiguous addresses in the same memory vaults.