A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> Programming Model <s> Processing -in-memory (PIM) is rapidly rising as a viable solution for the memory wall crisis, rebounding from its unsuc­ cessful attempts in I990s due to practicality concerns, which are alleviated with recent advances in 3D stacking technolo­ gies. However, it is still challenging to integrate the PIM architectures with existing systems in a seamless manner due to two common characteristics: unconventional progr amming models for in-memor y computation units and lack of ability to utilize large on- chip caches. In this paper, we propose a new PIM architecture that (I) does not change the existing sequential progr amming mod­ el s and (2) aut omatically decides whet her to exe cut e PIM operations in memory or processor s de pending on the local­ ity of data. The key idea is to implement simple in-memory computation using compute- capable memory commands and use specialized instructions, which we call PIM- enabled in­ structions, to invo ke in-memory computation. This allows PIM operations to be interoperable with existing program­ ming models, cache coherence protocols, and virtual memo ry mechanisms with no modification. In addition, we introduce a simple har dware struct ure that monitors the locality of data acce ssed by a PIM-en abled instruction at runtime to adap ­ tively execut e the instruction at the host processor (instead of in memory) when the instruction can benefit from large <s> BIB001 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> Programming Model <s> Main memory bandwidth is a critical bottleneck for modern GPU systems due to limited off-chip pin bandwidth. 3D-stacked memory architectures provide a promising opportunity to significantly alleviate this bottleneck by directly connecting a logic layer to the DRAM layers with high bandwidth connections. Recent work has shown promising potential performance benefits from an architecture that connects multiple such 3D-stacked memories and offloads bandwidth-intensive computations to a GPU in each of the logic layers. An unsolved key challenge in such a system is how to enable computation offloading and data mapping to multiple 3D-stacked memories without burdening the programmer such that any application can transparently benefit from near-data processing capabilities in the logic layer. Our paper develops two new mechanisms to address this key challenge. First, a compiler-based technique that automatically identifies code to offload to a logic-layer GPU based on a simple cost-benefit analysis. Second, a software/hardware cooperative mechanism that predicts which memory pages will be accessed by offloaded code, and places those pages in the memory stack closest to the offloaded code, to minimize off-chip bandwidth consumption. We call the combination of these two programmer-transparent mechanisms TOM: Transparent Offloading and Mapping. Our extensive evaluations across a variety of modern memory-intensive GPU workloads show that, without requiring any program modification, TOM significantly improves performance (by 30% on average, and up to 76%) compared to a baseline GPU system that cannot offload computation to 3D-stacked memories. <s> BIB002 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> Programming Model <s> Poor DRAM technology scaling over the course of many years has caused DRAM-based main memory to increasingly become a larger system bottleneck. A major reason for the bottleneck is that data stored within DRAM must be moved across a pin-limited memory channel to the CPU before any computation can take place. This requires a high latency and energy overhead, and the data often cannot benefit from caching in the CPU, making it difficult to amortize the overhead. Modern 3D-stacked DRAM architectures include a logic layer, where compute logic can be integrated underneath multiple layers of DRAM cell arrays within the same chip. Architects can take advantage of the logic layer to perform processing-in-memory (PIM), or near-data processing. In a PIM architecture, the logic layer within DRAM has access to the high internal bandwidth available within 3D-stacked DRAM (which is much greater than the bandwidth available between DRAM and the CPU). Thus, PIM architectures can effectively free up valuable memory channel bandwidth while reducing system energy consumption. A number of important issues arise when we add compute logic to DRAM. In particular, the logic does not have low-latency access to common CPU structures that are essential for modern application execution, such as the virtual memory and cache coherence mechanisms. To ease the widespread adoption of PIM, we ideally would like to maintain traditional virtual memory abstractions and the shared memory programming model. This requires efficient mechanisms that can provide logic in DRAM with access to CPU structures without having to communicate frequently with the CPU. To this end, we propose and evaluate two general-purpose solutions that minimize unnecessary off-chip communication for PIM architectures. We show that both mechanisms improve the performance and energy consumption of many important memory-intensive applications. <s> BIB003
A critical challenge in adoption of NMC is to support a heterogeneous processing environment of a host system and near-memory processing unit. It is not trivial to determine how and who identifies which part of an application should run in the near-memory processing units. Currently, this is left onto the programmer or the compiler BIB002 to mark sections of the code that must be processed near-memory compute units BIB003 . In another approach, Ahn et al. BIB001 use some special set of NMC instructions which invoke NMC logic units. However, this approach calls for sophisticated mechanism (e.g., taking care of the data locality) on the host side to avoid unnecessary exchange of information. Hence, there is still a lot of research required in coming up with an efficient approach to ease the programming burden.