A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> VII. CASE STUDY <s> Multi-core processor performance is limited by memory system bandwidth. The Hybrid Memory Cube is a three-dimensional DRAM architecture that improves latency, bandwidth, power and density. Through-silicon vias (TSVs), 3D packaging and advanced CMOS performance enable a new approach to memory system architecture. Heterogeneous die are stacked with significantly more connections, thereby reducing the distance signals travel. <s> BIB001 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> VII. CASE STUDY <s> While Processing-in-Memory has been investigated for decades, it has not been embraced commercially. A number of emerging technologies have renewed interest in this topic. In particular, the emergence of 3D stacking and the imminent release of Micron's Hybrid Memory Cube device have made it more practical to move computation near memory. However, the literature is missing a detailed analysis of a killer application that can leverage a Near Data Computing (NDC) architecture. This paper focuses on in-memory MapReduce workloads that are commercially important and are especially suitable for NDC because of their embarrassing parallelism and largely localized memory accesses. The NDC architecture incorporates several simple processing cores on a separate, non-memory die in a 3D-stacked memory package; these cores can perform Map operations with efficient memory access and without hitting the bandwidth wall. This paper describes and evaluates a number of key elements necessary in realizing efficient NDC operation: (i) low-EPI cores, (ii) long daisy chains of memory devices, (iii) the dynamic activation of cores and SerDes links. Compared to a baseline that is heavily optimized for MapReduce execution, the NDC design yields up to 15X reduction in execution time and 18X reduction in system energy. <s> BIB002 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> VII. CASE STUDY <s> Many studies point to the difficulty of scaling existing computer architectures to meet the needs of an exascale system (i.e., capable of executing $10^{18} $ floating-point operations per second), consuming no more than 20 MW in power, by around the year 2020. This paper outlines a new architecture, the Active Memory Cube, which reduces the energy of computation significantly by performing computation in the memory module, rather than moving data through large memory hierarchies to the processor core. The architecture leverages a commercially demonstrated 3D memory stack called the Hybrid Memory Cube, placing sophisticated computational elements on the logic layer below its stack of dynamic random-access memory (DRAM) dies. The paper also describes an Active Memory Cube tuned to the requirements of a scientific exascale system. The computational elements have a vector architecture and are capable of performing a comprehensive set of floating-point and integer instructions, predicated operations, and gather-scatter accesses across memory in the Cube. The paper outlines the software infrastructure used to develop applications and to evaluate the architecture, and describes results of experiments on application kernels, along with performance and power projections. <s> BIB003
We built a high level analytic model which incorporates evaluation methodology as described in figure 3 . To see the potential of computing close to the memory and when it is useful to follow a data-centric approach we make a comparison of a CPU-centric approach consisting of a multi-core system which has DDR3 as a main memory, and compared it to a data-centric approach which has HMC like 3D stacked memory instead of DDR3 and is connected to a host system. In the multi-core system each core has its private L1 cache Table 5 : System parameters and example set representing ARM Cortex-A9 (ARM) processor and a shared L2 cache. Figure 4 shows our 3D stacked near-memory system connected with a host system through a serializer/deserializer (SerDes) I/O link. We embed nearmemory compute units in the logic layer of 3D stack memory. The near-memory compute units are modeled as ARM Cortex-A9 units (table 5) , which are placed in the logic layer of 3D stacked DRAM. We consider a 4-GB HMC like design, with 8 DRAM layers. Each DRAM layer is split into 16 dual banked partitions with 4 vertical partitions forming a vault. Each vault has its own vault controller in the logic layer which is able to access the DRAM layers at a 10 GB/s bandwidth BIB003 . In the near-memory system we assume certain parts of computation, which are memory intensive, to be offloaded to the nearmemory compute units and the other parts will be executed in the host system. The number of vaults inside the HMC and external SerDes links are kept as varying parameters which have an impact on internal and the external bandwidth of the 3D memory respectively. Performance calculation: The model estimates the execution time as the ratio between the number of memory access required in order to move the block of data to and from the I/O system and the available I/O bandwidth of each memory subsystem. The execution time takes into account the time to process the instructions and latency due to memory access at different memory levels. The latency of different memory types have been mentioned in table 5. Energy calculation: A power model for both CPU-centric and near-memory system were built which take into account dynamic as well as static energy. Dynamic energy is the product of the energy per memory access and number of memory accesses at each level of the memory hierarchy. It is assumed to scale linearly with utilization. Static energy is estimated as the product of time and static power for each component. It is scaled based on the number of cores and the cache size. For modeling the HMC we referred to BIB001 and BIB002 . We consider energy per bit of 3.7 pJ/b for accessing the DRAM layers and 1.5 pJ/b for the operations and data movements in the HMC logic layer. Also a static power of 0.96 W BIB002 was considered to model the additional components in the logic layer. Design Space Exploration: We vary the cache miss rate, which encapsulates application characteristics, both at L1 and L2 level to see the impact it has on performance of the entire system. In case of a near-memory system, we considered the scenario when all access are done to the different vaults inside the HMC memory which can exploit the inherent parallelism offered by the 3D memory. In figure 5 , the x and y axis represent the miss rate at L1 and L2 respectively. Results in figure 5 are normalized to NMC+host system. In both the figures we can see a similar trend if the miss rate (L1 and L2) increases the multi core system performance degrades compared to the NMC systems this because of increase in data movement. The data has to be fetched from the off-chip DRAM. If the data brought into the caches is not reused by the CPU, the use of caching becomes inefficient. Therefore, the application (or function) with low locality should take advantage of the near-memory approach whereas the other application (or function) with high locality will benefit more from the traditional approach.