A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Static cache analysis characterizes a program's cache behavior by determining in a sound but approximate manner which memory accesses result in cache hits and which result in cache misses. Such information is valuable in optimizing compilers, worst-case execution time analysis, and side-channel attack quantification and mitigation.Cache analysis is usually performed as a combination of `must' and `may' abstract interpretations, classifying instructions as either `always hit', `always miss', or `unknown'. Instructions classified as `unknown' might result in a hit or a miss depending on program inputs or the initial cache state. It is equally possible that they do in fact always hit or always miss, but the cache analysis is too coarse to see it.Our approach to eliminate this uncertainty consists in (i) a novel abstract interpretation able to ascertain that a particular instruction may definitely cause a hit and a miss on different paths, and (ii) an exact analysis, removing all remaining uncertainty, based on model checking, using abstract-interpretation results to prune down the model for scalability.We evaluated our approach on a variety of examples; it notably improves precision upon classical abstract interpretation at reasonable cost. <s> BIB001 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Decision support systems (DSS) and data warehousing workloads comprise an increasing fraction of the database market today. I/O capacity and associated processing requirements for DSS workloads are increasing at a rapid rate, doubling roughly every nine to twelve months [38]. In response to this increasing storage and computational demand, we present a computer architecture for decision support database servers that utilizes “intelligent” disks (IDISKs). IDISKs utilize low-cost embedded general-purpose processing, main memory, and high-speed serial communication links on each disk. IDISKs are connected to each other via these serial links and high-speed crossbar switches, overcoming the I/O bus bottleneck of conventional systems. By off-loading computation from expensive desktop processors, IDISK systems may improve cost-performance. More importantly, the IDISK architecture allows the processing of the system to scale with increasing storage demand. <s> BIB002 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> The increasing performance and decreasing cost of processors and memory are causing system intelligence to move into peripherals from the CPU. Storage system designers are using this trend toward “excess” compute power to perform more complex processing and optimizations inside storage devices. To date, such optimizations have been at relatively low levels of the storage protocol. At the same time, trends in storage density, mechanics, and electronics are eliminating the bottleneck in moving data off the media and putting pressure on interconnects and host processors to move data more efficiently. We propose a system called Active Disks that takes advantage of processing power on individual disk drives to run application-level code. Moving portions of an application’s processing to execute directly at disk drives can dramatically reduce data traffic and take advantage of the storage parallelism already present in large systems today. We discuss several types of applications that would benefit from this capability with a focus on the areas of database, data mining, and multimedia. We develop an analytical model of the speedups possible for scan-intensive applications in an Active Disk system. We also experiment with a prototype Active Disk system using relatively low-powered processors in comparison to a database server system with a single, fast processor. Our experiments validate the intuition in our model and demonstrate speedups of 2x on 10 disks across four scan-based applications. The model promises linear speedups in disk arrays of hundreds of disks, provided the application data is large enough. <s> BIB003 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> This paper considers the design, fabrication, and characterization of very small MOSFET switching devices suitable for digital integrated circuits using dimensions of the order of 1μ. Scaling relationships are presented which show how a conventional MOSFET can be reduced in size. An improved small device structure is presented that uses ion implantation to proVide shallow source and drain regions and a nonuniform substrate doping profile. Onedimensional models are used to predict the substrate doping profile and the corresponding threshold voltage versus source voltage characteristic. A two-dimensional current transport model is used to predict the relative degree of short-channel effects for different device parameter combinations. Polysilicon-gate MOSFETs with channel lengths as short as 0.5μ were fabricated, and the device characteristics measured and compared with predicted values. Ibe performance improvement expected from using these very small devices in highly miniaturized integrated circuits is projected. Reprintedfrom the IEEE Journal of Solid-State Circuits, Vol. SC-9, October 1974, pp. 256-268.] <s> BIB004 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Since 2005, processor designers have increased core counts to exploit Moore's Law scaling, rather than focusing on single-core performance. The failure of Dennard scaling, to which the shift to multicore parts is partially a response, may soon limit multicore scaling just as single-core scaling has been curtailed. This paper models multicore scaling limits by combining device scaling, single-core scaling, and multicore scaling to measure the speedup potential for a set of parallel workloads for the next five technology generations. For device scaling, we use both the ITRS projections and a set of more conservative device scaling parameters. To model single-core scaling, we combine measurements from over 150 processors to derive Pareto-optimal frontiers for area/performance and power/performance. Finally, to model multicore scaling, we build a detailed performance model of upper-bound performance and lower-bound core power. The multicore designs we study include single-threaded CPU-like and massively threaded GPU-like multicore chip organizations with symmetric, asymmetric, dynamic, and composed topologies. The study shows that regardless of chip organization and topology, multicore scaling is power limited to a degree not widely appreciated by the computing community. Even at 22 nm (just one year from now), 21% of a fixed-size chip must be powered off, and at 8 nm, this number grows to more than 50%. Through 2024, only 7.9x average speedup is possible across commonly used parallel workloads, leaving a nearly 24-fold gap from a target of doubled performance per generation. <s> BIB005 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> This article consists of a collection of slides from the author's conference presentation on the special features, system design, system architectures; processing capabilities, and targeted markets for Micron's Hybrid Memory Cube (HMC). <s> BIB006 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Multi-core processor performance is limited by memory system bandwidth. The Hybrid Memory Cube is a three-dimensional DRAM architecture that improves latency, bandwidth, power and density. Through-silicon vias (TSVs), 3D packaging and advanced CMOS performance enable a new approach to memory system architecture. Heterogeneous die are stacked with significantly more connections, thereby reducing the distance signals travel. <s> BIB007 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> A 1.2 V 1 Gb mobile SDRAM, having 4 channels with 512 DQ pins has been developed with 50 nm technology. It exhibits 330.6 mW read operating power during 4 channel operation, achieving 12.8 GB/s data bandwidth. Test correlation techniques to verify functions through micro bumps and test pads have been developed. Block based dual period refresh scheme is applied to reduce self refresh current with minimum chip size burden. Stacking of 2 dies with 7.5 μm diameter and 40 μm pitch TSVs has been fabricated and tested, which results in 76% overall package yield without difference in performances between top and bottom die. <s> BIB008 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Increasing demand for higher-bandwidth DRAM drive TSV technology development. With the capacity of fine-pitch wide I/O [1], DRAM can be directly integrated on the interposer or host chip and communicate with the memory controller. However, there are many limitations, such as reliability and testability, in developing the technology. It is advantageous to adopt a logic-interface chip between the interposer and stacked-DRAM with thousands of TSV. The logic interface chip in the base level of high-bandwidth memory (HBM) decreases the CIO, repairs the chip-to-chip connection failure, and supports better testability and improves reliability. <s> BIB009 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Many studies point to the difficulty of scaling existing computer architectures to meet the needs of an exascale system (i.e., capable of executing $10^{18} $ floating-point operations per second), consuming no more than 20 MW in power, by around the year 2020. This paper outlines a new architecture, the Active Memory Cube, which reduces the energy of computation significantly by performing computation in the memory module, rather than moving data through large memory hierarchies to the processor core. The architecture leverages a commercially demonstrated 3D memory stack called the Hybrid Memory Cube, placing sophisticated computational elements on the logic layer below its stack of dynamic random-access memory (DRAM) dies. The paper also describes an Active Memory Cube tuned to the requirements of a scientific exascale system. The computational elements have a vector architecture and are capable of performing a comprehensive set of floating-point and integer instructions, predicated operations, and gather-scatter accesses across memory in the Cube. The paper outlines the software infrastructure used to develop applications and to evaluate the architecture, and describes results of experiments on application kernels, along with performance and power projections. <s> BIB010 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> The end of Dennard scaling has made all systemsenergy-constrained. For data-intensive applications with limitedtemporal locality, the major energy bottleneck is data movementbetween processor chips and main memory modules. For such workloads, the best way to optimize energy is to place processing near the datain main memory. Advances in 3D integrationprovide an opportunity to implement near-data processing (NDP) withoutthe technology problems that similar efforts had in the past. This paper develops the hardware and software of an NDP architecturefor in-memory analytics frameworks, including MapReduce, graphprocessing, and deep neural networks. We develop simple but scalablehardware support for coherence, communication, and synchronization, anda runtime system that is sufficient to support analytics frameworks withcomplex data patterns while hiding all thedetails of the NDP hardware. Our NDP architecture provides up to 16x performance and energy advantageover conventional approaches, and 2.5x over recently-proposed NDP systems. We also investigate the balance between processing and memory throughput, as well as the scalability and physical and logical organization of the memory system. Finally, we show that it is critical to optimize software frameworksfor spatial locality as it leads to 2.9x efficiency improvements for NDP. <s> BIB011 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> One of the most critical challenges for today's and future data-intensive and big-data problems is data storage and analysis. This paper first highlights some challenges of the new born Big Data paradigm and shows that the increase of the data size has already surpassed the capabilities of today's computation architectures suffering from the limited bandwidth, programmability overhead, energy inefficiency, and limited scalability. Thereafter, the paper introduces a new memristor-based architecture for data-intensive applications. The potential of such an architecture in solving data-intensive problems is illustrated by showing its capability to increase the computation efficiency, solving the communication bottleneck, reducing the leakage currents, etc. Finally, the paper discusses why memristor technology is very suitable for the realization of such an architecture; using memristors to implement dual functions (storage and logic) is illustrated. <s> BIB012 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Growing main memory capacity has fueled the development of in-memory big data management and processing. By eliminating disk I/O bottleneck, it is now possible to support interactive data analytics. However, in-memory systems are much more sensitive to other sources of overhead that do not matter in traditional I/O-bounded disk-based systems. Some issues such as fault-tolerance and consistency are also more challenging to handle in in-memory environment. We are witnessing a revolution in the design of database systems that exploits main memory as its data storage layer. Many of these researches have focused along several dimensions: modern CPU and memory hierarchy utilization, time/space efficiency, parallelism, and concurrency control. In this survey, we aim to provide a thorough review of a wide range of in-memory data management and processing proposals and systems, including both data storage systems and data processing frameworks. We also give a comprehensive presentation of important technology in memory management, and some key factors that need to be considered in order to achieve efficient in-memory data management and processing. <s> BIB013 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Computer memories continue to serve the role that they first served in the electronic discrete variable automatic computer (EDVAC) machine documented by John von Neumann, namely that of supplying instructions and operands for calculations in a timely manner. As technology has made possible significantly larger and faster machines with multiple processors, the relative distance in processor cycles of this memory has increased considerably. Microarchitectural techniques have evolved to share this memory across ever-larger systems of processors with deep cache hierarchies and have managed to hide this latency for many applications, but are proving to be expensive and energy-inefficient for newer types of problems working on massive amounts of data. New paradigms include scale-out systems distributed across hundreds and even thousands of nodes, in-memory databases that keep data in memory much longer than the duration of a single task, and near-data computation, where some of the computation is off-loaded to the location of the data to avoid wasting energy in the movement of data. This paper provides a historical perspective on the evolution of memory architecture, and suggests that the requirements of new problems and new applications are likely to fundamentally change processor and system architecture away from the currently established von Neumann model. <s> BIB014 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> A major shift from compute-centric to data-centric computing systems can be perceived, as novel big data workloads like cognitive computing and machine learning strongly enforce embarrassingly parallel and highly efficient processor architectures. With Moore's law having surrendered, innovative architectural concepts as well as technologies are urgently required, to enable a path for tackling exascale and beyond -- even though current computing systems face the inevitable instruction-level parallelism, power, memory, and bandwidth walls. As part of any computing system, the general perception of memories depicts unreliability, power hungriness and slowness, resulting in a future prospective bottleneck. The latter being an outcome of a pin limitation derived by packaging constraints, an unexploited tremendous row bandwidth is determinable, which off-chip diminishes to a bare minimum. Building upon a shift towards data-centric computing systems, the near-memory processing concept seems to be most promising, since power efficiency and computing performance increase by co-locating tasks on bandwidth-rich in-memory processing units, whereas data motion mitigates by the avoidance of entire memory hierarchies. By considering the umbrella of near-data processing as the urgent required breakthrough for future computing systems, this survey presents its derivations with a special emphasis on Processing-In-Memory (PIM), highlighting historical achievements in technology as well as architecture while depicting its advantages and obstacles. <s> BIB015 </s> A Review of Near-Memory Computing Architectures: Opportunities and Challenges <s> I. INTRODUCTION <s> Poor DRAM technology scaling over the course of many years has caused DRAM-based main memory to increasingly become a larger system bottleneck. A major reason for the bottleneck is that data stored within DRAM must be moved across a pin-limited memory channel to the CPU before any computation can take place. This requires a high latency and energy overhead, and the data often cannot benefit from caching in the CPU, making it difficult to amortize the overhead. Modern 3D-stacked DRAM architectures include a logic layer, where compute logic can be integrated underneath multiple layers of DRAM cell arrays within the same chip. Architects can take advantage of the logic layer to perform processing-in-memory (PIM), or near-data processing. In a PIM architecture, the logic layer within DRAM has access to the high internal bandwidth available within 3D-stacked DRAM (which is much greater than the bandwidth available between DRAM and the CPU). Thus, PIM architectures can effectively free up valuable memory channel bandwidth while reducing system energy consumption. A number of important issues arise when we add compute logic to DRAM. In particular, the logic does not have low-latency access to common CPU structures that are essential for modern application execution, such as the virtual memory and cache coherence mechanisms. To ease the widespread adoption of PIM, we ideally would like to maintain traditional virtual memory abstractions and the shared memory programming model. This requires efficient mechanisms that can provide logic in DRAM with access to CPU structures without having to communicate frequently with the CPU. To this end, we propose and evaluate two general-purpose solutions that minimize unnecessary off-chip communication for PIM architectures. We show that both mechanisms improve the performance and energy consumption of many important memory-intensive applications. <s> BIB016
Over the years memory technology has not been able to keep up with advancements in processor technology in terms of latency and energy consumption which is infamously termed as the memory wall BIB001 . Earlier, system architects tried to bridge this gap by introducing memory hierarchies which mitigated some of the disadvantages of off-chip DRAMs. But the limited number of pins on the memory package is not able to meet today's bandwidth demands of multicore processors. Furthermore, with the demise of Dennard scaling BIB004 , slowing of Moore's law, and dark silicon computer performance has reached a plateau BIB005 . At the same time, we are witnessing an enormous amount of data being generated across multiple areas like radio astronomy, material science, chemistry, health sciences etc BIB010 . In radio astronomy for example, the first phase of the Square Kilometre Array (SKA) aims at processing over 100 terabytes of raw data samples per second, yielding of the order of 300 petabytes of SKA data products annually  . The SKA currently is in the design phase with anticipated construction in South Africa and Australia in the first half of the coming decade. These radio-astronomy applications usually exhibit massive data parallelism and low operational intensity with limited locality. On traditional systems, these applications cause frequent data movement between the memory subsystem and the processor which has a severe impact on performance and energy efficiency. Therefore, a lot of current research is being focused on coming up with innovative manufacturing technologies and architectures to overcome these problems. Today's memory hierarchy usually consists of multiple levels of cache, a main memory, and a storage. The traditional approach is of moving data all the way up to caches from the storage and then processing it. In contrast, near-memory computing (NMC) aims at processing close to where the data resides. This data-centric approach couples compute units close to the data and seeks to minimize the expensive data movements. Conceptually, this principle can be applied to any level of memory subsystem and with several technology options, including processing on buffer-on-board (BoB), edgebonding small processor dies on DRAM chips etc BIB011 . Notably, three-dimensional stacking is touted as the true enabler of processing close to the memory. It allows the stacking of logic die and memory together by means of through-silicon via's (TSVs) which helps in reducing memory access latency, power consumption, and provides much higher bandwidth BIB007 . Micron's Hybrid Memory Cube (HMC) BIB006 , High bandwidth Memory (HBM) BIB009 from AMD and Hynix, and Samsung's Wide I/O BIB008 are the memory industries competing 3D memory products. Figure 1 depicts the system evolution based on the information referenced by a program during execution which is referred to as a working set BIB012 . Prior systems were based on a CPU-centric approach where data is moved to the core for processing ( Fig.1 (a) -(c)), whereas now with near-memory processing ( Fig.1 (d) ) the processing cores are brought to the place where data resides. Computation in-memory ( Fig.1 (e) ) is an extreme step of completely reducing data movement by using novel devices (eg. memristors, phase change memory (PCM)) as these memories have inherent compute capability Figure 2 shows a high level view of our classification which is based on the level in the memory hierarchy and further split into the type of compute implementation (Programmable, Fixed-Function, or Reconfigurable). In our taxonomy we don't include disk-based systems as they can no longer offer timely response due to the high access latency and high failure rate of disks BIB013 . Nevertheless, there have been research efforts towards providing processing capabilities in the disk. However, it was not adopted widely by the industry due to of the marginal performance improvement that could not justify the associated cost BIB002 , BIB003 . Instead, we include emerging non-volatile memories termed as storageclass memories (SCMs) BIB014 which are trying to fill the latency gap between DRAMs and disks. Building up on similar efforts BIB015 , BIB016 , we make the following contributions: 1) We analyze and organize the literature related to the landscape of near-memory computing under various dimensions; 2) We provide guidelines for design space exploration focusing on near-memory systems; 3) We present a case study to illustrate the potential of nearmemory computing for memory intensive application with a comparison to the current CPU-centric approach; 4) We outline the direction for future research and highlight current challenges in the domain of near-memory computing. The remainder of this article is structured as follows. Section II outlines the evaluation and classification scheme for NMC at main memory (Section III) and storage class memory (Section IV). Section V highlights the challenges with cache coherence, virtual memory, the lack of programming models and data mapping schemes for NMC. In Section VI we look into the tools and techniques used while performing the design space exploration for these systems. This section also illustrates the importance of application characterization for these systems. Section VII presents a high level analytic approach to model NMC systems with a comparison to a traditional CPU-centric approach. Section VIII highlights the lessons learned and future research directions. • Memory -The decision of using what kind of memory is one of the most fundamental questions on which the near-memory architecture depends. • Processing -Type of processing unit implemented and the granularity of processing it performs plays a critical role during the design space exploration. • Tool -Any architecture's success depends heavily on the available tool support. As NMC is a fledgling field, there is lack of generic design methodologies. • Interoperability -One of the main challenges faced by NMC architectures is its inadequacy at providing support for a programming model, cache coherence, virtual memory, and efficient data mapping. • Application -Usually these architectures focus on big data applications/algorithms and this massive amount of data is produced across all the markets. Therefore, in our table we include the domain of the application.