Die-stacked DRAM caches are increasingly advocated to bridge the performance gap between on-chip Cache and main memory. It is essential to improve DRAM cache hit rate and lower cache hit latency simultaneously. Prior DRAM cache designs fall into two categories according to the data mapping polices: set-associative and direct-mapped, achieving either one. In this paper, we propose a partial direct-mapped die-stacked DRAM cache to achieve the both objectives simultaneously, called Gemini, which is motivated by the following observations: applying unified mapping policy to different blocks cannot achieve high cache hit rate and low hit latency in terms of mapping structure. Gemini cache classifies data into leading blocks and following blocks, and places them with static mapping and dynamic mapping respectively in a unified set-associative structure. Gemini also designs a replacement policy to balance the different blocks miss penalty and the recency, and provides strategies to mitigate cache thrashing due to block type transitions. Experimental results demonstrate that Gemini cache can narrow the hit latency gap with direct-mapped cache significantly, from 1.75X to 1.22X on average, and can achieve comparable hit rate with set-associative cache. Compared with the state-of-the-art baselines, i.e., enhanced Loh-Hill cache, Gemini improves the IPC by up to 20 respectively.
. Loh-Hill @cite_16 @cite_3 propose to organize each DRAM row (2 KB) as a 29-way cache set, which provides good replacement flexibility. The tag and data are stored in the same row, and the data is requested right after the tag read to improve row buffer efficiency. Unison Cache @cite_0 and Bi-Modal Cache @cite_12 increase the cache line granularity to exploit the spatial locality, and they rely on the set-associative structure to mitigate the cache conflicts.
@cite_0 @cite_16 @cite_12 @cite_3
2169865228 2148797773 2075293780
Recent research advocates large die-stacked DRAM caches in many core servers to break the memory latency and bandwidth wall. To realize their full potential, die-stacked DRAM caches necessitate low lookup latencies, high hit rates and the efficient use of off-chip bandwidth. Today's stacked DRAM cache designs fall into two categories based on the granularity at which they manage data: block-based and page-based. The state-of-the-art block-based design, called Alloy Cache, collocates a tag with each data block (e.g., 64B) in the stacked DRAM to provide fast access to data in a single DRAM access. However, such a design suffers from low hit rates due to poor temporal locality in the DRAM cache. In contrast, the state-of-the-art page-based design, called Footprint Cache, organizes the DRAM cache at page granularity (e.g., 4KB), but fetches only the blocks that will likely be touched within a page. In doing so, the Footprint Cache achieves high hit rates with moderate on-chip tag storage and reasonable lookup latency. However, multi-gigabyte stacked DRAM caches will soon be practical and needed by server applications, thereby mandating tens of MBs of tag storage even for page-based DRAM caches. We introduce a novel stacked-DRAM cache design, Unison Cache. Similar to Alloy Cache's approach, Unison Cache incorporates the tag metadata directly into the stacked DRAM to enable scalability to arbitrary stacked-DRAM capacities. Then, leveraging the insights from the Footprint Cache design, Unison Cache employs large, page-sized cache allocation units to achieve high hit rates and reduction in tag overheads, while predicting and fetching only the useful blocks within each page to minimize the off-chip traffic. Our evaluation using server workloads and caches of up to 8GB reveals that Unison cache improves performance by 14 compared to Alloy Cache due to its high hit rate, while outperforming the state-of-the art page-based designs that require impractical SRAM-based tags of around 50MB. Die-stacking technology enables multiple layers of DRAM to be integrated with multicore processors. A promising use of stacked DRAM is as a cache, since its capacity is insufficient to be all of main memory (for all but some embedded systems). However, a 1GB DRAM cache with 64-byte blocks requires 96MB of tag storage. Placing these tags on-chip is impractical (larger than on-chip L3s) while putting them in DRAM is slow (two full DRAM accesses for tag and data). Larger blocks and sub-blocking are possible, but less robust due to fragmentation. This work efficiently enables conventional block sizes for very large die-stacked DRAM caches with two innovations. First, we make hits faster than just storing tags in stacked DRAM by scheduling the tag and data accesses as a compound access so the data access is always a row buffer hit. Second, we make misses faster with a MissMap that eschews stacked-DRAM access on all misses. Like extreme sub-blocking, our implementation of the MissMap stores a vector of block-valid bits for each "page" in the DRAM cache. Unlike conventional sub-blocking, the MissMap (a) points to many more pages than can be stored in the DRAM cache (making the effects of fragmentation rare) and (b) does not point to the "way" that holds a block (but defers to the off-chip tags). For the evaluated large-footprint commercial workloads, the proposed cache organization delivers 92.9 of the performance benefit of an ideal 1GB DRAM cache with an impractical 96MB on-chip SRAM tag array. This work efficiently enables conventional block sizes for very large die-stacked DRAM caches with two innovations: it makes hits faster with compound-access scheduling and misses faster with a MissMap. The combination of these mechanisms enables the new organization to deliver performance comparable to that of an idealistic DRAM cache that employs an impractically large SRAM-based on-chip tag array.