Selective GPU caches to eliminate CPU GPU HW cache coherence
Cache coherence is ubiquitous in shared memory multiprocessors because it provides a simple, high performance memory abstraction to programmers. Recent work suggests extending hardware cache coherence between CPUs and GPUs to help support programming models with tightly coordinated sharing between CPU and GPU threads. However, implementing hardware cache coherence is particularly challenging in systems with discrete CPUs and GPUs that may not be produced by a single vendor. Instead, we propose, selective caching, wherein we disallow GPU caching of any memory that would require coherence updates to propagate between the CPU and GPU, thereby decoupling the GPU from vendor specific CPU coherence protocols. We propose several architectural improvements to offset the performance penalty of selective caching: aggressive request coalescing, CPU side coherent caching for GPU uncacheable requests, and a CPU GPU interconnect optimization to support variable size transfers. Moreover, current GPU workloads access many read only memory pages; we exploit this property to allow promiscuous GPU caching of these pages, relying on page level protection, rather than hardware cache coherence, to ensure correctness. These optimizations bring a selective caching GPU implementation to within 93% of a hardware cache coherent implementation without the need to integrate CPUs and GPUs under a single hardware coherence protocol.
2016 IEEE International Symposium on High Performance Computer Architecture (HPCA)