Cache partitioning to serve multiple client processes (“clients”) in computing systems has several benefits. Partition size can vary to match performance requirements of the client. Clients experience robust performance since data in its private cache is not subject to evictions due to other clients' accesses. Idle partitions can be turned off to reduce power consumption. However, these benefits are not as great as they could be due to limitations imposed by the way current cache partitioning is accomplished.
Currently caches may be partitioned across cache ways only, which has a number of limitations. The cache partition number is limited by the number of cache ways, and adding associativity to the cache partitions to reduce cache conflicts reduces the maximum number of cache partitions possible. Minimum way partition size is determined by the associativity. For example, the smallest partition for an 8 MB cache with 8 ways that is 8-way associative is 1 MB. Also, smaller cache partitions could result in loss of cache space due to fragmentation. Using the same example, 512 KB of data stored in the smallest 1 MB cache partition would result in 512 KB of unused space in the cache partition. Minimum way partition power collapse granularity is determined by the associativity. In the same example, the minimum granularity that can be power collapsed is a 1 MB cache partition. The size of collapse granularity affects power savings/usage, especially if a shared system cache replaces multiple private memories that can be independently power collapsed.
Another problem faced in mobile system on chip (SoC) computing systems is the diverse access patterns generated by subsystems. For example, the multimedia subsystems, like video and graphics, have longer access times and benefit from longer cache lines, while the accesses from the applications central processing unit (CPU) are more random and sensitive to latency and thus benefit from smaller cache lines and higher associativity. Not all of the potential access patterns work well with current cache partitioning that relies on way cache partitioning. Conventional cache partitioning provides one-size caches for all applications and subsystems. The size of the cache partitions may benefit certain access patterns more than others. Another example is the producer-to-consumer access pattern seen between an image processing engine and a display engine, which may benefit from a first in first out (FIFO) replacement policy. However, a cache holding the transit data between the engines may use an inefficient replacement policy for the application.