One of the main contributing factors to the expansion of DNNs has been the introduction of more powerful CPUs and GPUs. However, as DNNs become deeper and more complex, their large memory footprint results in on-chip and off-chip memory bottlenecks. Indeed, the extreme data-intensive nature of DNNs mandates the design of computers that are efficient in storing, accessing and processing large amounts of data. To improve efficiency and increase throughput, low-precision number formats for weights and/or inputs can be adopted (e.g., from 64/32-bit floating point to 8/4-bit integer). The most extensive examples are the Google Tensor Processing Units (TPUs) and the tensor cores of new NVIDIA GPUs [1]}, [2]}. Yet, despite these techniques, the time and energy for accessing the data still dominate over the data processing, even in the more aggressive technology nodes [1]}. Data has to be brought from the memory to the processing unit (CPU or accelerators such as GPUs), and the result should be written back to the memory after processing.
