The corresponding **data** shapes are shown in Table 5. Recall that cuDNN treats (N, P, Q, K, C, R, S) convolutions as implicit (NPQ, K, CRS) matrix multiplications. DeepSpeech  16 79  341 32  1  5 20 431024 100  Conv1  16 38  166 32  32  5 10 100928 1600  Conv2  OCR  16 24  240 32  16  3 3  92160  144  Conv3  16 12  120 64  32  3 3  23040  288  Conv4  Face Recognition  8  54  54  64  64  3 3  23328  576  Conv5  8  27  27  128  128  3 3  5832  1152  Conv6  16    The performance benefits of ISAAC (see Figure 9) are noticeable but not as large as they were for GEMM. This is because cuDNN was optimized from the ground up with both Maxwell and DeepBench-like problems in mind (Large NPQ, small K and intermediate CRS).
N P Q K C R S NPQ CRS Name
Nonetheless, we note substantial performance gains (1.5× to 2×) over cuDNN for the deep reductions found in Conv7 and Conv8.