Supplementary File Second order Attention Network for Single Image Super resolution
In this supplementary file, we first give more details about the forward propagation (FP) and backward propagatio (BP) of covariance normalization. Then we provide the convergence analysis of some very deep networks to verify the effectiveness of the proposed non locally enhanced residual group (NLRG) structure. Meanwhile, we provide more comparison results with the state of the art CNN based SR methods under the Bicubic (BI) degradation model. All quantitative results are evaluated in terms of PSNR and SSIM metrics. 1. FP and BP of Covariance Normalization FP of Newton Schulz Iteration. Traditional convariance normalization relies heavily on EIG, which is however not well supported on GPU platform, thus leading to inefficient training. As explored in [7] we also relied on NewtonSchulz iteration to speed up the computation of covariance normalization. Given Y0 S,Z0 I, for n 1, N the Newton Schulz iteration is then updated alternately as follows: Yn 1 2Yn 1(3I Zn 1Yn 1) Zn 1 2 (3I Zn 1Yn 1)Zn 1. (1) After enough iterations, Yn and Zn quadratically converges to Y and Y 1. Such iterative operation is suitable for parallel implementation on GPU. In practice, one can achieve *The first two authors contribute equally to this work. SSCorresponding author: Lei Zhang +This work is supported in part by the National Natural Science Foundation of China under Grant 61771273, the R &D Program of Shenzhen under Grant JCYJ20180508152204044, and the research fund of PCL Future Regional Network Facilities for Large scale Experiments and Applications (PCL2018KP001) approximate solution with few iterations, e.g. no more than 5 iterations in our method. FP of Pre normalization. Since Newton Schulz iteration only converge locally, to guarantee the convergence, we pre normalize S first via