Recent image super-resolution(SR) studies leverage very deep convolutional neural networks and the rich hierarchical features they offered, which leads to better reconstruction performance than conventional methods. However, the small receptive fields in the up-sampling and reconstruction process of those models stop them to take full advantage of global contextual information. This causes problems for further performance improvement. In this paper, inspired by image reconstruction principles of human visual system, we propose an image super-resolution global reasoning network (SRGRN) to effectively learn the correlations between different regions of an image, through global reasoning. Specifically, we propose global reasoning up-sampling module (GRUM) and global reasoning reconstruction block (GRRB). They construct a graph model to perform relation reasoning on regions of low resolution (LR) images.They aim to reason the interactions between different regions in the up-sampling and reconstruction process and thus leverage more contextual information to generate accurate details. Our proposed SRGRN are more robust and can handle low resolution images that are corrupted by multiple types of degradation. Extensive experiments on different benchmark data-sets show that our model outperforms other state-of-the-art methods. Also our model is lightweight and consumes less computing power, which makes it very suitable for real life deployment. Image Super-Resolution (SR) aims to reconstruct an accurate high-resolution (HR) image given its low-resolution (LR) counterpart. It is a typical ill-posed problem, since the LR to HR mapping is highly uncertain. In order to solve this problem, a large number of methods have been proposed, including interpolation-based (Zhang & Wu., 2006) , reconstruction-based (Zhang et al., 2012) , and learning-based methods (Timofte et al., 2013; Peleg & Elad., 2014; Schulter et al., 2015; Huang et al., 2015; Tai et al., 2017; Tong et al., 2017; Zhang et al., 2018a; Dong et al., 2016) . In recent years, deep learning based methods have achieved outstanding performance in superresolution reconstruction. Some effective residual or dense blocks Zhang et al., 2018b; Lim et al., 2017; Ledig et al., 2017; Ahn et al.; Li et al., 2018) have been proposed to make the network wider and deeper and achieved better results. However, they only pay close attention to improving the feature extraction module, ignoring that the upsampling process with smaller receptive fields does not make full use of those extracted features. Small convolution receptive field means that the upsampling process can only perform super-resolution reconstruction based on local feature relationships in LR. As we all know, different features interact with each other, and features which are in different regions have corresponding effects on upsampling and reconstruction of a certain region. That is to say that a lot of information is lost in the process of upsampling and reconstruction due to the limitation of the receptive field, although the network extracts a large number of hierarchical features which are from low frequency to high frequency. Chariker et al. (2016; show that the brain generates the images we see based on a small amount of information observed by the human eye, ranther than acquiring the complete data from the point-by-point scan of the retina. This process of generating an image is similar to a SR process. According to their thought, we add global information in SR reconstruction and propose to use relational reasoning to implement the process that the human visual system reconstructs images with observed global information. In general, extracting global information requires a large receptive field. A large convolution receptive field usually requires stacking a large number of convolutional layers, but this method does not work in the upsampling and reconstruction process. Because this will produce a huge number of parameters. Based on the above analysis, we propose an image super-resolution global reasoning network (SR-GRN) which introduces the global reasoning mechanism to the upsampling module and the reconstruction layer. The model can capture the relationship between disjoint features of the image with a small respective field, thereby fully exploits global information as a reference for upsampling and reconstruction. We mainly propose global reasoning upsampling module (GRUM) and global reasoning reconstruction block (GRRB) as the core structure of the network. GRUM and GRRB first convert the LR feature map into N nodes, each of which not only represents a feature region in the LR image, but also contains the influence of pixels in other regions on this feature. Then they learn the relationship between the nodes and fuse the information of each node in a global scope. After that, GRUM learns the relationship between the channels in each node and amplifies the number of channels for the upsampling process. And then they convert N nodes into pixels with global reasoning information. Finally, GRUM and GRRB complete the upsampling and reconstruction process respectively. In general, our work mainly has the following three contributions: • We propose an image super-resolution global reasoning network (SRGRN) which draws on the idea of image reconstruction principles of human visual system. We mainly focus on the upsampling module and the reconstruction module. The model reconstructs SR images based on relational reasoning in a global scope. • We propose a global reasoning upsampling module (GRUM) and global reasoning reconstruction block (GRRB), which construct a graph model to implement the relational reasoning among the feature regions in an image via 1D and 2D convolution, and finally adds the information obtained by global reasoning to each pixel. It can provide more contextual information to help generate more accurate details. • Our proposed GRUM and GRRB are lightweight, which makes it suitable for real life deployment. More importantly, GRUM and GRRB balance the number of parameters and the reconstruction performance well. They can be easily inserted into other models. In this paper, inspired by the process of reconstructing images from the human visual system, we propose an super-resolution global reasoning network (SRGRN) for image SR, which aims at completing the reconstruction of SR images through global reasoning. We mainly propose global reasoning upsampling module (GRUM) and global reasoning reconstruction block (GRRB) as the core of the network. The GRUM can give the upsampling module the ability to perform relational reasoning in a global scope, which allows this process to overcome the limitations of the receptive field and recover more faithful details by analyzing more contextual information. The GRRB also enables the reconstruction block to make full use of the interaction between the regions and pixels to reconstruct SR images. We exploit SRGRN not only to handle low resolution images that are corrupted by three degradation model, but also to handle real-world images. Extensive benchmark evaluations demonstrate the importance of GRUM and GRRB. It also indicates that our SRGRN achieves superiority over state-of-the-art methods through global reasoning.
A state-of-the-art model based on global reasoning for image super-resolution
{"src": "scitldr-AIC_train", "paper_id": "S1gE6TEYDB"}