Super Resolution (SR) is a fundamental and important low-level computer vision (CV) task. Different from traditional SR models, this study concentrates on a specific but realistic SR issue: How can we obtain satisfied SR results from compressed JPG (C-JPG) image, which widely exists on the Internet. In general, C-JPG can release storage space while keeping considerable quality in visual. However, further image processing operations, e.g., SR, will suffer from enlarging inner artificial details and result in unacceptable outputs. To address this problem, we propose a novel SR structure with two specifically designed components, as well as a cycle loss. In short, there are mainly three contributions to this paper. First, our research can generate high-qualified SR images for prevalent C-JPG images. Second, we propose a functional sub-model to recover information for C-JPG images, instead of the perspective of noise elimination in traditional SR approaches. Third, we further integrate cycle loss into SR solver to build a hybrid loss function for better SR generation. Experiments show that our approach achieves outstanding performance among state-of-the-art methods. With the marvelous achievement of deep learning (DL) in computer vision (CV), Super Resolution (SR) attracts much attention for its crucial value as the basis of many high-level CV tasks He et al., 2016) . Deep learning Super Resolution (DL-SR) algorithms (Kim et al., 2016; Lim et al., 2017; Haris et al., 2018; Zhang et al., 2018c; b) strive for finding the complex nonlinear mapping between low resolution (LR) images and their high resolution (HR) counterparts. However, the learned model only reflects the inverse of down-scaled mapping, which is used to obtain LR images from their HR fathers. In other words, if there are some spots/stains in LR inputs, the SR model will treat them as inherent elements, and the corresponding SR outputs will enlarge these undesirable details. In reality, on the Internet, JPG compression is probably the most commonly used pattern for storage space reduction. That is to say, the LR image will be further processed into a compressed JPG (C-JPG) image. The quality of C-JPG will greatly drop, and the compression may yield unpleasant artifacts, for example, the presence of obvious partition lines, which vastly deteriorates the overall visual feeling. Hence, directly solving high-level CV tasks with these C-JPG images will lead to poor performance. In this paper, we propose a lossless SR model to obtain images with satisfying quality from the low-quality inputs (C-JPG). The deterioration in C-JPG makes the SR processing a huge challenge. In this paper, we focus on the more realistic C-JPG SR problem. Many SR methods regarding to the real-world condition images have been already developed, such as Zhang et al. (2018a) ; Yuan et al. (2018) . Among them, some models regard the noise as a kernel estimating problem which can be solved by addictive Gaussian noises. However, the distribution of most real images are inconsistent with the hypothetical Gaussian distribution. Taking C-JPG images as a example, the image compression operation is related to decreasing information from original image instead of adding specific noises. Other models learn the related information from irrelevant LR-HR images to obtain similar representations by unsupervised strategy. All of them cannot solve the problem well. In general, most LR images are produced through performing traditional interpolation method (mostly bicubic) on their HR fathers. The SR training process should recover this down-scaled mapping in a reverse manner. Referring to our C-JPG SR issue, when searching images from Google, a lot of unpleasant details are displayed, especially in the edges of objects. However, the low quality of image makes former SR methods fail to generate applicable images. As shown in Fig. 1 , it is shown that the SR generations of traditional bicubic interpolation, leading SR algorithm RCAN, and RCAN with pre-denoising input all demonstrate poor quality with the low quality C-JPG inputs. Damaged grids are apparently enlarged by the approaches designed for traditional non-JPG datasets. More specialized analysis can be found in the research of Köhler et al. (2017) . Note that the image pairs with fixed down-scaled kernel have been successfully learnt by SR models, such as SRGAN (Ledig et al., 2017) , EDSR (Lim et al., 2017) , and RDN (Zhang et al., 2018c) . In this study, we deliberately build a more complicated dataset by adding JPG format LR images to the training data. To be specific, we have three kinds of training inputs: C-JPG LR, LR, and HR images. The whole training process includes two separate functional components: missing detail recuperative part (JPG recovering stage) and SR mapping learning part (SR generating stage). In order to remove ring, checkerboard effects, as well as other noise, the former half sub-model is trained with pre-processed C-JPG LR images as inputs, and original LR ones as the supervised information. The function of this stage is to recover LR image from its compression counterpart. Hence, the outputs (LR(C − JP G)) of the first part are greatly improved and free of partition lines phenomenon. Based on these improved LR images, the latter sub-model continues to learn the mapping between (LR(C −JP G)) and HR. Therefore, an integrated pipeline for SR representation between C-JPG and HR images is achieved through the jointly two sub-models. In short, there are mainly three contributions in this study: • Our research can be regarded as an universal SR method that generates SR images from C-JPG inputs, which is empirically proved to be more difficult than SR with non-JPG inputs. • We regard this specific SR task as a recovering information process for the inputs, compared with the former denoising assumption including down-sampling and degradation parts. In this viewpoint, a recovering model is firstly introduced to generate satisfied intermediates from C-JPG inputs. • We further propose an integrated SR model training pipeline with two-level data, i.e., C-JPG LR and LR images, as well as a new integrated loss function. The experimental results demonstrate our method can surpass traditional SR models. In this paper, we propose a lossless SISR model for low-quality C-JPG images which is extensive used on the Internet. Based on our redefined C-JPG SR pipeline, two functional stages are integrated to fulfill the SR task on C-JPG images. In addition, we employ cycle loss to guarantee the consistency after above two stages. The intensive experiments demonstrate that our model can learn capable representations of LR inputs for C-JPG SR task and outperform other cutting edges in SISR. More exploration should be executed on other CV tasks with C-JPG images inputs as the future work.
We solve the specific SR issue of low-quality JPG images by functional sub-models.
{"src": "scitldr-AIC_train", "paper_id": "r1l0VCNKwB"}