BACKGROUND

In image processing, super-resolution (SR) refers to a class of techniques for increasing the resolution of an imaging system. In a typical scenario, a SR system receives an input image and upscales the input image to a higher resolution output image. Prior to the upscaling, the details of the higher resolution output image are essentially unknown. The SR system operates by estimating the details of the higher resolution output image from characteristics of the input image.

There are various approaches for estimating a high-resolution (HR) image from its low-resolution (LR) image. For example, with the nearest neighbor approach, the color of a newly-created pixel in the HR image is determined based on the color of a nearest pixel in the LR image. With the bilinear and bicubic approaches, colors for newly-created pixels in the HR image are interpolated from colors of surrounding pixels in the LR image.

Other approaches estimate a HR image from its LR image using machine learning. For example, some approaches utilize convolutional neural networks (CNNs) to establish a mapping between LR image information and HR image information. CNNs are neural networks that include multiple layers, and use convolution in at least one of the layers. More specifically, CNNs include convolution layers that perform a convolution by sliding a filter, referred to as a convolution filter, over an input. As a simplified example, the input may be a 10×10 matrix of input values, and the convolution filter may be a 3×3 matrix of filter values. At each slide position, the convolution layer performs a matrix multiplication between the convolution filter and the portion of the input identified by the slide position. The portion identified by the slide position is referred to as the receptive field. The resulting sum is then added to a feature map at a position corresponding to the receptive field.

A Super-Resolution Generative Adversarial Network (SRGAN) is a machine learning system that uses two competing neural networks in order to generate synthetic SR images that appear to be real images. The two competing neural networks are referred to as a generator network and a discriminator network. The generator network is a CNN that receives a LR image as input and generates a HR image as output. The discriminator network is a separate CNN that is trained to distinguish SR images generated by the generator network from real images. During a training process, the generator network and the discriminator network can be optimized in an alternating manner, such that the generator network learns to generate SR images that are very similar to real images and, as such, difficult for the discriminator network to distinguish from real images. After sufficient training, the generator network can be used for SR.

SUMMARY

An example method for processing images includes receiving an input image. The example method includes providing the input image as input to a first module implemented on a computer processing system and configured to increase a resolution of the input image to produce a first upscaled image. The example method includes detecting, with the computer processing system, white pixels in the input image. The example method includes generating, with the computer processing system, a mask associated with the input image. The mask includes mask bits and one or more of the mask bits are set to mark the white pixels in the input image. The example method includes upscaling, with the computer processing system, the mask to produce an upscaled mask matching a resolution of the first upscaled image. The example method includes identifying, with the computer processing system, one or more target pixels of the first upscaled image that correspond to the one or more set mask bits in the upscaled mask. The example method includes modifying, with the computer processing system, the first upscaled image to produce an output image by replacing one or more of the target pixels of the first upscaled image with a replacement pixel having greater whiteness. The example method includes outputting, by the computer processing system, the output image.

An example system for processing images includes one or more computer storage devices configured to store a first module configured to increase a resolution of an input image to produce a first upscaled image. The example system includes one or more processors configured to execute instructions. The instructions cause the one or more processors to receive the input image. The instructions cause the one or more processors to provide the input image as input to the first module to produce the first upscaled image. The instructions cause the one or more processors to detect white pixels in the input image. The instructions cause the one or more processors to generate a mask associated with the input image. The mask includes mask bits and one or more of the mask bits are set to mark the white pixels in the input image. The instructions cause the one or more processors to upscale the mask to produce an upscaled mask matching a resolution of the first upscaled image. The instructions cause the one or more processors to identify one or more target pixels of the first upscaled image that correspond to the one or more set mask bits in the upscaled mask. The instructions cause the one or more processors to modify the first upscaled image to produce an output image by replacing one or more of the target pixels of the first upscaled image with a replacement pixel having greater whiteness. The instructions cause the one or more processors to output the output image.

One or more example non-transitory computer-readable storage media have computer-executable instructions stored thereon. When executed by one or more processors, the computer-executable instructions cause the one or more processors to receive an input image. The computer-executable instructions cause the one or more processors to provide the input image as input to a first module configured to increase a resolution of the input image to produce a first upscaled image. The computer-executable instructions cause the one or more processors to detect white pixels in the input image. The computer-executable instructions cause the one or more processors to generate a mask associated with the input image. The mask includes mask bits and one or more of the mask bits are set to mark the white pixels in the input image. The computer-executable instructions cause the one or more processors to upscale the mask to produce an upscaled mask matching a resolution of the first upscaled image. The computer-executable instructions cause the one or more processors to identify one or more target pixels of the first upscaled image that correspond to the one or more set mask bits in the upscaled mask. The computer-executable instructions cause the one or more processors to modify the first upscaled image to produce an output image by replacing one or more of the target pixels of the first upscaled image with a replacement pixel having greater whiteness. The computer-executable instructions cause the one or more processors to output the output image.

The features, functions, and advantages that have been discussed can be achieved independently in various embodiments or may be combined in yet other embodiments further details of which can be seen with reference to the following description and figures.

