BACKGROUND

In various computer vision tasks, such as image classification, object detection, semantic segmentation, image captioning, visual question answering, and video understanding, an input image may be processed by a deep neural network. The input resolution of the image is a key factor in determining the memory requirements of the deep neural network during training and inference stages. In other words, the available computational memory determines the maximum input image resolution that can be processed. For example, depending on the width and depth of the neural network, the maximum possible input resolution may be limited (e.g., to below 1024×1024 pixels) even on state-of-the-art hardware, and even when using resource efficient neural networks.

Depending on the computer vision task being performed, the input image may need to be processed at different resolutions. For example, if the input image contains fine details that need to be analyzed, then the input image may need to be processed at a high resolution.

In some solutions, an input image is processed by a deep neural network by generating multiple resolutions of the entire input image. While such solutions can analyze small features, they have significant limitations. For example, each resolution of the entire image can require significant computing resources (e.g., memory, CPU, and/or time) to process, and it may not be possible or practical to process the entire image at higher resolutions.

Therefore, there exists ample opportunity for improvements in image processing technology using neural networks.

SUMMARY

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

Various technologies are described herein for performing adaptive high-resolution digital image processing using neural networks. For example, a number of different regions can be defined representing portions of a digital image. One of the regions covers the entire digital image at a reduced resolution. The other regions cover less than the entire digital image at resolutions higher than the region covering the entire digital image. Neural networks are then used to process each of the regions (e.g., one neural network operating on each region). The neural networks share information using prolongation and restriction operations. Prolongation operations propagate activations from a neural network operating on a lower resolution region to context zones of a neural network operating on a higher resolution region. Restriction operations propagate activations from the neural network operating on the higher resolution region back to the neural network operating on the lower resolution region.

For example, a process for adaptive high-resolution digital image processing using neural networks can comprise receiving a digital image, decomposing the digital image into a plurality of regions, performing a plurality of operations for a plurality of neural networks, and outputting results of the processing. The plurality of regions can comprise a first region that covers an entire area of the digital image, where the first region has a first resolution, and where the first resolution is a lower resolution representation of the digital image. The plurality of regions can also comprise one or more additional regions that cover less than the entire area of the digital image and that are contained within the first region, where the one or more additional regions have one or more additional resolutions that are higher than the first resolution. The plurality of neural networks can comprise a first neural network operating on the first region and at least a second neural network operating on the one the one or more additional regions. The operations for the plurality of neural networks can comprise prolongation operations that propagate activations from the first neural network onto context zones of the second neural network, where a context zone for a given region is outside of the given region, and restriction operations that propagate activations from the second neural network back to the first neural network.

