1. A method comprising: receiving, by an image classification system, an input image having a first resolution; down-sampling, by the image classification system, the input image to generate a second image having a second, lower resolution; processing, by the image classification system, the second image using a low-resolution neural network to generate a plurality of features of the second image and a plurality of first scores, wherein each first score corresponds to a respective object category from a predetermined set of object categories, and wherein each first score represents a respective likelihood that the second image contains an image of an object belonging to the corresponding object category; selecting, by the image classification system, an initial, first-resolution patch of the input image, comprising generating coordinates of a reference point in the input image by processing the plurality of features of the second image and the first scores using a patch locator neural network and selecting a patch of the input image that is centered at the reference point as the initial, first-resolution patch of the input image; processing, by the image classification system, the initial patch of the input image using an initial patch neural network to generate a plurality of second scores, wherein each second score corresponds to a respective object category from the predetermined set of object categories, and wherein each second score represents a respective likelihood that the initial patch contains an image of an object belonging to the corresponding object category; and generating, by the image classification system, a respective third score corresponding to each object category from the predetermined set of object categories from the first scores and the second scores, wherein each third score represents a respective likelihood that the input image contains an image of an object belonging to the corresponding object category.