1. A method comprising: receiving, by an image manipulation application executed by a processing device, an input image; comparing, by the image manipulation application, a value for an attribute of at least one input patch of the input image to a threshold value, wherein the value for the attribute is indicative of at least one of (i) a similarity between the at least one input patch and an adjacent patch and (ii) a variance of features within the at least one input patch; based on comparing the value for the attribute to the threshold value, adjusting, by the image manipulation application, a first patch size of the at least one input patch to a second patch size, wherein the second patch size improves performance of a patch-based image enhancement operation as compared to the first patch size; and performing, by the image manipulation application, the patch-based image enhancement operation based on one or more input patches of the input image having the second patch size.