1. A super-resolution processor that performs super-resolution processing on an input image to generate an output image of a higher resolution than the input image, the super-resolution processor comprising: an N enlargement unit configured to generate an N-enlarged image by enlarging a size of the input image by a factor N, where N is a real number larger than 1; an M enlargement unit configured to generate an M-enlarged image by enlarging the size of the input image by a factor M, where M is a real number larger than 1; a high-pass filter unit configured to extract a high-frequency component of the M-enlarged image, as an M-enlarged high-frequency image; a patch extraction unit configured to extract an estimated patch of a predetermined size from the M-enlarged high-frequency image, the estimated patch being a region of the M-enlarged high-frequency image; and an addition unit configured to add the estimated patch to a processing target block of the predetermined size in the N-enlarged image, to generate the output image, wherein M is smaller than N.