1. An image processing device comprising: a scaling unit configured to enlarge an input image represented by an input image signal to generate an enlarged image represented by an enlarged image signal; a gain calculating unit configured to calculate a gain for enhancing the enlarged image for each target pixel of the enlarged image, respectively, based on the input image signal; and an adding unit configured to add, for each target pixel of the enlarged image, the gain for the target pixel with a pixel value of the target pixel, respectively, wherein the gain calculating unit calculates the gain through a filtering process using a plurality of filters corresponding to an enlargement ratio of the scaling unit with respect to the input image, and filter coefficients of the plurality of filters are different from each other.