1. An image processing apparatus which generates enlarged image data by increasing the number of pixels of original image data in accordance with an enlargement ratio, said apparatus comprising: a setting unit adapted to set the enlargement ratio; an enlargement unit adapted to enlarge the original image data in accordance with the set enlargement ratio; a smoothing unit adapted to smooth the enlarged image data generated by said image enlargement unit by using a smoothing filter having a size corresponding to the set enlargement ratio; a difference image generation unit adapted to generate difference image data by calculating a difference between the enlarged image data and smoothed image data obtained by said smoothing unit; and an output image generation unit adapted to generate output enlarged image data on the basis of the difference image data generated by said difference image generation unit and the enlarged image data obtained by said enlargement unit.