1. An image processing apparatus which generates enlarged image data by increasing the number of pixels of original image data in accordance with an enlargement ratio, said apparatus comprising: a setting unit configured to set the enlargement ratio; an enlargement unit configured to enlarge the original image data in accordance with the set enlargement ratio; a smoothing unit configured to smooth the enlarged image data generated by said image enlargement unit by using a smoothing filter having a size corresponding to the set enlargement ratio; a difference image generation unit configured to generate difference image data by calculating a difference between the enlarged image data and smoothed image data obtained by said smoothing unit; a multiplying unit configured to generate a product to multiply said difference image data generated by said difference image generation unit by an emphasis coefficient based on a value of each pixel of said difference image data; an addition unit configured to add said enlarged image data generated by said enlargement unit and said product, and wherein said multiplying unit multiplies a larger emphasis coefficient as a larger value of each pixel of said difference image data.