1. An image processing apparatus comprising: a holding unit configured to hold a filter to be used to reduce a blur at an image height position for each image height on a line from a center position of an image to an end of the image; a read unit configured to read out the filter from said holding unit; and a process unit configured to perform image quality recovery processing of reducing the blur in the image using the filter read out by said read unit, said process unit comprising: a lay-out unit configured to lay out a filter of interest to be laid out at a position of an image height of interest on the line so as to reduce a blur at the position, the filter of interest being rotated about the center position of the image by a filter layout angle; and a unit configured to apply the filter of interest laid out by said lay-out unit to the position of the image height of interest on a line obtained by rotating the line about the center position of the image by the filter layout angle, thereby performing the image quality recovery processing for the position.