1. An image processing apparatus which executes image processing on image data obtained by forming an image on an image pickup element, the image processing apparatus comprising: an obtaining unit configured to obtain a pixel in the image data corresponding to a specific position in the image pickup element as a specific pixel; a determination unit configured to determine a target region including a target pixel and a reference region including a reference pixel which are used for determining a degree of similarity between the target pixel and the reference pixel in accordance with the specific pixel obtained by the obtaining unit; and a calculation unit configured to calculate a degree of similarity between the target pixel and the reference pixel by comparing the target region and the reference region determined by the determination unit.