1. An image processing apparatus which executes image processing on image data obtained by forming an image on an image pickup element, the image processing apparatus comprising a memory and a processor that execute a program to function as: an obtaining unit configured to obtain a pixel in the image data corresponding to a fixed position in the image pickup element as a specific pixel; a determination unit configured to determine a target region including a target pixel and a reference region including a reference pixel which are used for determining a degree of similarity between the target pixel and the reference pixel in accordance with the specific pixel obtained by the obtaining unit; and a calculation unit configured to calculate a degree of similarity between the target pixel and the reference pixel by comparing the target region and the reference region determined by the determination unit, wherein the determination unit sets the target region and the reference region such that the degree of similarity is calculated using pixels other than specific pixels included in the image data.