1. An image processing apparatus comprising: a processor; and a memory including instructions that, when executed by the processor, cause the processor to: perform an interpolation process in each of a plurality of defined directions on a mosaic image to output interpolated images having undergone the interpolation process for each pixel, wherein the mosaic image is generated by an image pickup device which outputs an image signal corresponding to one of colors of a plurality of color filters for each pixel; acquire an evaluation value representing a correlation for each of the defined directions for each of pixels; correct the evaluation values for at least one pixel of interest based on the evaluation values for peripheral pixels of the at least one pixel of interest in a case where the evaluation values for the at least one pixel of interest and the evaluation values for the peripheral pixels satisfy a predetermined condition; and output an image signal for each pixel based on the corrected evaluation values and the interpolated images.