1. An image processing apparatus comprising: an interpolation unit adapted to carry out an interpolation process in which, with respect to a color mosaic image signal related to each pixel generated by photoelectrical conversion of an optical image reflecting a subject image, signals of missing colors in the each pixel are interpolated using image signals related to pixels in predetermined directions around the each pixel, wherein said interpolation unit comprises: a direction-by-direction interpolation unit that carries out the interpolation process in each of a plurality of predetermined directions; an evaluation unit that evaluates color image signals in the predetermined directions subjected to the interpolation process by the direction-by-direction interpolation unit; and a generation unit that generates a color image signal to be outputted as an interpolation process result based on a result of the evaluation by the evaluation unit, wherein the evaluation unit evaluates the color image signals in the predetermined directions subjected to the interpolation process with respect to homogeneity and signal intensity.