1. An image processing apparatus comprising: an input unit configured to input an output of an image pickup element having ordinary pixels arranged in a horizontal direction and a vertical direction and functional pixels arranged discretely between the ordinary pixels; a first estimating unit configured to perform a gain correction operation to an output of a functional pixel, thereby estimating an image signal at a position of the functional pixel; a second estimating unit configured to select a plurality of reference pixels from the ordinary pixels around the functional pixel and estimate the image signal at the position of the functional pixel on the basis of outputs of the plurality of selected reference pixels; and a determining unit configured to determine the output of the image signal at the position of the functional pixel on the basis of an output of the first estimating unit and from one or a plurality of estimation results which are obtained by the second estimating unit on the basis of combinations of the plurality of reference pixels.