1. An imaging apparatus comprising: an imaging element having two-dimensionally arranged imaging pixels that receive light of an image formed by an optical system and output pixel signals corresponding to the received light, and in which non-imaging pixels, different from the imaging pixels, are arranged amongst the imaging pixels; continuity detection means for detecting a direction of continuity of the pixel signals based on the pixel signals of the imaging pixels arranged around the non-imaging pixels; and calculation means for determining a pixel signal at a position of the non-imaging pixels by processing the pixel output of the imaging pixels arranged around the non-imaging pixels in response to the detected direction of continuity.