1. A stereoscopic image taking apparatus comprising: a single photographing optical system; an imaging device on which pupil-split object images which have passed through a first region and a second region of the photographing optical system, are formed, respectively, the first region and the second region being positioned differently in a predetermined direction, the imaging device configured to perform photoelectric conversions to an object image which has passed through the first region and an object image which has passed through second region to output a first image and a second image, respectively; and an image processing unit configured to perform a first image processing to the first image and to perform a second image processing different from the first image processing to the second image, the image processing unit configured to perform the first image processing and the second image processing so as to reduce a difference in image quality between the first image and the second image which have been processed.