1. A stereoscopic panoramic image synthesis device comprising: an image acquisition device which acquires multiple images continuously captured at a predetermined frame rate while panning a single image capturing device including a single imaging optical system and an imaging element; an image selection device which selects images for panoramic image synthesis from the multiple images acquired by the single image capturing device, the image selection device selects images in which a horizontal component of a translation vector of an optical flow between the images is within a predetermined range set in advance; an overlap area detection device which detects an overlap area with disparity between temporally adjacent images in the selected images; a trimming device which cuts out images of the detected overlap area; a storage device which stores the cut images of the overlap area as a left image and a right image; a panoramic image synthesis device which synthesizes a left panoramic image and a right panoramic image based on an image group of the stored left image and an image group of the stored right image; a correspondence point detection device which sets a predetermined image among the acquired multiple images as a reference image, and which detects correspondence points having a same feature between the reference image and one or multiple different images captured at different time from that of the reference image; and an optical flow calculation device which calculates the optical flow based on the correspondence points detected by the correspondence point detection device, wherein the image selection device selects an image in which a horizontal component of a translation vector of the calculated optical flow is within a predetermined range set in advance, and sets the different selected image as a next reference image.