1. A phase difference detection device comprising: a photoelectric conversion unit in which a plurality of pixels for photoelectrically converting received light are arrayed, the photoelectric conversion unit being configured to photoelectrically convert at least a pair of optical images received through a lens so as to generate a first image signal and a second image signal; and a phase difference detection unit comprising: unit for obtaining first and second image signals from each of at least two pixel positions in the photoelectric conversion unit, comparison unit for comparing the first image signal with the second image signal for each pixel position, fluctuation calculation unit for calculating a fluctuation of first and second image signals, unit for comparing the calculated fluctuation over a range of pixel positions to obtain a resultant first image signal and a resultant second image signal, and phase difference calculation unit for calculating a phase difference between the resultant first and second image signals in order to enable the determination of a focus state of the lens on the basis of the calculated fluctuation.