1. An imaging device, comprising: a photoelectric conversion unit configured to convert optical signals received by a plurality of pixels to electrical signals; a first charge holding unit configured to accumulate the electrical signals obtained by the photoelectric conversion unit and hold the accumulated signals as charge signals; a second charge holding unit configured to accumulate the electric signals obtained by the photoelectric conversion unit with a different timing from accumulation by the first charge holding unit and hold the accumulated signals as charge signals; a difference calculation unit configured to calculate, for each of the plurality of pixels, a differential value between the corresponding charge signal held by the first charge holding unit and the corresponding charge signal held by the second charge holding unit and obtain differential signals based on the differential values; a multiple sampling unit configured to perform, on the differential signals obtained by the difference calculation unit, multiple sampling processing which is processing for sampling signals of pixels each located at a predetermined position from an original charge signal and adding up the sampled signals to generate a new signal; and an analog digital conversion unit configured to convert an output signal of the multiple sampling unit to a digital signal.