1. An imaging device, comprising: a photoelectric conversion unit including a plurality of pixels and configured to convert optical signals received by the plurality of pixels to electrical signals; a first charge holding unit configured to accumulate the electrical signals, as first accumulated electrical signals, obtained by the photoelectric conversion unit and hold the first accumulated electrical signals as first charge signals; a second charge holding unit configured to accumulate the electric signals, as second accumulated electrical signals, obtained by the photoelectric conversion unit with a different timing from accumulation by the first charge holding unit and hold the second accumulated electrical signals as second charge signals; a difference calculation unit configured to obtain differential signals based on differential values between the first charge signals and the second charge signals, by calculating, for each of the plurality of pixels, a differential value between a corresponding first charge signal held by the first charge holding unit and a corresponding second charge signal held by the second charge holding unit; a multiple sampling unit configured to perform, on the differential signals obtained by the difference calculation unit, multiple sampling processing which is processing for sampling signals of pixels each located at a predetermined position from an original charge signal and adding up the sampled signals to generate a new signal; an analog digital conversion unit configured to convert an output signal of the multiple sampling unit to a digital signal; and a processing switching unit configured to be capable of switching an input of the multiple sampling unit to, instead of the differential signal, the charge signal held by the first or second charge holding unit, wherein at a start of imaging, the processing switching unit sets, instead of the differential signal, the charge signal held by the first or second electric holding unit as an input of the multiple sampling unit.