1. An image processing device comprising: an imaging unit that shoots a subject and generates a plurality of RAW image data; a gradation conversion unit configured to perform gradation conversion on a plurality of color signals of the plurality of RAW image data of continuously shot images according to a shooting condition for each of the plurality of color signals based on a pixel array of the imaging unit; and a compression processing unit configured to determine a number of compressed frames based on the shooting condition, and to perform compression processing utilizing a correlation between a spatial direction and a temporal direction for each of the plurality of color signals on which the gradation conversion has been performed, wherein the gradation conversion unit performs the gradation conversion on one color signal from among the plurality of color signals according to the shooting condition for each of first and second signals relating to the one color signal based on the pixel array.