1. A method for removing color noise on a slowly varying component contained in color difference component image data of image data which is imported from an image sensor and is converted to brightness component image data and the color difference component image data, comprising the steps of: determining whether it is necessary to perform a color noise removal process on the slowly varying component; when, in the determining step, it is determined that it is not necessary to perform the color noise removal process, sampling pixels of said color difference component image data by thinning out according to a first defined sampling format; when, in the determining step, it is determined that it is necessary to perform the color noise removal process, sampling pixels of said color difference component image data by thinning out according to a second defined sampling format, wherein the second defined sampling format is obtained by switching the sampling rate of the first defined sampling format such that the color difference component image data is produced in a small data volume corresponding to a compressed image data size smaller than image data size without said color noise removal process; and recording said color difference component image data and said brightness component image data.