1. An image processing device comprising: a central processing unit (CPU) configured to: generate, based on an image of a current frame and an image of a preceding frame of the current frame, a first depth image indicating a position of a subject in a depth direction in the image of the preceding frame as a second depth image of the current frame, wherein a difference is detected between the image of the current frame and the image of the preceding frame, and wherein the second depth image is generated according to a frequency less than the frequency at which the first depth image of the preceding frame is determined, when the difference between the image of the current frame and the image of the preceding frame is equal to or less than a threshold value, and wherein the second depth image is generated according to a frequency that is the same as the frequency at which the first depth image of the preceding frame is determined, when the difference between the image of the current frame and the image of the preceding frame is greater than the threshold value.