1. An image processing apparatus for playing back a moving image including a plurality of time-divided frame images so as to reflect a human spatial visual characteristic and temporal visual characteristic, the apparatus comprising: an object extraction unit configured to extract object images from a frame image of interest in the plurality of frame images; an object adaptation time acquisition unit configured to acquire an object adaptation time of each of the extracted object images in a predetermined processing period up to the frame of interest; an object adaptation feature amount calculation unit configured to calculate an object adaptation feature amount of each object image based on the object adaptation time; and a color conversion unit configured to perform, for the frame image of interest, color conversion based on the object adaptation feature amount of each object image.