1. An image processing device comprising: an image sequence acquisition section that acquires an image sequence that includes a plurality of constituent images, the image sequence being an image sequence for observing a target object; and a processing section that performs an image summarization process that deletes some of the plurality of constituent images included in the image sequence acquired by the image sequence acquisition section to generate a summary image sequence, wherein the processing section selects a reference image and a determination target image from the plurality of constituent images, detects an unnecessary area from the determination target image, the unnecessary area being an area in which the target object is not sufficiently captured enough to be observed, calculates a loss area based on deformation information about the reference image and the determination target image, and the unnecessary area, the loss area being an area that is lost when the determination target image is deleted, and determines whether or not the determination target image can be deleted based on the calculated loss area.