1. An image processing device which generates images for three-dimensional display of an object of interest from a plurality of transformation target images containing the object of interest, comprising: an image display unit which displays at least one transformation target image; a first reference point receiving/determining unit which receives information on first reference point candidates as candidates for first reference points serving for reference in input of geometric transformation according to a user operation, receives according to a user operation a determination signal targeted at the first reference point candidates displayed on the transformation target image based on the information on the first reference point candidates, and determines the first reference points based on the information on the first reference point candidates targeted by the received determination signal; a second reference point receiving/determining unit which determines second reference points serving for reference in output of the geometric transformation by receiving information on the second reference points; a geometric transformation unit which outputs a transformed image by conducting the geometric transformation to the transformation target image based on the first reference points determined by the first reference point receiving/determining unit and the second reference points determined by the second reference point receiving/determining unit; a camera position/posture estimating unit which estimates camera position/posture at the time of shooting in regard to each transformation target image; a first reference point candidate projecting unit which generates projected first reference point candidates in each transformation target image by projecting the first reference point candidates, represented by three-dimensional coordinates, onto the transformation target image based on the camera position/posture estimated by the camera position/posture estimating unit, and a processor configured to implement the first reference point receiving/determining unit, the second reference point receiving/determining unit, the geometric transformation unit, and the camera position/posture estimating unit, wherein: the first reference point receiving/determining unit receives information on the first reference point candidates represented by three-dimensional coordinates and a determination signal targeted at the projected first reference point candidates displayed on at least one transformation target image based on the information on the first reference point candidates, and the first reference point receiving/determining unit receiving the determination signal determines the projected first reference point candidates in each transformation target image, generated based on the information on the first reference point candidates as the projection source of the projected first reference point candidates targeted by the determination signal, as the first reference points in the transformation target image.