1. An image processing device comprising: a shadow base calculating unit configured to calculate, from a registration image representing a photographed object and three-dimensional shape data in which respective points of a three-dimensional shape of said photographed object are correlated with pixels of said registration image, by assuming uniform albedo, a shadow base vector group having components as vector values from which an image under an arbitrary illumination condition can be generated through a linear combination; a perfect diffuse image estimating unit configured to estimate a shadow in said registration image with using said shadow base vector group and generate a perfect diffuse component image as an image including said shadow; and a highlight removing unit configured to generate, based on said perfect diffuse component image, a highlight removal image in which a specular reflection component is removed from said registration image.