1. An image processing device which matches a second image to a first image based on a shift vector, comprising: a matching degree acquisition unit adapted to acquire a degree of matching of corresponding points corresponding mutually between the first and second images; a texture estimation unit adapted to estimate a texture on the periphery of the corresponding points; and a shift vector weighting unit adapted to weight the shift vector based on the degree of matching and the estimation of the texture.