1. An image processing apparatus, comprising: a calculating section which calculates a gradient covariance matrix for each of a plurality of points in an image based on neighboring pixel values around the point, and which calculates multiple eigenvalues for each of the gradient covariance matrices; a full feature point detection section for detecting, and identifying as a full feature point, a point whose gradient covariance matrix has a minimum eigenvalue that is more than a certain amount; a semi feature point detection section for detecting, and identifying as a semi feature point, a point whose gradient covariance matrix has a largest maximum eigenvalue among points whose gradient covariance matrices have a ratio of a maximum eigenvalue to a minimum eigenvalue that is more than a predetermined amount; a first tracking section for tracking the identified full feature point between a first image and a second image; a second tracking section for tracking the identified semi feature point; and an alignment section for aligning the first and second images based on a tracking result of the first tracking section and the second tracking section.