1. An image processing apparatus that performs three-dimensional processing for three-dimensional display on a plurality of images which includes a plurality of faces and has different points of view and generates an image for three-dimensional display which is three-dimensionally displayed on a display unit for three-dimensional display, comprising: a face detecting unit that detects a face in the image; a distance calculating unit that calculates the distance of each face in a depth direction on the basis of the amount of parallax between the plurality of images; a main object determining unit that determines a face disposed at an intermediate distance, serving as a face with the highest priority, to be a main object when three or more faces are detected in the image and are arranged in order of distance; and a three-dimensional processing unit that performs the three-dimensional processing on the plurality of images such that the parallax of the main object is the minimum, thereby generating the image for three-dimensional display; wherein the main object determining unit includes a priority adjusting unit that adjusts the priority; wherein a three-dimensional effect evaluation value for each face is determined on the basis of a three-dimensional effect evaluation function having a point that is one-third of the distance from a front of the image to a rear of the image in the depth direction as the vertex; and wherein a face having a maximum three-dimensional effect evaluation value is used as the face with the highest priority and is determined to be the main object.