1. An image processing apparatus comprising: a unit which extracts an area of an object from an image as an object area; a division unit which divides the object area into a plurality of meshes and obtains position information, gradient information, and color information of respective vertices which form the meshes; and a calculation unit which determines the position information, gradient information, and color information of the vertices by dividing each of the meshes into a plurality of small areas, and performing processing for changing the position information, gradient information, and color information of the vertices so as to locally minimize differences between color information within the small areas and color information within the object area corresponding to the small areas, wherein to divide each of the meshes into a plurality of small areas, the calculation unit changes the number of small areas or a size of each small area in accordance with the number of times of the changing processing that is the number of times of change in position information, gradient information, and color information of the vertices.