1. An image processing apparatus comprising: a memory which stores a plurality of first images obtained by photographing an object to be rendered from a plurality of different photographing directions, and second images that pertains to geometry information of the object to be rendered; a geometrical shape model generation unit which generates a geometrical shape model of the object to be rendered on the basis of the second images; a microfacet generation unit which generates a plurality of microfacets used to approximate a shape of the geometrical shape model; a billboarding processing unit which rotates the plurality of microfacets to make a predetermined angle with a view direction; and a texture mapping unit which generates a third image associated with the object to be rendered in correspondence with the view direction by selecting texture images for respective microfacets from the plurality of first images on the basis of the plurality of photographing directions and view direction, and by projecting the selected texture images onto the microfacets.