1. An image processing apparatus comprising: an information acquisition section configured to acquire information of an observation position and a visual line direction for observing an inside of a subject by using an objective optical system; a boundary position generation section configured to generate boundary position information indicating a boundary position of an angle-of-view region and another region of the objective optical system on a three-dimensional image of the subject based on the information of the observation position and the visual line direction that change along with temporal change observation and based on angle of view information indicating an angle of view in observation using the objective optical system; a non-angle-of-view region extraction section configured to extract a non-angle-of-view region not in the angle of view on the three-dimensional image based on the boundary position information generated by the boundary position generation section; and an image generation section configured to generate an identification image for a luminal organ model based on the three-dimensional image, the identification image being provided with identification information for identifying the non-angle-of view region.