1. An image processing method for reconstructing an image of a view position and orientation from plural photographed images, comprising: using an imaging apparatus to perform steps including: a setting step of setting the view position and orientation of said reconstructed image; a first calculation step of calculating first light beam data of said reconstructed image based on the view position and orientation of said reconstructed image, wherein the first light beam data is data indicating a straight line according to the view position of said reconstructed image and a position of a pixel of the reconstructed image; an image pickup step of picking up a photographed image according to the view position and orientation at a time of photographing by using an image pickup unit of the imaging apparatus; a second calculation step of calculating second light beam data of the photographed image from the view position and orientation at the time of photographing, wherein the second light beam data is data indicating a straight line according to the view position at the time of photographing and a position of the pixel of the photographed image; an extraction step of extracting light beam data corresponding to the first light beam data of said reconstructed image from the second light beam data of the photographed image; a generation step of generating said reconstructed image by disposing data of a pixel of the photographed image corresponding to the extracted light beam data in a pixel corresponding to said reconstructed image; and a display step of displaying the image which was generated in said generation step and is in the course of reconstruction on a display unit of the imaging apparatus, wherein said image pickup step, said second calculation step, and said extraction step are repeated until said reconstructed image is generated, and wherein, in the display of said display step, the image which is in the course of reconstruction and a live image which is intended to be picked up in said image pickup step are switchable back and forth, wherein a decision is made, based on the display, whether or not to pick up the live image as the photographed image in said image pickup step.