1. A method for restoration of a building structure using an infinity homography calculation method based on parallelograms, comprising: restoring the structure of a building and cameras on an affine space by using an infinity homography; and transforming said affine space to a metric space by using intrinsic parameters of said respective cameras or constraints on a space recognized in said respective images, wherein said restoring of a building structure further includes: choosing one of said images as a reference image; defining a camera equation on an affine space for a reference camera taking said chosen image; defining a camera equation for an arbitrary camera using an infinity homography between an image taken by said reference camera and another taken by said arbitrary camera other than the reference camera; defining an equation for a relationship among points that need to be restored on said affine space and said respective points that are projected to the ith camera; and restoring said respective cameras and building structure on an affine space by calculations in said camera equations and said defining equations.