1. An intermediate image generation apparatus, comprising: a non-transitory computer readable recording medium storing a program and a hardware processor executing the program to operate as: a generation unit to generate an intermediate image using one image from among a left image and a right image, and by referring to a disparity map of the one image; and an interpolation unit to interpolate the intermediate image by referring to the other image from among the left image and the right image, which was not used to generate the intermediate image, wherein the interpolation unit comprises: a block setting unit to set, when an occlusion region exists in the intermediate image, a reference block including a non-occlusion region and located at an adjacent area of the occlusion region in the intermediate image, the reference block having a predetermined size; a retrieval unit to retrieve a sub-reference block in the other image matching the non-occlusion region from the reference block; and a region complement unit to complement the occlusion region in the intermediate image by applying a complementary block to the occlusion region, the complementary block being obtained directly from the other image and including the sub-reference block and an adjacent area of the sub-reference block, wherein the adjacent area of the sub-reference block in the complementary block is larger than the sub-reference block; wherein the adjacent area of the sub-reference block in the complementary block is used to complement the occlusion region, and the resolution of the complemented occlusion region is maintained with a resolution of the sub-reference block, thereby blurring effect of the intermediate image is prevented.