1. A camera calibration device comprising: a normal vector acquisition unit which acquires normal vectors perpendicular to a reference horizontal plane as a line segment on an image from the image of a camera to be calibrated; and a rotation matrix estimation unit which projects the acquired normal vectors on a projection virtual plane perpendicular to the reference horizontal plane and evaluates that the projected normal vectors are perpendicular to the reference horizontal plane thereby to estimate a rotation matrix employed for calibrating the camera; a translation matrix estimation unit which estimates a translation matrix employed for calibrating a camera, wherein the normal vector acquisition unit acquires height information on the heights of the end points of at least one normal vector from among the acquired normal vectors, and the translation matrix estimation unit projects one end point of a normal vector the height information of which is acquired in the real space by use of a rotation matrix estimated by the rotation matrix estimation unit thereby to calculate a 3D coordinate in the real space, calculates a 3D coordinate of the other end point paired with the one end point by use of the calculated 3D coordinate and the height information, re-projects the calculated 3D coordinate of the other end point on the image by use of the rotation matrix thereby to calculate a coordinate of the other end point on the image, and estimates the translation matrix such that an error between the coordinate of the other end point re-projected on the image and the coordinate of the other end point directly acquired from the image is minimum.