1. A camera calibration device comprising: a normal vector acquisition unit which acquires normal vectors perpendicular to a reference horizontal plane from an image of a camera to be calibrated; and a rotation matrix estimation unit which projects the acquired normal vectors on a projection virtual plane perpendicular to the reference horizontal plane and evaluates that the projected normal vectors are perpendicular to the reference horizontal plane thereby to estimate a rotation matrix employed for calibrating the camera.