1. A method for estimating orientation of cameras mounted on vehicles, comprising: determining, by one or more processors, an initial pose estimate of a vehicle having a camera mounted thereon, the initial pose estimate being a position and orientation of the vehicle relative to the world; processing, by the one or more processors, image data from images captured by the camera to obtain one or more tracks, each track including a sequence of matched feature points stemming from a single three-dimensional location; and determining, by the one or more processors, from the initial pose estimate and the tracks, a correction parameter to correct orientations of the images captured by the camera, wherein determining the correction parameter includes evaluating a set of vehicle poses, three-dimensional locations from the one or more tracks, and an orientation of the camera relative to one or more sensors of the vehicle.