1. A method of analyzing video frames capturing a 3D scene over time to automatically generate a road map of the 3D scene comprising: detecting positions of objects in the video frames; estimating 3D transformation parameters for the objects; predicting heights of the objects based at least in part on the 3D transformation parameters; and estimating road boundaries of the 3D scene using the object positions to generate the road map.