1. A method for correcting for image distortion during camera motion using a system including a processor and a camera having a rolling shutter, the method comprising: representing the camera motion by an ego-motion matrix; capturing a plurality of image frames by the camera equipped with the rolling shutter, wherein the captured image frames include a base image frame and a previous image frame; recording a plurality of time stamps respectively for a plurality of corresponding image points in the previous and base image frames; and computing a plurality of ego-motions between capture times of the corresponding image points by correcting the ego-motion matrix responsive to the time stamps of the corresponding image points of the base image frame and the previous image frame thereby correcting for the image distortion caused by the rolling shutter.