1. A method for retargeting a video comprising a plurality of frames, the method comprising: determining saliency information for the plurality of frames; determining a cost metric for the video, wherein the cost metric considers loss due to cropping, scaling, temporal factors, and spatial factors; and retargeting the video based on the saliency information and the cost metric to produce a retargeted video, wherein the retargeting makes use of a crop window for each frame in the plurality of frames.