1. One or more computer readable media having stored thereon a plurality of instructions to track faces from frame to frame of content, wherein the plurality of instructions, when executed by one or more processors, causes the one or more processors to: predict, using a plurality of cues, where a contour of a face will be in a frame; encode a smoothness constraint that penalizes roughness; apply the smoothness constraint to a plurality of possible contour locations; and select the contour location having the smoothest contour as the location of the face in the frame.