1. A computer-implemented method for selecting representative frames for videos, comprising: receiving a video including a set of frames; identifying a set of features for each of the frames of the video, the features including frame-based features and semantic features, the semantic features identifying likelihoods of semantic concepts being present as content in the frames of the video; generating a set of video segments for the video, each video segment including a chronological subset of frames from the video, and each frame associated with at least one of the semantic features; generating a score for each frame of the subset of frames for each video segment based at least on the semantic features; and selecting a representative frame for each video segment based on the scores of the frames in the video segment, the representative frame representing and summarizing the video segment.