1. A method for identifying scenes in captured video for inclusion in a video summary, the method comprising; accessing a video, the video including a plurality of frames; identifying a motion value associated with each frame of the plurality of frames; identifying a differential motion value associated with each frame of the plurality of frames based on the motion value associated with each of one or more of the frames; determining a score for each frame of the plurality of frames based on the motion value and differential motion value associated with each frame; selecting one or more frames in the video based on the determined scores; identifying, for each selected frame, a corresponding video scene, the video comprising a threshold amount of video occurring before and after the selected frame; and selecting one or more identified video scenes for inclusion in a video summary.