1. A method for identifying scenes in captured video for inclusion in a video summary, the method comprising; accessing metadata associated with a video, the accessed metadata representative of capture of the video; identifying a plurality of events of interest within the video based on the accessed metadata; for each identified event of interest, identifying a best scene in the video associated with the identified event of interest, the identified best scene comprising a threshold amount of video occurring before and after a video frame or portion corresponding to the identified event of interest; and selecting one or more identified best scenes for inclusion in a video summary.