1. A method of annotating a video stream comprising a sequence of frames, the method comprising: scanning a first frame in the sequence of frames for a person; adding, to a video track, a representation of a region of interest of the first frame likely to contain an image of the person; scanning a next frame in the sequence of frames for a person, beginning at a location in the next frame based on a location of the region of interest of a preceding frame; adding, to the video track, a representation of a region of interest of the next frame; and assigning a label to the video track based on whether at least one of the regions of interest in the video track contains an image of a particular person.