1. A computer-implemented method for video annotation of a video sequence that includes a plurality of frames, the method comprising: storing, by a computer, information representative of a bounding box around a given target object in the video sequence in both a first frame and a last frame of the video sequence that include the target object; generating, by the computer based on the information, estimated box annotations of the given target object in suggested frames from among the plurality of frames, the suggested frames being determined based on an annotation uncertainty measure calculated using a set of already provided annotations for at least some of the plurality of frames together with optical flow information for the video sequence; and displaying, by the computer, various ones of the estimated box annotations of the given target object in the suggested frames.