1. A computer-implemented method for identifying key frames, comprising: receiving user access data for a video, the access data describing user interaction with the video; determining, from the user access data, an access interest level for each of a plurality of frames in the video; analyzing the content of the video to determine a content interest level for each of the plurality of frames in the video; determining an overall interest level for each of the plurality of frames based on the access interest level and content interest level; grouping the plurality of frames based on the overall interest levels of each of the plurality of frames to form a plurality of frame groups; identifying a plurality of key frame locations corresponding to one of the frames in each of the frame groups; and storing the plurality of key frame locations in a key frame database; for each key frame location, determining a key frame portion, the key frame portion including the key frame and additional frames of a portion length; for each key frame portion, determining a key frame portion interest level based on the overall interest levels of the key frame location and other frames in the key frame portion; and prioritizing the key frame portions for buffering by a client.