BACKGROUND

Field of the Invention

The methods and systems of asynchronous collaboration relate to the technical field of interactive digital video.

Description of the Related Art

Many video platforms include functionality for users to comment on shared videos. Most commonly, user comments are uploaded in text format and associated with a particular video. In some streaming video platforms, user comments are associated with a particular time in the video, as it plays, at which the comments are received.

SUMMARY

Disclosed herein are systems and methods for asynchronous multi-modal collaboration, such as commenting, associated with a video. This collaboration may be associated with particular frames in the video, pixel locations in one or more frames in the video, as well as objects that may be linked to the one or more pixel locations. A user may be enabled to create an adapted version of a video, such as by incorporating collaborative elements such as comments in text, audio and video formatted content, graphical augmentations, links within and external to the video, changes in playback speed and direction of the video, conditional action elements that may impact playback and the like. A set of collaborative elements, which may be related to each other by time locality, spatial locality, or other common feature or theme, may be grouped together and considered as a multi-modal collaboration. Users may be enabled to share such adapted videos for use and collaboration by other users as well as respond to individual collaboration elements or to multi-modal collaboration.

In embodiments, systems and methods for asynchronous multi-modal collaboration extend the general concept of text remarks associated with the whole video, to multimedia comments associated with frames/pixels of the video, along with an interface that provides user interface elements making it easy to enter multimedia comments, navigate between comments, respond to comments, and resolve comments (each multimedia comment may consist of text, audio, video, changes in playback speed/direction and/or graphical augmentations, and the like).

In embodiments, the methods and systems of asynchronous video collaboration described herein may facilitate establishing a channel of communication between parties (e.g., a player and a remote coach) that the two parties could use for collaboration, such as by exchanging comments, augmentations, dialog, video, interactive content, and the like.

The examples in this section are merely representative of some possible embodiments but do not reflect all possible embodiments, combinations of elements, or inventions disclosed in this application. In an example, a method may include presenting, in an electronic user interface, a base video and responsive to receiving an indication of a collaboration action in the electronic user interface, identifying at least one of a frame of the base video and a region within a frame of the base video that is presented in the user interface contemporaneously with the receiving an indication. The method may further include generating a collaboration for the base video based on a type of the collaboration action and a duration of the collaboration action and producing a collaboration video that includes the base video and the generated collaboration that commences at the identified at least one of a frame of the base video and a region of a frame of the base video and lasting for a duration based on the duration of the collaboration action. In one example, the type of collaboration may be one of playback manipulation, an in-line visual insertion, a visual overlay, a visual replacement of an object identified in the region, a picture in a picture window, a visual text comment, or an audio comment. In one example, the duration of the collaboration action may be determined based on a duration of the indication of the collaboration action in the electronic user interface. In embodiments, generating a collaboration for the base video may include combining at least a portion of a second video with the base video based on the type of collaboration action. A portion of a second video may be automatically selected based on computer vision-based analysis of the region of the frame. The computer vision-based analysis may determine at least one object in the region of the base video and a corresponding object in the second video. In embodiments the method may further include replacing the at least one object in the region of the based video with the corresponding object in the second video thereby producing a collaboration video.

In an example, a method may include receiving an indication of a of a frame of a video, receiving one or more collaborations for the frame, determining collaboration elements, determining video frame elements in the frame, generating, based on the collaboration elements and the video frame elements, a span criteria for evaluating frames of the video, and searching for a span of the video for which the frames meet the span criteria. Determining the video frame elements may include searching for the collaboration elements in the frame. Determining the collaboration elements may include searching for the video frame elements in the collaborations. The collaboration elements may include at least one of words, phrases, lines, sounds, or shapes. The video frame elements include at least one of objects, people, colors, video quality, video motion, or actions. The method may further include determining a video frame element score based on a position of the elements in the frame and wherein the span criteria comprises a criteria based on the video frame element score. In some embodiments, the method may also include determining a video frame element score based on a size of the elements in the frame and wherein the span criteria comprises a criteria based on the video frame element score. The span of the video may include a gap where the frames do not meet the span criteria. The method may also include identifying keywords in the one or more collaborations, corroborating the keywords in the span based at least in part on motion of objects in the frames of the span, and generating a recommendation for a playback configuration of the span. In some cases, the recommendation may include at least one of a playback at a reduced speed, a playback at an increased speed, or a repetition of a playback of the span. Searching may include searching away from the frame and searching in one direction from the frame. In some cases, the span criteria may be different for a forward search direction and a backward search direction from the frame.

