BACKGROUND OF THE INVENTION

(1) Field of the Invention

The invention relates to systems and methods of targeting auxiliary content to a video stream at the edge of a distribution network, and more particularly to using a bit-vector block consisting of a multi-frame sequence of image bit-vectors linked to a target frame to synchronize auxiliary content to a video stream.

(2) Description of Related Art

In the field of broadcast communications, it is sometimes desirable to link auxiliary content to a broadcast stream of video at a location downstream of where the stream of video originated, i.e., at the edge of the distribution network, with a high degree of accuracy. For instance, in the field of virtual advertising images may be inserted into video streams so as to make them appear as if they were in the original scene. While the promotional image may be inserted at the origination point of the broadcast, it may be desirable to have targeted advertising in which the promotional image may be inserted downstream of the broadcast, preferably by an end user's viewing device. In this way, the inserted image may be selected to suit the individual user. This may, for instance, be accomplished by sending the keying, occlusion and tracking data used for making the virtual insertion to the end user device and have the virtual inserted by that device.

While such information may be included in the metadata accompanying a video stream, there is a problem in that the video stream and the metadata needed for tasks such as targeted advertisement insertion are typically managed by different entities. For instance, the video stream is typically supplied and managed by a first party such as, but not limited to, the content owner or service providing the video streaming, while the data required for virtual targeted advertising is typically supplied and managed by a second party such as, but not limited to a tech company providing the targeted advertising service.

If the metadata required for the auxiliary content is embedded in the video, those two separate technologies have to be integrated. This integration is not only technically and commercially challenging to manage, but would need to be redone for each content provider. The scope of the challenge may be seen in that, for instance, tens of thousands of YouTube channels are each a potential content provider. Redoing the integration of each of them would be difficult to implement.

The problem then becomes how to synchronize the auxiliary content with the video stream with single frame accuracy when the video and auxiliary content are provided separately.

The relevant prior art includes:

U.S. Pat. No. 5,543,856 issued to Rosser et al. on Aug. 6, 1996 entitled “System and method for downstream application and control electronic billboard system” that describes a system and method for authorizing, controlling, and monitoring the insertion of advertising indicia and other video images into a broadcast video stream by an Electronic Billboard system operated at a location remote from a video program origination location. The disclosed control means will be implemented in the form of a user key which is presupplied to an operator of the remote Electronic Billboard system, a broadcast key supplied to that operator at or near the time of transmission of the broadcast video stream, and a verification means operated in conjunction with the remote Electronic Billboard system to verify that those keys correspond to a predetermined criteria. Upon such verification, the remote Electronic Billboard system is enabled to operate on the broadcast video stream. The control means may also be applied for maintaining the integrity of either or both the products being advertised and the event being broadcast.

U.S. Pat. No. 11,004,456 issued to Harkness et al. on May 11, 2021, entitled “Methods, apparatus and articles of manufacture to provide secondary content in association with primary broadcast media content” that describes methods, apparatus, and articles of manufacture to provide media content are disclosed. Example apparatus disclosed herein include means for identifying primary media content presented on a first media presentation device. Disclosed example apparatus also include means for accessing a secondary content schedule associated with the primary media content. The secondary content schedule includes a plurality of secondary content items and time values associated with the secondary content items. The secondary content items are also associated with a loyalty value for a user of the first media presentation device. Disclosed example apparatus further include means for selecting one of the secondary content items to be accessed in response to a timestamp associated with the primary media content being between time values associated with the selected one of the secondary content items in the secondary content schedule.

U.S. Pat. No. 9,514,381 issued to Ludwigsen et al. on Dec. 6, 2016 entitled “Method of identifying and replacing an object or area in a digital image with another object or area” that describes a system and method for object and area detection and replacement in an image includes identifying an object or area in one or more sequential images that form a moving image sequence and replacing some or all of the identified object or areas with another image such that the image looks to be part of the original composition of the original image including lighting, shadows, placement, occlusion, orientation, position, and deformation.

Various implementations are known in the art, but fail to address all of the problems solved by the invention described herein. Various embodiments of this invention are illustrated in the accompanying drawings and will be described in more detail herein below.

BRIEF SUMMARY OF THE INVENTION

An inventive system and method of synchronizing auxiliary content to a video stream is disclosed.

In a preferred embodiment, a target frame may be identified in a video stream. A block of bit-vectors linked to the target frame may then be created. The block of bit-vectors may consist of a multi-frame sequence of image bit-vectors. Each bit-vector may, for instance, be a relatively short binary string uniquely identifying an image, or a feature in that image. Such a string may, for instance, be obtained by reducing the image to a small number of sub-units, say 64, and then making a binary comparison of a value of each subunit to an adjacent subunit or some metric of all the subunits. The result of such comparisons may then be a bit-vector consisting of 64 ones and zeros. The comparison may, for instance, be of average values, of median values or of frequencies to create an average bit-vector, a median bit-vector, and a frequency bit-vector respectively.

The video stream and block of bit-vectors may then be transmitted to an end user device. The video stream and the block of bit-vectors may be transmitted via the same transmission path, or they may be transmitted via different transmission paths.

The end user device may then use the block of bit-vectors to uniquely identify the target frame.

The target frame may then be used, for instance, to synchronize auxiliary content to the video stream. The auxiliary content may be resident on the end user device, or it may have been obtained from a third party, or it may have been sent along with the block of bit-vectors. The auxiliary content may be any audio-visual content such as, but not limited to, images, video, or audio. The auxiliary content may also or instead contain process instructions that may provide details of how to integrate the auxiliary content with the video stream.

The end user device may then display the synchronized video stream and said auxiliary content on said end user device.

In one exemplary use of the method, the target frame may be used for virtual insertion that may be performed at the end user device instead of at the point of origin of the broadcast. This may, for instance, be implemented by, at the point of origin, obtaining any necessary insertion data such as, but not limited to, insertion keying data, occlusion data and frame tracking data. That insertion data may be linked to the target frame and may be sent to the end user device along with the block of bit-vectors. The end user device may then use the block of bit-vectors to identify the target frame, and then use that and the insertion data to virtually insert auxiliary content into the video stream.

While the video stream may be compressed for transmission by a lossy compression method such as, but not limited to, MPEG, it may be desirable that the block of bit-vectors and insertion data be compressed by a lossless compression method such as, but not limited to, Huffman compression or Arithmetic compression.

Therefore, the present invention succeeds in conferring the following, and others not mentioned, desirable and useful benefits and objectives:

It is an object of the present invention to provide a means of synchronizing s video stream supplied by a first party with auxiliary content that may be provided a second party that may operate independently of the first party.

It is a further object of the present invention to provide a means of synchronizing video streams that is independent of time codes or other data that may be corrupted or removed during transmission.

