TECHNICAL FIELD

The present disclosure is directed at methods, systems, and techniques for enhancing the use of two-dimensional video analytics by using depth data.

BACKGROUND

Video analytics refers generally to the ability to automatically analyze video to determine properties or characteristics of that video. For example, video analytics may be used to automatically segment the foreground and/or background of a video; detect, track, or classify objects depicted in the video; or recognize faces of individuals in the video. Conventional two-dimensional video is limited in that it lacks explicit information regarding the depths of objects and environment depicted in the video.

SUMMARY

According to a first aspect, there is provided a method comprising: obtaining two-dimensional (2D) image data representing an image comprising a first object; obtaining depth data of a portion of the image that comprises the first object, wherein the depth data indicates a depth of the first object; generating an initial 2D classification of the portion of the image using the 2D image data without using the depth data; determining whether the initial 2D classification of the portion of the image is consistent with the depth data; and when the initial 2D classification of the portion of the image is consistent with the depth data, storing the initial 2D classification as an approved 2D classification.

The method may further comprise when the initial 2D classification is inconsistent with the depth data, rejecting the initial 2D classification.

Determining whether the initial 2D classification is consistent with the depth data may comprise: determining a 2D size of the first object using the 2D image data without using the depth data; comparing the 2D size of the first object to an expected size of the first object for the depth of the first object; and determining that the initial 2D classification is consistent with the depth of the first object when the 2D size of the first object is the expected size of the first object.

Determining whether the initial 2D classification is consistent with the depth data may comprise: determining by using the depth data whether the depth across the first object varies; and when the depth data indicates that the depth across the first object varies, storing the initial 2D classification as the approved 2D classification.

The depth data may indicate that the depth across the first object is constant, and the first object may be a shadow or a reflection.

The portion of the image represented by the depth data may further comprise an obstruction abutting the first object, the depth data may indicate the obstruction is nearer to a sensor that originated the depth data than the first object, and determining whether the initial 2D classification is consistent with the depth data may comprise: determining by using the depth data an unobstructed portion of the first object; determining a subsequent classification using the unobstructed portion of the first object and without using the obstruction; and when the subsequent classification and the initial 2D classification match, storing the initial 2D classification as the approved 2D classification.

The first object may comprise a first face, the initial 2D classification may be that the first face is of a person who interacted with a credentials reader, the portion of the image represented by the depth data may further comprise a second face, and determining whether the initial 2D classification is consistent with the depth data may comprise: determining, using the depth data, whether the first face or the second face is nearer to the credentials reader; and when the depth data indicates that the first face is nearer to the credentials reader than the second face, storing the initial 2D classification as the approved 2D classification.

The method may further comprise: when the depth data indicates that the second face is nearer to the credentials reader than the first face, storing as the approved 2D classification that the second face is of the person who interacted with the credentials reader.

The first object may comprise a first face, the portion of the image represented by the depth data further may comprise a second object comprising a second face, and determining whether the initial 2D classification is consistent with the depth data may comprise: determining, using the depth data and without using the 2D image data, that the second face is present in the portion of the image; and when the initial 2D classification classifies each of the first and the second objects as a face, storing the initial 2D classification as the approved 2D classification.

The initial 2D classification may be performed by a processor applying a first video analytics based method that causes the processor to determine the initial 2D classification as classifying the first object as a face and not the second object as a face, and the method may further comprise, when the initial 2D classification classifies the first object and not the second object as a face, guiding the first video analytics based method or applying a second video analytics based method to generate another 2D classification of the portion of the image that classifies each of the first and the second objects as a face.

The method may further comprise classifying, using the depth data without using the 2D image data, the portion of the image to identify fixed scene features, and determining whether the initial 2D classification is consistent with the depth data may comprise determining whether the initial 2D classification is consistent with the fixed scene features.

The first object may comprise at least part of a person, and determining whether the initial 2D classification is consistent with the fixed scene features may comprise: determining, from the fixed scene features, whether the portion of the image permits the person to be present; and when the portion of the image permits the person to be present, storing the initial 2D classification as the approved 2D classification.

According to another aspect, there is provided a method comprising: obtaining two-dimensional (2D) image data representing an image comprising a first object; obtaining depth data of a portion of the image that comprises the first object, wherein the depth data indicates a depth of the first object; generating an initial 2D classification of the portion of the image using the 2D image data without using the depth data; determining whether the initial 2D classification of the portion of the image is consistent with the depth data; and when the initial 2D classification of the portion of the image is consistent with the depth data, increasing a confidence level of the initial 2D classification.

Increasing the confidence level of the initial 2D classification may comprise storing metadata representing an increase in the confidence level of the initial 2D classification.

The method may further comprise when the initial 2D classification is inconsistent with the depth data, decreasing a confidence level of the initial 2D classification.

Decreasing the confidence level of the initial 2D classification may comprise storing metadata representing a decrease in the confidence level of the initial 2D classification.

According to another aspect, there is provided a method comprising: obtaining two-dimensional (2D) image data representing an image comprising a first object; obtaining depth data of a portion of the image that comprises the first object, wherein the depth data indicates a depth of the first object; classifying, using the depth data, the portion of the image to identify fixed scene features; and generating a 2D classification using the fixed scene features and the 2D image data.

According to another aspect, there is provided a method comprising: obtaining, using a first image sensor, a first two-dimensional (2D) image; obtaining, using a second image sensor co-located with the first image sensor and having an overlapping field of view with the first image sensor, a second 2D image captured concurrently with the first 2D image; and generating: a three-dimensional image by performing a stereoscopic combination of the first and second 2D images; and from the first and second 2D images, a third 2D image that has a higher resolution than a resolution of either the first and second 2D images.

The first image sensor may be a red green blue (RGB) image sensor and the second image sensor may be a monochrome image sensor.

Each of the first and second image sensors may be a monochrome sensor.

Generating the third 2D image may comprise interpolating pixels of the third 2D image from the first and second images.

According to another aspect, there is provided a memory device having stored thereon computer program code that is executable by a processor and that, when executed by the processor, causes the processor to perform the method of any of the foregoing aspects or suitable combinations thereof.

According to another aspect, there is provided a system, comprising: a camera configured to output two-dimensional (2D) image data representing an image comprising a first object and depth data of a portion of the image that comprises the first object, wherein the depth data indicates a depth of the first object; and a computing device, comprising: a processor communicatively coupled to the camera; and a memory device according to one of the foregoing aspects.

According to another aspect, there is provided a camera, comprising: a housing; a first lens and a second lens respectively focusing light on a first image sensor and a second image sensor located within the housing, wherein the first and second image sensors have overlapping fields of view; a processor located within the housing and communicatively coupled to the first and second image sensors; and a memory device according to one of the foregoing aspects.

This summary does not necessarily describe the entire scope of all aspects. Other aspects, features and advantages will be apparent to those of ordinary skill in the art upon review of the following description of specific embodiments.

