FIELD

The present invention relates to a method and system to determine the location and/or orientation of a head, for example, a head of a driver of a vehicle.

BACKGROUND

A number of applications including driver monitoring systems, DMS, are concerned with determining a location and orientation for a head being imaged by a camera. In a DMS, this information can be used to assist in safety-critical systems such as intelligent air-bag deployment.

Known DMS systems are already quite sophisticated in determining the head-pose from a near infra-red (NIR) camera using advanced image analysis and deep learning techniques.

Some such systems can use anthropometric data, such as estimated eye separation distance, for a region of interest comprising a face being tracked for example, as disclosed in PCT Application WO 2016/091545 (Ref: FN-399), to determine a distance to the face.

It is however challenging to provide an accurate distance in this manner because of the variability of human anatomical features. J. S. Pointer, “The interpupillary distance in adult Caucasian subjects, with reference to ‘readymade’ reading spectacle centration” inOphthalmic Physiol. Opt.,2012 discloses that the spacing of the eyes for each individual can vary quite significantly, ranging from 55 mm to 75 mm. This variation introduces a depth scale ambiguity. Therefore, even with a large data sample that is representative of many people, is can be challenging to achieve an overall accuracy in absolute depth measurement of less than +/−5% variations.

Relative depth, that is the depth of the face relative to one or more points on the face, does not have this depth scale ambiguity problem. A relative depth map of a face can be accurately calculated in several ways from image data of a face. For example, M. Zollhöfer et al., “State of the art on monocular3D face reconstruction, tracking, and applications”, Comput. Graph. Forum,2018 discloses determining a 3D geometric shape of a head based on a sequence of 2D multi-view images of the head. A relative depth map may then be determined from the determined 3D geometric shape, the determined relative depth map being relative to the absolute dimensions of the subject's head. The determined relative depth map can be converted into an absolute depth map by comparision of anatomical feature dimensions to expected values. However, such a conversion will introduce a +/−5% error due to the above-mentioned variability of anatomical features. All individual pixels of the resultant absolute depth map will therefore exhibit a potential error of +/−5%. Given there is a distance of about 1 meter between the DMS and the subject this translates into a potential variation of +/−5 cm in the accuracy of an absolute depth map.

As such an error in the absolute depth map arises due to the inherent variability among humans, it is not possible to address this by increasing the size of the dataset of human head images. There will always be cases where a person with wide-set eyes in a broad face will be determined to be closer to the camera that they actually are; and correspondingly, a person with a narrow face and narrow-set eyes will be determined to be further from the camera than they actually are.

The present invention addresses at least some of the deficiencies in this known systems.

SUMMARY OF THE INVENTION

The present invention is defined by the independent claims. Further optional features are defined in the dependent claims.

The present invention relates to a method and system for producing an absolute depth map for a facial region being imaged by a camera, allowing the position and/or pose of the head relative to the camera to be determined and used for subsequent applications.

Embodiments can be employed in driver monitoring systems where the head comprises a driver of a vehicle and where the position and/or pose can be determined in real-time. An absolute depth measurement is used to enable accurate conversion of relative depth measurements determined from real-time analysis of image data from a user-facing camera.

Embodiments include a distance sensing sub-system which may comprise at least one of a short-range radar element, an ultrasonic distance measuring element, and a non-visible wavelength laser distance measuring element. Additionally or alternatively, the distance sensing sub-system may comprise a stereo camera formed from a user-facing camera and an additional offset user-facing camera.

Embodiments can be implemented in a system adapted to receive information relating to the image data from a user-facing camera and the at least one depth measurement, and to produce real-time images that report the position and pose of the head of a driver of a vehicle.

By using the method or system of the present invention, an error rate of <1% for individual pixels of an absolute depth map is achieved.

