BACKGROUND

Imaging devices are used in various types of systems and perform a variety of functions. Some imaging devices may be used for applications involving manufacturing, assembly, image analyses, and depth sensing, among others. Over time, the manner in which these imaging systems operate is becoming more robust, efficient, and intuitive. As imaging systems are integrated into numerous aspects of modern life, it is desirable for imaging systems to operate effectively under many different types of conditions. Therefore, a demand for robust and accurate imaging systems has led to advances in the various components that form imaging systems, as well as within the image processing techniques used to analyze and interpret data from images captured by imaging systems.

SUMMARY

Example embodiments are provided herein for disparity interpolation for stereo depth sensing. To determine the disparity between a target pixel in a first image and a given row in a second image, correlation scores may first be computed between the target pixel and pixels from the given row based on a first similarity measure. A candidate matching pixel for the target pixel may be identified based on these correlation scores. Additional correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel, and a right neighboring pixel may be computed based on a second similarity measure, which may be a more nuanced input feature than the first similarity measure. The candidate matching pixel may be confirmed to be the best-match pixel for the target pixel based on the additional correlation scores. A disparity value for the target pixel may then be computed based on an interpolation of the additional correlation scores.

In one example, a method is provided that includes receiving a first image from a first imager and a second image from a second imager. The method further includes determining first correlation scores between a target pixel from the first image and respective pixels from a given row of the second image based on a first similarity measure. The method additionally includes identifying a candidate matching pixel to the target pixel from the given row of the second image based on the first correlation scores. The method further includes determining second correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a second similarity measure. The method also includes determining whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores. After determining that the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores, the method further includes determining a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores. The method additionally includes providing the disparity value between the target pixel from the first image and the given row of the second image.

In a further example, a system is provided that includes a first imager, a second imager, and a control system configured to receive a first image from the first imager and a second image from the second imager. The control system may be further configured to determine first correlation scores between a target pixel from the first image and respective pixels from a given row of the second image based on a first similarity measure. The control system may additionally be configured to identify a candidate matching pixel to the target pixel from the given row of the second image based on the first correlation scores. The control system may also be configured to determine second correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a second similarity measure. The control system may be further configured to determine whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores. After determining that the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores, the control system may be further configured to determine a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores. The control system may also be configured to provide the disparity value between the target pixel from the first image and the given row of the second image.

In another example, a non-transitory computer readable medium is disclosed having stored therein instructions executable by one or more processors to cause a computing system to perform functions comprising receiving a first image from a first imager and a second image from a second imager. The functions may further include determining first correlation scores between a target pixel from the first image and respective pixels from a given row of the second image based on a first similarity measure. The functions may additionally include identifying a candidate matching pixel to the target pixel from the given row of the second image based on the first correlation scores. The functions may also include determining second correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a second similarity measure. The functions may further include determining whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores. The functions may also include after determining that the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores, determining a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores. The functions may further include providing the disparity value between the target pixel from the first image and the given row of the second image.

In a further example, a system is disclosed that includes means for receiving a first image from a first imager and a second image from a second imager. The system further includes means for determining first correlation scores between a target pixel from the first image and respective pixels from a given row of the second image based on a first similarity measure. The system additionally includes means for identifying a candidate matching pixel to the target pixel from the given row of the second image based on the first correlation scores. The system further includes means for determining second correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a second similarity measure. The system also includes means for determining whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores. After determining that the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the second correlation scores, the system further includes means for determining a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores. The system additionally includes means for providing the disparity value between the target pixel from the first image and the given row of the second image.

In an additional example, an example method is disclosed that includes receiving a first image from a first imager and a second image from a second imager. The method further includes providing the first image and the second image to stereo matching hardware. The method additionally includes receiving from the stereo matching hardware a candidate matching pixel to a target pixel of the first image from a given row of the second image. The method further includes determining correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a similarity measure. The method also includes determining whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the correlation scores. If the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel, the method additionally includes determining a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the correlation scores. If a neighboring pixel of the candidate matching pixel is better correlated to the target pixel than the candidate matching pixel, the method also includes determining a correlation score between the target pixel and a second neighboring pixel of the neighboring pixel that is opposite the candidate matching pixel based on the similarity measure, and determining the disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores for the neighboring pixel, the second neighboring pixel, and the candidate matching pixel. The method further includes providing the disparity value between the target pixel from the first image and the given row of the second image.

In a further example, a system is provided that includes a first imager, a second imager, and a control system configured to receive a first image from the first imager and a second image from the second imager. The control system may be further configured to provide the first image and the second image to stereo matching hardware. The control system additionally may be configured to receive from the stereo matching hardware a candidate matching pixel to a target pixel of the first image from a given row of the second image. The control system may be further configured to determine correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a similarity measure. The control system may also be configured to determine whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the correlation scores. If the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel, the control system additionally may be configured to determine a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the correlation scores. If a neighboring pixel of the candidate matching pixel is better correlated to the target pixel than the candidate matching pixel, the control system also may be configured to determine a correlation score between the target pixel and a second neighboring pixel of the neighboring pixel that is opposite the candidate matching pixel based on the similarity measure, and determine the disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores for the neighboring pixel, the second neighboring pixel, and the candidate matching pixel. The control system may be further configured to provide the disparity value between the target pixel from the first image and the given row of the second image.

In another example, a non-transitory computer readable medium is disclosed having stored therein instructions executable by one or more processors to cause a computing system to perform functions comprising receiving a first image from a first imager and a second image from a second imager. The functions may further include providing the first image and the second image to stereo matching hardware. The functions additionally include receiving from the stereo matching hardware a candidate matching pixel to a target pixel of the first image from a given row of the second image. The functions further include determining correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a similarity measure. The functions also include determining whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the correlation scores. If the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel, the functions additionally include determining a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the correlation scores. If a neighboring pixel of the candidate matching pixel is better correlated to the target pixel than the candidate matching pixel, the functions also include determining a correlation score between the target pixel and a second neighboring pixel of the neighboring pixel that is opposite the candidate matching pixel based on the similarity measure, and determining the disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores for the neighboring pixel, the second neighboring pixel, and the candidate matching pixel. The functions further include providing the disparity value between the target pixel from the first image and the given row of the second image.

In a further example, a system is disclosed that includes means for receiving a first image from a first imager and a second image from a second imager. The system further includes means for providing the first image and the second image to stereo matching hardware. The system additionally includes means for receiving from the stereo matching hardware a candidate matching pixel to a target pixel of the first image from a given row of the second image. The system further includes means for determining correlation scores between the target pixel and each of the candidate matching pixel, a left neighboring pixel of the candidate matching pixel, and a right neighboring pixel of the candidate matching pixel based on a similarity measure. The system also includes means for determining whether the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel based on the correlation scores. If the candidate matching pixel is better correlated to the target pixel than each of the left neighboring pixel of the candidate matching pixel and the right neighboring pixel of the candidate matching pixel, the system additionally includes means for determining a disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the correlation scores. If a neighboring pixel of the candidate matching pixel is better correlated to the target pixel than the candidate matching pixel, the system also includes means for determining a correlation score between the target pixel and a second neighboring pixel of the neighboring pixel that is opposite the candidate matching pixel based on the similarity measure, and means for determining the disparity value between the target pixel from the first image and the given row of the second image based on an interpolation of the second correlation scores for the neighboring pixel, the second neighboring pixel, and the candidate matching pixel. The system further includes means for providing the disparity value between the target pixel from the first image and the given row of the second image.

The foregoing summary is illustrative only and is not intended to be in any way limiting. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features will become apparent by reference to the figures and the following detailed description and the accompanying drawings.

