CROSS REFERENCE TO RELATED APPLICATION

This application claims priority and benefits to Chinese Application No. 202111340133.X, filed on Nov. 12, 2021, the entire contents of which is incorporated herein by reference.

TECHNICAL FIELD

The disclosure relates to a field of three-dimensional information reconstruction and free-space optoelectronic computing technologies, in particular to a method and device for intelligent depth classification of a light field based on optoelectronic computing.

BACKGROUND

Three-dimensional (3D) vision is a perception pathway for human beings. Among five perception pathways of human beings, vision accounts for 70% to 80% of information sources; while the brain has about 50% of the ability to perceive visual information. The single-lens imaging system (like a single human eye) only supports two-dimensional. However, observing the three-dimensional world through two-dimensional images causes loss of rich information of the real world. To fully represent the light in the three-dimensional world, the concept of light field is proposed. By capturing and displaying the light field, a stereo real word can be visually reproduced. At the state of art, the light field can be captured or imaged by methods of scanning through the image sensor, imaging through the image sensor, or imaging through micro lens array, the essence of which is to achieve multi-view observation of a target object. Human eyes are the simplest light field imaging system from two viewpoints. Based on the triangular ranging principle, two-dimensional observation of the same scene from at least two different viewpoints can obtain enough information to restore and reconstruct the three-dimensional light field.

SUMMARY

In order to achieve the above objectives, embodiments of a first aspect of the disclosure propose a method for intelligent light field depth classification based on optoelectronic computing, including:

capturing and identifying binocular images of a scene within a depth range through a pair of binocular cameras;

mapping each depth value in the depth range to a disparity value between the binocular images based on the depth value and an arrangement manner of the pair of binocular cameras, to obtain a disparity range of the scene within the depth range;

labeling training data based on the disparity range to obtain a pre-trained diffraction neural network model;

loading a respective weight for each layer of a network obtained after training into a corresponding optical element based on the pre-trained diffraction neural network model;

after the respective weight for each layer of the network is loaded, performing forward propagation inference on new input data of the scene, and outputting a depth classification result corresponding to each pixel in the binocular images of the scene;

in which labeling training data based on the disparity range includes: determining that an ideal output of an optoelectronic diffraction neural network is a serial number of a respective disparity range where an object corresponding to each pixel position in a field of view is located, and obtaining an upper disparity limit and a lower disparity limit based on a selected depth range, in which the serial number being −1 indicates that the disparity value of a current pixel is higher than the upper disparity limit, the serial number being 0 indicates that the disparity value of the current pixel is between the upper disparity limit and the lower disparity limit, and the serial number being 1 indicates that the disparity value of the current pixel is lower than the lower disparity limit.

Additional aspects and advantages of the disclosure will be set forth, in part, from the following description, and in part will be apparent from the following description, or may be learned by practice of the disclosure.

