This application is a U.S. national stage of PCT/IB2019/052351 filed on 22 Mar. 2019.

FIELD OF THE INVENTION

The present invention relates to a plenoptic image capturing technique to capture images in the field of photography, cinematography, microscopy and stereoscopy.

In particular, the plenoptic shooting process according to the present invention is a novel concept of Correlation Plenoptic Imaging (CPI) that can be defined as “Correlation Plenoptic Imaging between Arbitrary Planes” (CPI-AP).

Plenoptic Imaging (PI) is a technique aimed at recording both the location and the propagation direction of light in a given scene. Knowledge of the propagation direction is exploited, in post-processing, to geometrically re-trace the light path, thus enabling re-focusing different planes within the acquired image, extending the image depth of field (DOF) and reconstructing a tridimensional (3D) image.

PRIOR ART

Current plenoptic imaging systems resemble standard cameras having a conventional main lens and an image sensor, such as a CCD, a CMOS, etc. However, a microlens array is inserted in the native image plane and the sensor array is moved behind it. On one hand, the microlenses act as imaging pixels to gain the spatial information of the scene; on the other hand, they reproduce multiple images of the camera lens on the sensor, providing the angular information associated with each imaging pixel. As a result, the image resolution is reduced by a factor Nuwith respect to the diffraction limit set by the main lens numerical aperture, with Nuthe number of pixels per side associated to each microlens. Hence, standard plenoptic images are typically characterized by a poor resolution.

U.S. Pat. No. 7,620,309 describes a device for the plenoptic capture of images comprising a conventional main lens and an image sensor to digitally capture images. The described device also comprises a microlens array which, unlike other previously known devices, is arranged between the scene to be captured and the main lens of the device. In fact, this document suggests to decrease the number of microlenses. In this way, a reduced number of high-resolution images can be captured instead of a high number of low-resolution images, as is the case for the other previously known plenoptic devices. After detecting the image, an image processing device is required to obtain a large number of high-resolution images from the reduced number of high-resolution captured images.

Anyway, the capturing device described in U.S. Pat. No. 7,620,309 is not able to simultaneously increase the resolution of captured images and the number of original views of the scene to be captured. In fact, a high number of high-resolution views can be obtained only by means of an interpolation process following the capture of the low-resolution images.

A further problem of the plenoptic capturing devices of the known art is the poor achievable depth of field. This type of problem and the low angular resolution of the captured image are closely correlated, because the angular resolution is linked to the number of focused views that can be captured in a single plenoptic image, and therefore to the depth of field of the image.

A solution to the aforementioned problems of the known art is described in patent application n. EP3220185A1 in the name of the present Applicant. In this document, a process and device for the plenoptic capture are described, both based on the correlation measure or “Correlation Plenoptic Imaging” (CPI), wherein a primary light beam coming from at least one light source is split in two distinct light beams that are directed respectively towards a first image sensor to capture the angular measure and towards a second image sensor to capture the spatial measure.

Although by this solution high-resolution plenoptic images having high depth of field can be obtained, on the other side this solution does not allow real-time monitoring of the scene of interest during capturing. This problem is due to the fact that the first sensor captures the image of the light emitting source, while the second sensor captures the image (second-order or “ghost” image) of the scene as a result of measuring correlations between the two sensors, i.e. as a result of multiple captures by both sensors. Moreover, even if the device of EP3220185A1 does not entail a loss of resolution, acquiring the image of the focusing element is required for retrieving directional information; this is not always trivial, above all when complex imaging systems are used, such as a microscope, or a telescopic, objective. Another limitation of this scheme is that the image resolution of the scene of interest is defined by the numerical aperture of the light emitting source which could be difficult to control and optimize.

An object of the present invention is to solve the above-mentioned problems of the known prior art and to provide a process and an apparatus for the plenoptic capture of images that do not require the introduction of a microlens array.

Another object of the present invention is to provide a process and an apparatus for the plenoptic capture of images which enable to extend the depth of field and perform refocusing while maintaining the resolution of the focused images at the diffraction limit.

A further object of the invention is to provide a process and an apparatus which allow to maintain the advantages of plenoptic imaging, such as refocusing, increased depth of field, low-light imaging, 3D imaging etc., but without renouncing to diffraction-limited image resolution, and highly releasing the resolution versus depth of field compromise.

SUMMARY OF THE INVENTION

These and other objects of the present invention are achieved through a process for the plenoptic capture of photographic, cinematographic, microscopic or stereoscopic images of an object or a 3D scene of interest, comprising the steps of:providing a light emitting source;generating a first light beam and a second light beam coming from the light emitting source;directing the first light beam towards a first image sensor (e.g., CCD, CMOS, etc.) and the second light beam towards a second image sensor, the first light beam being adapted to be either reflected by the object or 3D scene or transmitted through the object or 3D scene.

The process of the present invention includes the step of retrieving the propagation direction of light by measuring spatio-temporal correlations between the light intensities detected by the two image sensors in the image planes of at least two arbitrary planes chosen in the vicinity of the object or within the 3D scene.

This enables to highly increase the maximum achievable depth of field of the plenoptic device, while maintaining high (diffraction-limited) resolution, as defined by the imaging lens. On the contrary, in the process described in the previous patent application n. EP3220185A1, two very precise imaging planes are imaged, namely, the plane of the focusing element and a plane within the scene of interest; this choice prohibits refocusing beyond the plane of the focusing element, thus highly restricting the range of accessible depth of field.

The present invention also offers a practical advantage when complex imaging systems, such as a microscope, or a telescopic, objective, are used as focusing elements.

Among the advantages of the present invention, it should be noted that no imaging of the focusing element is required and the real-time image of the object is immediately available by standard intensity measurement. Moreover, a further increase of the depth of field with respect to prior art is obtained thanks to the idea of imaging two arbitrary planes rather than imaging the focusing element and a generic plane of the object or 3D scene. For correlation measurement to be effective, the source must emit light characterized by spatio-temporal correlation.

Beside considering a chaotic light emitting source illuminating the object, the object can emit chaotic light by itself (e.g., it is a fluorescent object). Therefore, according to the present invention, the light emitting source can be an object or 3D scene which reflects/transmits the light from the chaotic source or can be the same object or 3D scene which emits the chaotic light.

Also, for the first time, the present invention allows the possibility of illuminating the object of interest with entangled photons/beams (e.g. emitted by spontaneous parametric down conversion). Again, the object will reflect/transmit the illuminating entangled photons/beams.

According to an aspect of the present invention, the image of the object or the 3D scene can be obtained by direct intensity measurement on a standard image sensor (e.g., CCD, CMOS, . . . ) such as, for example, the first image sensor. On the contrary, in the known scheme of EP3220185A1, intensity correlation measurement between the two image sensors is required to reconstruct the “ghost” image of the sample. Direct observation and monitoring of the sample, as well as comparison/integration with standard plenoptic imaging, are thus now possible.

The information about the propagation direction of light can be obtained by intensity correlation measurement between the two image sensors, both retrieving the standard (i.e., intensity) image of two arbitrary planes in the vicinity of the object or 3D scene. On the contrary, in the known scheme of EP3220185A1, one image sensor retrieves the standard image of the focusing element, while the other sensor retrieves the standard image of an “empty” plane, chosen in such a way to retrieve, by intensity correlation measurement between the two image sensors, the “ghost” image of a plane in the vicinity of the object or within the 3D scene. In both cases, correlation measurement between the two sensors are required for obtaining the propagation direction of light, and thus retracing light paths for refocusing and 3D imaging.

According to the present invention, the depth of field (DOF) of the original image can be enhanced by reconstructing the direction of light between the two arbitrary planes (P′, P″; D′b, D″a) and by retracing the light paths to obtain refocused images. The image of the object with enhanced depth of field or the 3D image of the scene can be reconstructed by stacking the refocused images.

According to an embodiment of the invention, the first light beam and the second light beam are quantum entangled beams generated, for example, by a spontaneous parametric down-conversion light (SPDC) emitting source. In this case, the object or 3D scene can be placed in the optical path of one of the two light beams, for example in the optical path of the second light beam.

The present invention also relates to an apparatus for the plenoptic capture of photographic, cinematographic, microscopic or stereoscopic images of an object or a 3D scene of interest, comprising:a first image sensor to detect images coming from the object or 3D scene along a path of a first light beam;a second image sensor to detect images coming from the object or 3D scene along a path of a second light beam;a processing unit of the intensities detected by the image sensors.

According to an embodiment of the present invention, the processing unit of the apparatus is configured to retrieve the propagation direction of light by measuring spatio-temporal correlations between the light intensities detected by the image sensors in the image planes of at least two arbitrary planes chosen in the vicinity of the object or within the 3D scene.

The apparatus further includes a main lens (or more complex imaging system) Lf, with positive focal length f and diameter D; the latter can also be the diameter of any other entrance pupil. The two arbitrary planes are placed at different distances zaand zbfrom the front principal plane of the main lens Lf.

The first image sensor is placed at a distance z′afrom the back principal plane of the main lens Lfand the second image sensor is placed at a distance z′bfrom the back principal plane of the main lens Lf.

An embodiment of the apparatus according to the invention can further include a splitting element placed between the main lens Lfand the image sensors so as to generate the first light beam and the second light beam from a primary light beam coming from the object or 3D scene through the main lens Lf. Alternatively, the first light beam and the second light beam can also be the result of a free space propagation.

According to an embodiment, the apparatus according to the invention is configured to detect the first light beam and the second light beam from a source of entangled photons or beams, such as an SPDC emitting source.

In this case, the apparatus comprises two identical lenses L2of positive focal length f2whose back principal planes are placed at distances z′aand z′bfrom the image sensors along the respective paths of the first light beam and the second light beam. Lenses L2define two conjugate planes at distances za=(1/f−1/z′a)−1and zb=(1/f−1/z′b)−1, respectively, from the front principal planes of lenses L2. The focused images of the at least two arbitrary planes are thus retrieved at different distances z′aand z′bfrom the back principal plane of said lenses L2along the respective paths of the first light beam and the second light beam.

The apparatus further includes an additional lens L1with focal length f1placed between the spontaneous parametric down-conversion light source and the two identical lenses L2.

In all the embodiments of the apparatus according to the present invention, the first image sensor and the second image sensor can be distinct image sensor devices. In this case, a synchronization unit is also required to guarantee simultaneous measurements by the two image sensors. Alternatively, the first image sensor and the second image sensor can be two disjoint parts of a same image sensor device. Interestingly, also standard plenoptic devices (not exploiting correlated sources and correlation measurement) can be modified according to the present invention in order to increase the maximum available depth of field (DOF) without further reducing the resolution with respect to the state of the art.

