This application is the U.S. national phase of International Application No. PCT/EP2019/063890 filed May 28, 2019 which designated the U.S. and claims priority to FR Patent Application No. 1855901 filed Jun. 28, 2018, the entire contents of each of which are hereby incorporated by reference.

1. FIELD OF THE INVENTION

The field of the invention is that of digital holography, in particular the generation of a digital hologram from geometric information of a 3D scene.

The invention relates to a method for rapid calculation of a digital hologram and can in particular, but not exclusively, be applied to the real-time rendering of a 3D scene on a head-mounted display in a context of virtual or augmented reality. The holographic head-mounted display immerses the user into a three-dimensional virtual world superimposed or not to the real environment, in which he can move freely. As a function of the movements and displacements of the user, the hologram displayed on the head-mounted display must be recalculated in real-time, which requires a high power of calculation.

2. PRESENTATION OF THE PRIOR ART

It is known from the U.S. Patent application, published under No US2016/0231706 A1, a method for generating a digital hologram from a pre-calculated omnidirectional angular spectrum of an object of a 3D scene. The description information of the scene comprises a meshing of the object. As illustrated inFIG. 1, the omnidirectional angular spectrum of the object corresponds to the plane-wave decomposition of the light field emitted by the object. Each plane wave is represented by a frequency coordinate vector corresponding to its direction of propagation and by a complex amplitude. According to this method, the angular spectrum of the light field emitted by the object is calculated at the surface of a sphere centered on this object. The surface of this sphere is sampled by successive subdivisions of a regular icosahedron, a vertex of the icosahedron meshing being associated with a frequency coordinate vector of the spectrum.

As illustrated byFIG. 2, the hologram plane is perpendicular to an axis that passes through the center of the sphere. The position of this plane is defined by an angle of rotation and by a distance to the center of the sphere.

For each point of the object, a viewing cone is considered, which is centered on a normal to the surface of this point and whose angle is equal to or lower than a maximum diffraction angle of the hologram, itself linked to the size of the pixels of the hologram and to the wavelength. The frequency coordinate vectors included in this cone are selected. They are then tested to determine if they cross a meshing of the object. Those which do not intersect the object are not occulted and hence serve to calculate a Fourier coefficient of the angular spectrum of the light field emitted by the object. The Fourier coefficients obtained for each of the meshes of the object are accumulated to form the “omnidirectional” angular spectrum.

Based on this angular spectrum, it is possible to derive a hologram located in a plane that is perpendicular to an axis passing through the center of the sphere, such as that ofFIG. 2, whose angular position and distance to the center of the sphere are known. The knowledge of this angular position of the hologram plane is used to rotate the omnidirectional angular spectrum towards the hologram plane and to convert it into a planar spectrum defined in a plane tangent to the sphere and parallel to the hologram plane, as illustrated byFIG. 3. The obtained planar angular spectrum is then propagated over the distance that separates it from the hologram plane along an axis that passes through the center of the sphere. The hologram is generated by application of a (inverse) Fourier transform to the propagated spectrum.

A first advantage of this solution is that it pre-calculates an omnidirectional angular spectrum of the object of the scene, from which it derives in real time any hologram whose plane is parallel to a tangent to the sphere that contains the object.

A second advantage of this solution is that it manages the occultations of all or part of the object from the observer's point of view, so that the generated hologram offers a realistic viewing of the scene.

3. DRAWBACKS OF THE PRIOR ART

A first drawback of the proposed method is that it only provides for a single object, which limits its possibilities of use.

A second drawback of this method is that the occultation management is made using a test of intersection between each light ray and the geometry of the scene, which is extremely calculation-time consuming.

A third drawback of this method is that it calculates the omnidirectional angular spectrum at the surface of a sphere sampled by successive subdivisions of a regular icosahedron. To derive the planar spectrum of the hologram, the extraction of plane waves from this sampling requires the use of a graph travel algorithm to travel through all the vertices of the meshing, which is relatively complex.

A fourth drawback of this method is that the pre-calculation of the omnidirectional angular spectrum of the scene depends on the size of the pixels of the hologram, which prevents the so-calculated omnidirectional angular spectrum to be reusable for holograms having dimensions and resolutions, and hence pixel sizes, different from those which were initially planned.

A fifth drawback of this method is that it restricts the possible positions of the hologram plane, as shown by equation 2 of the document US2016/0231706, to a rotation in a plane of the sphere and a translation along one of these rays. It hence does not allow generating a hologram whose optical axis is not directed towards the center of the object.

4. OBJECTS OF THE INVENTION

The invention helps in improving the situation.

The invention has in particular for object to overcome these drawbacks of the prior art.

More precisely, an object of the invention is to propose a less complex solution while guarantying the generation of a hologram that offers a quality viewing of the scene.

Another object of the invention is to propose a solution for generating a hologram adapted to the viewing of a scene comprising several objects.

Still another object of the invention is to propose a solution for generating a hologram that allows a viewing of the scene from any position.

5. DISCLOSURE OF THE INVENTION

These objects, as well as others that will appear hereinafter, are achieved by means of a method for generating a digital hologram representative of a 3D scene, said scene comprising at least one object, said object being defined by a set of points and their associated intensity, said method comprising:

For each object, a preliminary step of calculating an “omnidirectional” angular spectrum of the light field emitted by the object at the surface of a geometric solid centered on said object, a surface of said solid being sampled according to a predetermined grid, a sample of the grid being associated with a frequency vector and an amplitude corresponding to a sum of contributions to the light field of points of the object along a direction equal to that of the frequency vector;

For the scene, the following steps:obtaining a pose of an observer;deriving an angular spectrum of the hologram of the scene as a function of the obtained pose and from the “omnidirectional” angular spectra calculated on a per-object basis.

According to the invention, the step of calculating an angular spectrum of the light field for each object of the scene comprises, for a plurality of predetermined viewing directions, the following sub-steps:for each viewing direction:determining a sub-set of points of the object, comprising the non-occulted points;calculating an angular sub-spectrum of the light field emitted by the points of the determined sub-set; andsumming over the plurality of predetermined angular directions the amplitudes of the calculated angular sub-spectra, associated with a same sample of the grid.

The invention is based on the approach described in the prior art for generating holograms, which consists in pre-calculating an omnidirectional angular spectrum for each object of a scene at the surface of a geometric solid whose samples are associated with frequency vectors of the spectrum, and, upon reception of an observer's pose, deriving from the spectra calculated on a per-object basis a hologram of the entire scene, adapted to this pose. Moreover, for the pre-calculation of the omnidirectional angular spectrum on a per-object basis, the invention proposes, in a completely new and inventive manner, to manage exactly the occultations for only a limited number of viewing directions of the scene. For each viewing direction, it determines a sub-set of the non-occulted points of the object then pre-calculates an angular sub-spectrum of the light field emitted by the points of this sub-set. The angular spectrum emitted by the object is then obtained by accumulation of the contributions of each of these sub-spectra.

Contrary to the prior art that manages exactly the occultations by a frequency vector associated with a sample of the surface of a sphere, the invention hence proposes an approximation that allows making the method for generating a hologram, in particular the pre-calculation step, less complex.

The geometric solid used may be of any type, for example a sphere or a faced geometric solid, such as a cube or an octahedron.

According to an aspect of the invention, the step of determining the sub-set of non-occulted points comprises the following sub-steps:2D+Z rendering the object by projecting the points of the object along the viewing direction and forming an intensity image and a depth map of the projected points; andinversely projecting the points of the formed intensity image to points of the object in a reference frame of the object and calculating the coordinates thereof as a function of the depth map.

According to this embodiment of the invention, the sub-set of the non-occulated points of an object for a given viewing direction is determined using a technique of 2D+Z rendering by projection on intermediate intensity image and depth map, followed with an inverse projection. This projection can for example be a perspective or an orthographic projection. An advantage of this approach is that it is simple to implement and that, contrary to the prior art, it does not require knowing a meshed model of the 3D scene. The obtained sub-set of points serves as a basis for the invention to pre-calculate the angular sub-spectrum associated with the viewing direction.

According to another aspect of the invention, the step of calculating the angular sub-spectrum comprises a sub-step of determining an area of validity of the occultations associated with a viewing direction at the surface of the geometric solid and the calculation of the sample amplitudes is performed for the samples comprised in said area.

An advantage is to delimit an area, at the surface of the geometric solid, in which the occultations calculated for a given viewing direction are valid, in order to guarantee the generation of a hologram of good quality.

According to still another aspect of the invention, the samples comprised in the area of validity satisfy the following condition:
Vj,xx+Vj,yy+Vj,zz>0
where (x,y,z) denotes the position of the sample and (Vj,x, Vj,y, Vj,z) denotes the viewing direction (Vj) in a reference frame (Ri) of the object (Obi).

An advantage of this particular condition is that it is simple to implement. Moreover, the portion of the surface of the geometric solid considered as valid is included in a cone centered on the viewing direction (Vj).

This particular cone has an angle of 90° with respect to the viewing direction, which allows guaranteeing an overlap of the directional angular spectra, even for a low number of viewings, and hence avoiding the “hops” or breaks in continuity in the viewing of the hologram, when the observer's pose changes and makes use, for the hologram calculation, of another viewing direction.

According to another aspect of the invention, the geometric solid comprises a plurality of plane faces and the predetermined grid is cut into a plurality of sub-grids forming a regular grid pattern, a sub-grid being associated with a face of the solid.

An advantage of a sampling grid of regular grid pattern is that the calculation of the angular spectrum of each object is simplified, because the predetermined grid neither includes successive subdivisions of a regular icosahedron, nor requires the use of a graph travel algorithm to travel through all the vertices of the meshing and to calculate the complex amplitudes of the plane waves associated with the samples. Moreover, the storage of the samples thereof and the access to their amplitudes are facilitated.

According to still another aspect of the invention, the amplitude associated with a sample of the sub-grid of a face of the solid, for the calculation of the angular sub-spectrum along a said viewing direction, is defined by the following formula:

Ai,j⁡(x,y,z)=∑k=1Mx⁢My⁢Ii,j⁡(uk,νk)⁢exp⁡(-j⁢2⁢πλ⁢(fx⁢xk+fy⁢yk+fz⁢zk)),

with Mx, My the dimensions of the intensity and depth images, λ the wavelength used, and

(fxfyfz)=1x2+y2+z2⁢(xyz)
the frequencies associated with the sample (x,y,z).

An advantage is that this amplitude does not depend on the resolution nor on the size of the pixels of the hologram to be derived. The angular spectrum obtained by summing the sub-spectra is hence usable to generate a hologram of any dimensions.

According to still another aspect of the invention, the geometric solid is a cube.

An advantage of its square faces is that they still facilitate the storage of the samples of the angular spectrum and the access thereto during the step of deriving the hologram spectrum. There indeed exist memory structures of the “cube map” type, which are dedicated to this format and very performing in terms of storage.

According to still another aspect of the invention, the plurality of viewing directions comprises as many directions as the solid has faces.

An advantage of this embodiment is that it is simple to implement while guaranteeing a uniform distribution in space of the viewing directions.

According to another aspect of the invention, the step of deriving an angular spectrum of the hologram of the scene comprises deriving an angular spectrum of a sub-hologram for each object of the scene from the angular spectrum of said object, summing the angular spectra of the derived sub-holograms and applying an inverse Fourier transform to the derived angular spectrum to obtain the hologram.

An advantage of this embodiment is the simplicity thereof, because the angular spectra of the objects of the scene are generated individually. Another advantage is that handling the objects of the scene is very easy, for example making appear or disappear an object of the scene, or also moving it in the scene independently from the others.

According to still another aspect of the invention, the angular spectrum of the hologram is derived from “multidirectional” angular spectra calculated on a per-object basis according to the following formula:

H^⁡(fh,x,fh,y)=∑i=1N⁢S⁢Ai⁡(fxλ⁢F,fyλ⁢F,fzλ⁢F)⁢exp⁡(j⁢2⁢π⁡(sx⁢fh,x+sy⁢fh,y+sz⁢λ-2-fh,x2-fh,y2)),
where:
=(O; {right arrow over (xh)}, {right arrow over (yh)}, {right arrow over (zh)}) denotes a local reference frame of the hologram whose origin is located at the center of the hologram H, whose axes defined by {right arrow over (xh)} and {right arrow over (yh)} coincide with the horizontal and vertical axes of the hologram, respectively, and whose axis defined by {right arrow over (zh)} coincides with the optical axis of the hologram. Let's note

xh→=(xh⁢0xh⁢1xh⁢2),yh→=(yh⁢0yh⁢1yh⁢2)⁢⁢and⁢⁢zh→=(zh⁢0zh⁢1zh⁢2).

F=max⁡(fx,fy,fz),⁢(sxsysz)=[xh⁢0xh⁢1xh⁢2yh⁢0yh⁢1yh⁢2zh⁢0zh⁢1zh⁢2]⁢(xi-x0yi-y0zi-z0)
denotes the position of the center of the object Obi in the reference frameh, and

(fxfyfz)=[xh⁢0yh⁢0zh⁢0xh⁢1yh⁢1zh⁢1xh⁢2yh⁢2zh⁢2]⁢⁢(fh,xfh,yλ-2-fh,x2-fh,y2)
denote the frequency coordinates of the angular spectrum of the hologram in the reference frame (Ri) of the object Obi.

An advantage of this embodiment is that it allows generating a hologram adapted to any pose of the observer.

The invention also relates to a device for generating a digital hologram adapted to implement the method for generating a digital hologram according to any one of the particular embodiments defined hereinabove.

This device for generating a hologram can of course include the different features relating to the method according to the invention. Hence, the features and advantages of this device are the same as those of the method, and are not described in more detail.

Correlatively, the invention also relates to a holographic rendering system comprising a device for generating a hologram according to the invention.

According to an aspect of the invention, said system comprises a head-mounted display intended to be worn on the observer's head, said head-mounted display comprising a storage module adapted for the storage of the omnidirectional angular spectrum pre-calculated for the at least one object of the scene.

An advantage is that, as the angular spectra of the objects are stored locally, the head-mounted display can derive in real time a hologram adapted to the observer's pose.

The invention also relates to a computer program including instructions for implementing the steps of a method for generating a digital hologram as described hereinabove, when this program is executed by a processor.

This program may use any programming language. It may be downloaded from a communication network and/or recorded on a computer-readable medium.

The invention finally relates to a recording medium, readable by a processor, integrated or not to the device for generating a digital hologram according to the invention, potentially removable, memorizing respectively a computer program implementing a method for generating a digital hologram, as described hereinabove.

