TECHNICAL FIELD

The present invention relates to the technical field of omnidirectional image processing, and more particularly relates to an omnidirectional image processing method and device.

BACKGROUND ART

A 360° omnidirectional image refers to all the scenes around an observation point in space which are composed of all the light that can be received by this observation point, and a sphere can describe all the scenes around the observation point. Since spherical images are difficult to store, and an existing image codec is designed for common non-omnidirectional images and is not ideal for the coding effect of the spherical images, it is necessary to convert the spherical images into omnidirectional image formats through projection, and the common projection formats include Equirectangular Projection (ERP), Cubemap Projection (CMP), etc. For omnidirectional images in different formats, an existing coding mode for common non-omnidirectional images such as HEVC, AVS2, AV1, etc. may be adopted for coding, and a locally adjusted coding method may also be adopted for different formats.

ERP sampling is shown inFIG. 1, and points on a sphere may be expressed with longitude φ and latitude θ. The sphere is mapped to a rectangle with an aspect ratio of 2:1 in a φ-θ coordinate system, and then the rectangle is uniformly sampled. The ERP format has oversampling and image texture overstretching issues in the high-latitude areas.

As shown inFIG. 2, a sphere is projected to six faces of a cube through a perspective projection, and then the cube is uniformly sampled. Compared with the ERP, the CMP samples the sphere more uniformly, and has higher coding efficiency. However, due to the characteristics of the perspective projection, the corners of each face of the cube have a higher sampling density than the central region. Correspondingly, for the sphere, the sampling uniformity of the sphere is insufficient.

After determining a position mapping relationship between the sphere and the omnidirectional image, the omnidirectional image may be sampled, so as to generate a pixel value for each pixel position, and then the omnidirectional image is coded and decoded. The position mapping relationship between the sphere and the omnidirectional image may also be utilized to perform format conversion of the omnidirectional image, and the ERP format may be converted to the CMP format or other formats.

SUMMARY OF THE INVENTION

In view of the defects in the prior art, the present invention provides a novel omnidirectional image processing method and device.

For the same area, in a Cubemap Projection (CMP) format, the central region of each face corresponds to a larger region on a sphere than marginal regions, which leads to non-uniform sampling on the sphere and redundant information in the marginal regions of the cube, and reduces the representation efficiency.

A main idea of the present invention is to enable the arc length of the unit interval at each latitude in an omnidirectional image to be as equal as possible on the sphere by adjusting a stretching relationship between the omnidirectional image and the sphere in different positions, thereby reducing information redundancy in the marginal regions in Cubemap projection, and improving the representation efficiency.

For this purpose, the present invention adopts the following technical solution:

A first objective of the present invention is to provide an omnidirectional image processing method, including:

decoding a code stream to obtain omnidirectional image coding format information; and determining a mapping relationship between a position (x, y) in at least one region in an omnidirectional image obtained by decoding the code stream and a position (θ, φ) on a sphere according to the format information, wherein the mapping relationship is:

x and φ are in linear relationship, and y and

θtan-1⁢cos⁢⁢φ
are in linear relationship,

where x represents a first dimensional coordinate position of the region in the decoded omnidirectional image, y represents a second dimensional coordinate position of the region in the decoded omnidirectional image, φ represents a longitude position of the sphere, θ represents a latitude position of the sphere, a line mapped to the sphere by a first dimensional coordinate axis is an equator line, and a line mapped to the sphere by a second dimensional coordinate axis is a prime meridian.

Preferably, the omnidirectional image format information is one of the following:

1) The format information is a format number, and the format number specifies a direction of a first dimensional coordinate of the region in the decoded omnidirectional image by default.

2) The format information includes a format number and direction information of the first dimensional coordinate of the region in the decoded omnidirectional image.

A second objective of the present invention is to provide an omnidirectional image processing method, including the following steps:

expressing at least one image region in an omnidirectional image by the following mapping relationship, wherein the mapping relationship between a position (x, y) in the image region and a position (θ, φ) on a sphere is:

x and φ are in linear relationship, and y and

θtan-1⁢cos⁢⁢φ
are in linear relationship,

where x represents a first dimensional coordinate position of the region in the omnidirectional image, y represents a second dimensional coordinate position of the region in the omnidirectional image, φ represents a longitude position of the sphere, θ represents a latitude position of the sphere, a line mapped to the sphere by a first dimensional coordinate axis is an equator line, and a line mapped to the sphere by a second dimensional coordinate axis is a prime meridian;

coding and writing the omnidirectional image expressed by the mapping relationship to a code stream; and

writing format information of the mapping relationship to the code stream.

Preferably, the omnidirectional image format information is one of the following:

1) The format information is a format number, and the format number specifies a direction of a first dimensional coordinate of the region in the omnidirectional image by default.

2) The format information includes a format number and direction information of the first dimensional coordinate of the region in the omnidirectional image.

A third objective of the present invention is to provide an omnidirectional image processing method, including the following steps:

decoding a code stream to obtain omnidirectional image coding format information; and

determining mapping relationships between a position (x1, y1) in a region 1 and a position (x2, y2) in a region 2 and their respective corresponding positions (θ1, φ1) and (θ2, φ2) on a sphere according to the format information since at least one pair of adjacent regions, i.e. the region 1 and the region 2, is present in a decoded omnidirectional image, wherein the mapping relationships are:

x1 and φ1 are in linear relationship, and y1 and

θ⁢1tan-1⁢cos⁢⁢φ⁢⁢1
are in linear relationship;

x2 and φ2 are in linear relationship, and y2 and

θ⁢2tan-1⁢cos⁢⁢φ2
are in linear relationship,

where x1 represents a first dimensional coordinate position of the region 1 in the omnidirectional image, y1 represents a second dimensional coordinate position of the image region 1, φ1 represents a longitude position of the sphere corresponding to the position (x1, y1) in the region 1 in the omnidirectional image, and θ1 represents a latitude position of the sphere corresponding to the position (x1, y1) in the region 1 in the omnidirectional image; x2 represents a first dimensional coordinate position of the region 2 in the omnidirectional image, y2 represents a second dimensional coordinate position of the region 2 in the omnidirectional image, φ2 represents a longitude position of the sphere corresponding to the position (x2, y2) in the image region 2, and θ2 represents a latitude position of the sphere corresponding to the position (x2, y2) in the region 2 in the omnidirectional image; and a line mapped to the sphere by a first dimensional coordinate axis of the region 1 is the same as a line mapped to the sphere by a first dimensional coordinate axis of the region 2, and is an equator line of the sphere.

A fourth objective of the present invention is to provide an omnidirectional image processing device, including the following modules:

a format information extraction module, wherein input of the format information extraction module is a coding code stream, output of the format information extraction module is omnidirectional image format information, and the module decodes the coding code stream to obtain the omnidirectional image format information; and

a position mapping module, wherein input of the position mapping module is the omnidirectional image format information, output of the position mapping module is a mapping relationship between a position (x, y) in at least one region in an omnidirectional image obtained by decoding the code stream and a position (θ, φ) on a sphere, and the mapping relationship is:

x and φ are in linear relationship, and y and

θtan-1⁢cos⁢⁢φ
are in linear relationship,

where x represents a first dimensional coordinate position of the region in the omnidirectional image, y represents a second dimensional coordinate position of the region in the omnidirectional image, φ represents a longitude position of the sphere, θ represents a latitude position of the sphere, a line mapped to the sphere by a first dimensional coordinate axis is an equator line, and a line mapped to the sphere by a second dimensional coordinate axis is a prime meridian.

Preferably, the omnidirectional image format information is one of the following:

1) The format information is a format number, and the format number specifies a direction of a first dimensional coordinate of the region in the omnidirectional image by default.

2) The format information includes a format number and direction information of the first dimensional coordinate of the region in the omnidirectional image.

A fifth objective of the present invention is to provide an omnidirectional image processing device, including the following modules:

a position mapping module, wherein output of the position mapping module is omnidirectional image format information, and a mapping relationship between a position (x, y) in at least one region in an omnidirectional image and a position (θ, φ) on a sphere is determined in the module as:

x and φ are in linear relationship, and y and

θtan-1⁢cos⁢⁢φ
are in linear relationship,

where x represents a first dimensional coordinate position of the region in the omnidirectional image, y represents a second dimensional coordinate position of the region in the omnidirectional image, φ represents a longitude position of the sphere, θ represents a latitude position of the sphere, a line mapped to the sphere by a first dimensional coordinate axis is an equator line, and a line mapped to the sphere by a second dimensional coordinate axis is a prime meridian; and

a format information and image coding module, wherein input of the format information and image coding module is the omnidirectional image format information and the omnidirectional image expressed by the mapping relationship determined by the format information, output of the format information and image coding module is a code stream including the omnidirectional image and the format information of the omnidirectional image, and the module codes and writes the omnidirectional image and the format information to the code stream.

Preferably, the omnidirectional image format information is one of the following:

1) The format information is a format number, and the format number specifies a direction of a first dimensional coordinate of the region in the omnidirectional image by default.

2) The format information includes a format number and direction information of the first dimensional coordinate of the region in the omnidirectional image.

Compared with the prior art, by designing the special position mapping relationship between the sphere and the omnidirectional image, the present invention enables the arc length of the unit interval at each latitude in the omnidirectional image to be as equal as possible on the sphere, and the omnidirectional image to be distributed more uniformly on the sphere, so that the spherical uniformity of expression is improved, the sampling loss is reduced under the same sample number, and the coding efficiency is improved.

