TECHNICAL FIELD

This application relates to the audio processing field, and in particular, to an audio rendering method and apparatus.

BACKGROUND

Three-dimensional audio is an audio processing technology that simulates a sound field of a real sound source in two ears to enable a listener to perceive that a sound comes from a sound source in three-dimensional space. A head related transfer function (HRTF) is an audio processing technology used to simulate conversion of an audio signal from a sound source to the eardrum in a free field, including impact imposed by the head, auricle, and shoulder on sound transmission. In an actual environment, a sound heard by the ear includes not only a sound that directly reaches the eardrum from a sound source, but also a sound that reaches the eardrum after being reflected by the environment. To simulate a complete sound, the conventional technology provides a binaural room impulse response (BRIR), to represent conversion of an audio signal from a sound source to the two ears in a room.

An existing BRIR rendering method is roughly as follows: A mono signal or a stereo signal is used as an input audio signal, a corresponding BRIR function is selected based on an azimuth of a virtual sound source, and the input audio signal is rendered according to the BRIR function to obtain a target audio signal.

However, in the existing BRIR rendering method, only impact of different azimuths on a same horizontal plane is considered, and an elevation angle of the virtual sound source is not considered. Consequently, a sound in the three-dimensional space cannot be accurately rendered.

SUMMARY

In view of this, this application provides a binaural audio processing method and audio processing apparatus, to accurately render an audio in three-dimensional space.

According to a first aspect, an audio rendering method is provided, including: obtaining a to-be-rendered BRIR signal, where an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; obtaining a direct sound signal based on the to-be-rendered BRIR signal; correcting, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal, to obtain a frequency-domain signal corresponding to the target elevation angle; obtaining a time-domain signal based on the corrected frequency-domain signal; and superposing the time-domain signal on a signal that is in the to-be-rendered BRIR signal and that is in a second time period after a first time period, to obtain a BRIR signal of the target elevation angle. The direct sound signal corresponds to the first time period in a time period corresponding to the to-be-rendered BRIR signal.

According to this embodiment, because there is a correspondence between the target elevation angle and the time-domain signal that is obtained based on the corrected frequency-domain signal, and the signal in the second time period can reflect audio transformation caused by environmental reflection, a target BRIR signal synthesized by the signal in the second time period and the time-domain signal is a stereo BRIR signal.

In an embodiment, the correcting, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal includes: determining a correction coefficient based on the target elevation angle and a correction function; and correcting, based on the correction coefficient, the frequency-domain signal corresponding to the direct sound signal, to obtain the corrected frequency-domain signal. The correction function includes a numerical relationship between coefficients of HRTF signals corresponding to different elevation angles.

According to this embodiment, the correction coefficient may be determined based on the target elevation angle and the correction function corresponding to the target elevation angle. The correction coefficient may be a vector including a group of coefficients. The correction coefficient is used to process the frequency-domain signal corresponding to the direct sound signal, so that an obtained corrected frequency-domain signal corresponds to the target elevation angle. Therefore, a method for correcting the frequency-domain signal corresponding to the direct sound is provided, so that the corrected frequency-domain signal can correspond to the target elevation angle.

In another embodiment, the correcting, based on a target elevation angle, a frequency-domain signal corresponding to the direct sound signal includes: correcting, based on the target elevation angle, at least one piece of information about a peak point or a valley point in a spectral envelope corresponding to the direct sound signal, to obtain at least one piece of corrected information about the peak point or the valley point, where the at least one piece of corrected information about the peak point or the valley point corresponds to the target elevation angle; determining a target filter based on the at least one piece of corrected information about the peak point or the valley point; and filtering the direct sound signal by using the target filter, to obtain the corrected frequency-domain signal.

According to this embodiment, a correction coefficient of the peak point in the spectral envelope may be determined based on the target elevation angle, and then at least one piece of information about the peak point is corrected by using the correction coefficient of the peak point. The at least one piece of information about the peak point includes a center frequency of the peak point, a bandwidth of the peak point, and a gain of the peak point. A peak point filter is determined based on at least one piece of corrected information about the peak point. In addition, a correction coefficient of the valley point in the spectral envelope may be determined based on the target elevation angle, and then at least one piece of information about the valley point is corrected by using the correction coefficient of the valley point. The at least one piece of information about the valley point includes but is not limited to a bandwidth of the valley point and a gain of the valley point. A valley point filter is determined based on at least one piece of corrected information about the valley point. The peak point filter and the valley point filter are cascaded to obtain the target filter. Because both the peak point filter and the valley point filter correspond to the corrected information, there is also a correspondence between the target filter and the corrected information. The corrected information is related to the target elevation angle. Therefore, after the direct sound signal is filtered by using the target filter, the obtained corrected frequency-domain signal is related to the target elevation angle. Therefore, another method for obtaining the direct sound frequency-domain signal corresponding to the target elevation angle is provided.

In another embodiment, the obtaining a time-domain signal based on the corrected frequency-domain signal includes: determining an energy adjustment coefficient based on the target elevation angle and an energy adjustment function; adjusting the corrected frequency-domain signal based on the energy adjustment coefficient to obtain an adjusted frequency-domain signal; and performing frequency-time conversion on the adjusted frequency-domain signal to obtain the time-domain signal. The energy adjustment function includes a numerical relationship between frequency band energy of the HRTF signals corresponding to different elevation angles.

According to this embodiment, the energy adjustment coefficient may be determined based on the target elevation angle and the energy adjustment function. Because the energy adjustment function includes the numerical relationship between frequency band energy of the HRTF signals corresponding to different elevation angles, the energy adjustment coefficient can represent a difference between frequency band energy distributions of the signals. The corrected frequency-domain signal is adjusted based on the energy adjustment coefficient, to adjust a frequency band energy distribution of the corrected frequency-domain signal, so as to reduce a problem that a sound disappears at an eccentric ear valley point, and optimize a stereo effect.

In another embodiment, the obtaining a direct sound signal based on the to-be-rendered BRIR signal includes: extracting a signal in the first time period from the to-be-rendered BRIR signal, and processing the signal in the first time period by using a Hanning window, to obtain the direct sound signal. According to this embodiment, windowing processing is performed on the signal in the first time period by using the Hanning window, so that a truncation effect in a time-frequency conversion process can be eliminated, interference caused by trunk scattering can be reduced, and accuracy of the signal can be improved. In addition, a Hamming window may alternatively be used to perform windowing processing on the signal in the first time period.

In another embodiment, the obtaining a direct sound signal based on the to-be-rendered BRIR signal includes: extracting a signal in the first time period from the to-be-rendered BRIR signal, and processing the signal in the first time period by using a Hanning window, to obtain the direct sound signal. The obtaining a time-domain signal based on the corrected frequency-domain signal includes: superposing a spectrum of the corrected frequency-domain signal on a spectrum detail, and performing frequency-time conversion on a signal corresponding to a spectrum obtained through superposition, to obtain the time-domain signal. The spectrum detail is a difference between a spectrum of the signal in the first time period and a spectrum of the direct sound signal, and may represent an audio signal lost in a windowing process. According to this embodiment, the corrected frequency-domain signal is corrected by using the spectrum detail, to increase the audio signal lost in the windowing process, so as to better restore the BRIR signal and achieve a better simulation effect.

In another embodiment, the obtaining a direct sound signal based on the to-be-rendered BRIR signal includes: extracting a signal in the first time period from the to-be-rendered BRIR signal, and processing the signal in the first time period by using a Hanning window, to obtain the direct sound signal.

The obtaining a time-domain signal based on the corrected frequency-domain signal includes: superposing a spectrum of the corrected frequency-domain signal on a spectrum detail, where the spectrum detail is a difference between a spectrum of the signal in the first time period and a spectrum of the direct sound signal; determining an energy adjustment coefficient based on the target elevation angle and an energy adjustment function; adjusting, based on the energy adjustment coefficient, a signal corresponding to a spectrum obtained through superposition, to obtain an adjusted frequency-domain signal; and performing frequency-time conversion on the adjusted frequency-domain signal to obtain the time-domain signal. The energy adjustment function includes a numerical relationship between frequency band energy of the HRTF signals corresponding to different elevation angles.

According to this embodiment, after the spectrum detail is superposed on the spectrum of the corrected frequency-domain signal, the signal corresponding to the spectrum obtained through is adjusted by using the energy adjustment coefficient, so that a frequency band energy distribution of the signal corresponding to the spectrum obtained through superposition can be adjusted, and a stereo effect can be optimized.

According to a second aspect, an audio rendering method is provided, including: obtaining a to-be-rendered BRIR signal, where an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; correcting, based on a target elevation angle, a frequency-domain signal corresponding to the to-be-rendered BRIR signal; and performing frequency-time conversion on a corrected frequency-domain signal to obtain a BRIR signal of the target elevation angle. According to this embodiment, the frequency-domain signal corresponding to the to-be-rendered BRIR signal is corrected based on the target elevation angle, so that the BRIR signal corresponding to the target elevation angle can be obtained. Therefore, a method for implementing a stereo BRIR signal is provided.

In another embodiment, the correcting, based on a target elevation angle, a frequency-domain signal corresponding to the to-be-rendered BRIR signal includes: determining a correction coefficient based on the target elevation angle and a correction function; and processing, by using the correction coefficient, the frequency-domain signal corresponding to the to-be-rendered BRIR signal, to obtain the corrected frequency-domain signal. The correction function includes a numerical correspondence between spectrums of HRTF signals corresponding to different elevation angles. According to this embodiment, the correction coefficient may be determined based on the target elevation angle and the correction function corresponding to the target elevation angle. The correction coefficient may be a vector including a group of coefficients, and each coefficient corresponds to one frequency-domain signal point. The correction coefficient is used to process the frequency-domain signal corresponding to the to-be-rendered BRIR signal, so that an obtained corrected frequency-domain signal corresponds to the target elevation angle. Therefore, a method for correcting the to-be-rendered BRIR signal is provided, so that the corrected frequency-domain signal can correspond to the target elevation angle.

According to a third aspect, an audio rendering method is provided, including: obtaining a to-be-rendered BRIR signal, where an elevation angle corresponding to the to-be-rendered BRIR signal is 0 degrees; obtaining an HRTF spectrum corresponding to a target elevation angle; and correcting the to-be-rendered BRIR signal based on the HRTF spectrum corresponding to the target elevation angle, to obtain a BRIR signal of the target elevation angle. According to this embodiment, a correction coefficient may be determined based on the HRTF spectrum corresponding to the target elevation angle. The correction coefficient is used to process a frequency-domain signal corresponding to the to-be-rendered BRIR signal, so that an obtained corrected frequency-domain signal corresponds to the target elevation angle. Therefore, another method for obtaining a stereo BRIR signal is provided.

According to a fourth aspect, an audio rendering apparatus is provided. The audio rendering apparatus may include an entity such as a terminal device or a chip, and the audio rendering apparatus includes a processor and a memory. The memory is configured to store instructions, and the processor is configured to execute the instructions in the memory, to enable the audio rendering apparatus to perform the method according to any one of the first aspect, the second aspect, or the third aspect.

According to a fifth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions, and when the instructions are run on a computer, the computer is enabled to perform the method according to the foregoing aspects.

According to a sixth aspect, a computer program product including instructions is provided. When the computer program product runs on a computer, the computer is enabled to perform the method according to the foregoing aspects.

