CROSS REFERENCE

The present application claims priority to Chinese Application No. 201910843603.0 filed on Sep. 6, 2019, the entire content of which is incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to audio processing technology, and more particularly, to a deep neural network (DNN) based audio processing method, device and storage medium.

BACKGROUND

Hearing assistance devices (also known as “hearing aids”) are widely used in hearing compensation for hearing-impaired patients, which can amplify sounds that are originally inaudible to the hearing-impaired patients. Then with the patients' residual hearing, the sounds can be transmitted to auditory cortex and thus sensed by the patients.

Since different hearing-impaired patients usually have different audiograms or hearing loss diagram (which can reflect different degrees of hearing loss of a patient at different audio frequencies), traditional hearing aids usually amplify audio signals within a specific frequency range based on some frequency compensation algorithms, for example, amplifying the high-frequency audio signals. However, these traditional hearing compensation algorithms are unable to distinguish between human speeches and background noises, and therefore noises may be amplified together with human speeches within the amplified hearing-aid frequency range, which may impact the reception and recognition of human speeches.

Thus, there is a need for an improved audio processing method.

SUMMARY

An objective of the present disclosure is to provide a DNN-based audio processing method, device and storage medium.

According to one aspect of the present disclosure, a deep neural network based audio processing method is provided. The audio processing method comprises: obtaining a DNN-based speech extraction model, wherein the speech extraction model is created through the following steps: obtaining a mixed audio training dataset having multiple mixed audio data frames each containing mixed speech data and non-speech data, the speech data and the non-speech data both being represented in time domain data format; acquiring at least one audiogram and at least one set of predetermined gain compensation coefficients associated with the at least one audiogram, wherein each audiogram corresponds to a set of predetermined gain compensation coefficients, and each set of predetermined gain compensation coefficients include multiple predetermined gain compensation coefficients corresponding to respective audio signal frequencies; performing, for each of the mixed audio data frames, gain compensation on the speech data included therein with the at least one set of predetermined gain compensation coefficients to generate compensated speech data; and training the DNN-based speech extraction model with the mixed audio training dataset and the compensated speech data corresponding to each of the mixed audio data frames of the mixed audio training dataset to obtain a trained speech extraction model; receiving an audio input object having a speech portion and a non-speech portion, wherein the audio input object includes one or more audio data frames each having a set of audio data samples sampled at a predetermined sampling interval and represented in time domain data format; obtaining a user audiogram and a set of user gain compensation coefficients associated with the user audiogram; and inputting the audio input object and the set of user gain compensation coefficients into the trained speech extraction model to obtain an audio output result represented in time domain data format outputted by the trained speech extraction model, wherein the non-speech portion of the audio input object is at least partially attenuated in or removed from the audio output result.

According to another aspect of the present disclosure, a DNN-based audio processing device is provided. The audio processing device comprises a non-transitory computer storage medium for storing one or more executable instructions that, when executed by a processor, causes the processor to perform the processing steps of the above aspect.

According to another aspect of the present disclosure, a non-transitory computer storage medium is provided. The non-transitory computer storage medium has stored therein one or more executable instructions that, when executed by a processor, causes the processor to perform the processing steps of the above aspect.

The foregoing is a summary of the present application, and may be simplified, generalized or omitted in detail. Thus, it should be appreciated by those skilled in the art that this part is only exemplary and illustrative, but not intend to limit the scope of the present application in any ways. This summary part is neither intend to determine the key feature or necessary feature of the subject sought to be protected, nor intend to be used as a supplementary method for determining the scope of the subject sought to be protected.

