BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is related to signal processing, and more particularly, to an audio processing method and an audio processing device, such as a multi-length convolution/deconvolution layer based audio processing method and device, and a non-transitory computer-readable medium thereof.

2. Description of the Prior Art

Audio separation aims to separate the audio mixture signal and extract the part that we are interested in. It can be applicable to many different fields, such as earphones, loudspeakers, and television (TV) industries, to improve user experience. For example, singing voice separation can separate the audio mixture signal into human voice and background music. The clean human voice can be used for singer recognition, automatic lyrics transcriptions, pitch tracking, vocal enhancement effect, etc. The separated background music may also be an asset for entertainment. Therefore, a novel method and related architecture are required to realize reliable audio separation processing without introducing side effects or in a way that is less likely to cause side effects.

SUMMARY OF THE INVENTION

It is an objective of the present invention to provide an audio processing method and audio processing devices, such as a multi-length-convolution/deconvolution-layer-based audio processing method and device, and a non-transitory computer-readable medium, to solve above-mentioned problems.

It is another objective of the present invention to provide an audio processing method and an audio processing device, such as a multi-length convolution/deconvolution layer based audio processing method and device, and a non-transitory computer-readable medium, to improve overall performance for a neural network architecture of the audio processing.

At least one embodiment of the present invention provides an audio processing method, wherein the method can be applicable to an electronic device with an audio processing function (e.g. TV, audio and video system). The method may include: utilizing a first convolution layer circuit and a second convolution layer circuit to perform convolution processing on an audio mixture signal, respectively, in order to generate an input feature map, wherein each of the first convolution layer circuit and the second convolution layer circuit includes multi-layer one-dimensional (1D) convolution kernels, and a convolution kernel layer length of the first convolution layer circuit and a convolution kernel layer length of the second convolution layer circuit are equal to a first length and a second length, respectively; performing separation processing corresponding to a predetermined neural network model on the input feature map, to obtain a first output feature map and a second output feature map respectively corresponding to a first type and a second type; and utilizing a first deconvolution layer circuit and a second deconvolution layer circuit to perform deconvolution processing on different portions of the first output feature map, respectively, in order to generate corresponding processing results, for generating a first audio output signal, and utilizing the first deconvolution layer circuit and the second deconvolution layer circuit to perform deconvolution processing on different portions of the second output feature map, respectively, in order to generate corresponding processing results, for generating a second audio output signal, wherein each of the first deconvolution layer circuit and the second deconvolution layer circuit includes multi-layer one-dimensional convolution kernels, and a convolution kernel layer length of the first deconvolution layer circuit and a convolution kernel layer length of the second deconvolution layer circuit are equal to the first length and the second length, respectively.

At least one embodiment of the present invention provides and audio processing device. The audio processing device may include: an audio signal analyzer, wherein the audio signal analyzer includes a first convolution layer circuit and a second convolution layer circuit; a separator, coupled to the audio signal analyzer; and an audio signal synthesizer, coupled to the separator, wherein the audio signal synthesizer includes a first deconvolution layer circuit and a second deconvolution layer circuit. The first convolution layer circuit and the second convolution layer circuit may be arranged to perform convolution processing on an audio mixture signal, respectively, in order to generate an input feature map, wherein each of the first convolution layer circuit and the second convolution layer circuit includes multi-layer one-dimensional convolution kernels, and a convolution kernel layer length of the first convolution layer circuit and a convolution kernel layer length of the second convolution layer circuit are equal to a first length and a second length, respectively. The separator may be arranged to perform separation processing corresponding to a predetermined neural network model on the input feature map, to obtain a first output feature map and a second output feature map respectively corresponding to a first type and a second type. The first deconvolution layer circuit and the second deconvolution layer circuit may be arranged to perform deconvolution processing on different portions of the first output feature map, respectively, in order to generate corresponding processing results, and perform deconvolution processing on different portions of the second output feature map, respectively, in order to generate corresponding processing results, for generating a second audio output signal, wherein each of the first deconvolution layer circuit and the second deconvolution layer circuit comprises multi-layer one-dimensional convolution kernels, and a convolution kernel layer length of the first deconvolution layer circuit and a convolution kernel layer length of the second deconvolution layer circuit are equal to the first length and the second length, respectively.

At least one embodiment of the present invention provides a non-transitory computer-readable medium. The non-transitory computer-readable medium stores computer-readable instruction code so that an audio processing device performs an audio processing procedure when executing the computer-readable instruction code, and the audio processing procedure comprises: utilizing a first convolution layer circuit and a second convolution layer circuit to perform convolution processing on an audio mixture signal, respectively, in order to generate an input feature map, wherein each of the first convolution layer circuit and the second convolution layer circuit includes multi-layer one-dimensional convolution kernels, and a convolution kernel layer length of the first convolution layer circuit and a convolution kernel layer length of the second convolution layer circuit are equal to a first length and a second length, respectively; performing separation processing corresponding to a predetermined neural network model on the input feature map, to obtain a first output feature map and a second output feature map respectively corresponding to a first type and a second type; and utilizing a first deconvolution layer circuit and a second deconvolution layer circuit to perform deconvolution processing on different portions of the first output feature map, respectively, in order to generate corresponding processing results, for generating a first audio output signal, and utilizing the first deconvolution layer circuit and the second deconvolution layer circuit to perform deconvolution processing on different portions of the second output feature map, respectively, in order to generate corresponding processing results, for generating a second audio output signal, wherein each of the first deconvolution layer circuit and the second deconvolution layer circuit comprises multi-layer one-dimensional convolution kernels, and a convolution kernel layer length of the first deconvolution layer circuit and a convolution kernel layer length of the second deconvolution layer circuit are equal to the first length and the second length, respectively.

The audio processing method and the audio processing device of the present invention can perform audio processing by the convolution layers with different lengths and the deconvolution layers with different lengths, to improve overall performance of the neural network architecture for the audio processing. Compared with the prior art, the audio processing method and the audio processing device of the present invention can greatly improve the sound output quality of audio separation processing.

These and other objectives of the present invention will no doubt become obvious to those of ordinary skill in the art after reading the following detailed description of the preferred embodiment that is illustrated in the various figures and drawings.

