Fourier methods have a long and proven track record as an excellent tool in data processing. We propose to integrate Fourier methods into complex recurrent neural network architectures and show accuracy improvements on prediction tasks as well as computational load reductions. We predict synthetic data drawn from synthetic-equations as well as real world power load data.
Generally, in both audio processing and speech recognition, only the magnitude of a frequency domain signal is processed, while the phase gets discarded @cite_1 @cite_12 . One likely reason for this is the fact that many machine learning methods and toolboxes are not designed to natively handle complex data. However, recent works in complex-valued networks @cite_29 @cite_5 @cite_6 @cite_3 makes it now possible process data fully in the frequency domain. The complex CNN presented in @cite_6 explores complex convolutions and applies them to the Fourier spectrum of music data, whereas in @cite_3 , a complex gated RNN was applied to similar data.
@cite_29 @cite_1 @cite_6 @cite_3 @cite_5 @cite_12
1666984270 2155273149 2591013610 2777685882
Abstract Automaticspeechrecognitionsystemstypicallymodeltherela-tionship between the acoustic speech signal and the phones intwo separate steps: feature extraction and classier training. Inourrecentworks, wehaveshownthat, intheframeworkofcon-volutionalneuralnetworks(CNN),therelationshipbetweentheraw speech signal and the phones can be directly modeled andASR systems competitive to standard approach can be built. Inthis paper, we rst analyze and show that, between the rst twoconvolutional layers, the CNN learns (in parts) and models thephone-specic spectral envelope information of 2-4 ms speech.Given that we show that the CNN-based approach yields ASRtrends similar to standard short-term spectral based ASR sys-tem under mismatched (noisy) conditions, with the CNN-basedapproach being more robust.Index Terms: automatic speech recognition, convolutionalneural networks, raw signal, robust speech recognition. 1. Introduction State-of-the-art automatic speech recognition (ASR) systemstypically model the relationship between the acoustic speechsignal and the phones in two separate steps, which are op-timized in an independent manner [1]. In a rst step, thespeech signal is transformed into features, usually composed ofa dimensionality reduction phase and an information selectionphase, based on the task-specic knowledge of the phenomena.These two phases have been carefully hand-crafted, leading tostate-of-the-art features such as Mel frequency cepstral coef-cients(MFCCs)orperceptuallinearpredictioncepstralfeatures(PLPs). In a second step, the likelihood of subword units suchas, phonemes is estimated using generative models or discrimi-native models.In recent years, in the hybrid HMM ANN framework [1],there has been growing interests in using intermediate rep-resentations instead of conventional features, such as cepstral-based features, as input for neural networks-based systems.ANNs with deep learning architectures, more precisely, deepneural networks (DNNs) [2, 3], which can yield better systemthan a single hidden layer MLP have been proposed to addressvarious aspects of acoustic modeling. More specically, useof context-dependent phonemes [4, 5]; use of spectral featuresas opposed to cepstral features [6, 7]; CNN-based system withMel lter bank energies as input [8, 9, 10]; combination of dif-ferent features [11], to name a few. Features learning from therawspeechsignalusingneuralnetworks-basedsystemshasalsobeen investigated in [12]. In all these approaches, the features Convolutional Neural Networks (CNN) have showed success in achieving translation invariance for many image processing tasks. The success is largely attributed to the use of local filtering and max-pooling in the CNN architecture. In this paper, we propose to apply CNN to speech recognition within the framework of hybrid NN-HMM model. We propose to use local filtering and max-pooling in frequency domain to normalize speaker variance to achieve higher multi-speaker speech recognition performance. In our method, a pair of local filtering layer and max-pooling layer is added at the lowest end of neural network (NN) to normalize spectral variations of speech signals. In our experiments, the proposed CNN architecture is evaluated in a speaker independent speech recognition task using the standard TIMIT data sets. Experimental results show that the proposed CNN method can achieve over 10 relative error reduction in the core TIMIT test sets when comparing with a regular NN using the same number of hidden layers and weights. Our results also show that the best result of the proposed CNN model is better than previously published results on the same TIMIT test sets that use a pre-trained deep NN model. Sound events often occur in unstructured environments where they exhibit wide variations in their frequency content and temporal structure. Convolutional neural networks CNNs are able to extract higher level features that are invariant to local spectral and temporal variations. Recurrent neural networks RNNs are powerful in learning the longer term temporal context in the audio signals. CNNs and RNNs as classifiers have recently shown improved performances over established methods in various sound recognition tasks. We combine these two approaches in a convolutional recurrent neural network CRNN and apply it on a polyphonic sound event detection task. We compare the performance of the proposed CRNN method with CNN, RNN, and other established methods, and observe a considerable improvement for four different datasets consisting of everyday sound events. The Fourier domain is used in computer vision and machine learning as image analysis tasks in the Fourier domain are analogous to spatial domain methods but are achieved using different operations. Convolutional Neural Networks (CNNs) use machine learning to achieve state-of-the-art results with respect to many computer vision tasks. One of the main limiting aspects of CNNs is the computational cost of updating a large number of convolution parameters. Further, in the spatial domain, larger images take exponentially longer than smaller image to train on CNNs due to the operations involved in convolution methods. Consequently, CNNs are often not a viable solution for large image computer vision tasks. In this paper a Fourier Convolution Neural Network (FCNN) is proposed whereby training is conducted entirely within the Fourier domain. The advantage offered is that there is a significant speed up in training time without loss of effectiveness. Using the proposed approach larger images can therefore be processed within viable computation time. The FCNN is fully described and evaluated. The evaluation was conducted using the benchmark Cifar10 and MNIST datasets, and a bespoke fundus retina image dataset. The results demonstrate that convolution in the Fourier domain gives a significant speed up without adversely affecting accuracy. For simplicity the proposed FCNN concept is presented in the context of a basic CNN architecture, however, the FCNN concept has the potential to improve the speed of any neural network system involving convolution.