There has been an increasing use of neural networks for music information retrieval tasks. In this paper, we empirically investigate different ways of improving the performance of convolutional neural networks (CNNs) on spectral audio features. More specifically, we explore three aspects of CNN design: depth of the network, the use of residual blocks along with the use of grouped convolution, and global aggregation over time. The application context is singer classification and singing performance embedding and we believe the conclusions extend to other types of music analysis using convolutional neural networks. The results show that global time aggregation helps to improve the performance of CNNs the most. Another contribution of this paper is the release of a singing recording dataset that can be used for training and evaluation. Deploying deep neural networks to solve music information retrieval problems has benefited from advancements in other areas such as computer vision and natural language processing. In this paper, experiments are designed to investigate whether a few of the recent signature advancements in the deep learning community can improve the learning capability of deep neural networks when applied on time-frequency representations. Because time-frequency representations are frequently treated as 2-D images similarly to image input for computer vision models, convolution layers are popular choices as the first processing layers for time-frequency representations in audio and music analysis applications. One of the recent convolutional layer variants is the residual neural network with a bottleneck design , ResNet. Another variant built upon ResNet is to use grouped convolution inside the bottleneck as a generalization of the Inception Net BID12 BID22 , ResNeXt. These two variants have enabled more deepening of the convolutional layers of deep neural networks. Most existing music information retrieval research using convolutional neural networks (CNNs), utilizes vanilla convolutional layers with no more than 5 layers. In this paper, the two convolution layer variants mentioned and a deeper architecture with more than 5 convolution layers is proposed and shown to be effective on audio time-frequency representations.Conceptually, convolution layers take care of learning local patterns (neighboring pixels in images or time frame/frequency bins in time-frequency representations) presented in the input matrices. After learning feature maps from the convolution layers, one of the reoccurring issues, when the input is a time-frequency representation, is how to model or capture temporal relations. Recurrent neural networks has been used to solve this problem BID6 BID1 BID3 BID4 . Recent developments from natural language processing in attention mechanisms BID0 BID3 BID15 provide a different approach to model temporal dependencies and relations. In this paper, the attention mechanism is viewed as a special case of a global aggregation operation along the timeaxis that has learnable parameters. Typical aggregation operations such as average or max have no learnable parameters. The effects of global aggregation along the time axis using either average, max or the attention mechanism is investigated experimentally.Two specific applications are investigated in this paper: 1) singer classification of monophonic recordings, and 2) singing performance embedding. The goal of singer classification is to predict the singer's identity given an audio recording as input. A finite set of possible singers is considered so this is a classification task. In singer performance embedding the goal is to create an embedding space in which singers with similar styles can be projected to be closer to each other compared to singers with different styles. Ideally, it should be possible to identify "singing style" or "singing characteristics" by examining (and listening to) the clusters formed from the projections of audio recordings onto the embedding space. Many tasks in music and audio analysis can be formulated in a similar way, in which similarity plays an essential role, therefore we believe that the conclusions of this paper generalize to other audio and music tasks. In this paper, empirical investigations into how recent developments in the deep learning community could help solving singer identification and embedding problems were conducted. From the experiment results, the obvious take away is that global aggregation over time improves performance by a considerable margin in general. The performances among the three aggregation strategies; max, average and feed-forward attention, are very close. The advantage of using feedforward attention from observing the experiment results is that it accelerates the learning process compared to other non-learnable global aggregations. One way to explain such observation is that the feed-forward attention layer learns a "frequency template" for each convolutional channel fed into it. These "frequency templates" are encoded in w and enable each convolutional channel fed into it to focus on different parts along the frequency axis (Since w ∈ R D with D = num of channels × num of frequency bins). In this paper we also have shown that training a deep neural networks having more than 15 convolutional layers on time-frequency input is definitely feasible with the help of global time aggregation. To the authors' knowledge, there is no previous music information retrieval research utilizing neural networks having more than 10 convolutional layers. A dataset consisting of over 20000 single singing voice recordings is also released and described in this paper. The released dataset, DAMP-balanced, could be partitioned in a way that for singer classification, the performed songs for each singer are the same.For future works, replacing max-pooling with striding in convolutional layers which recent works in CNN suggest will be experimented. To improve global-aggregation, taking temporal order into consideration during the global-aggregation operation as suggested in BID21 will also be experimented. Also the proposed neural network configurations will be experimented in other music information retrieval tasks such as music structure segmentation.
Using deep learning techniques on singing voice related tasks.
{"src": "scitldr-AIC_train", "paper_id": "H1cT3NTBM"}