Emotional aspects play an important part in our interaction with music. However, modelling these aspects in MIR systems have been notoriously challenging since emotion is an inherently abstract and subjective experience, thus making it difficult to quantify or predict in the first place, and to make sense of the predictions in the next. In an attempt to create a model that can give a musically meaningful and intuitive explanation for its predictions, we propose a VGG-style deep neural network that learns to predict emotional characteristics of a musical piece together with (and based on) human-interpretable, mid-level perceptual features. We compare this to predicting emotion directly with an identical network that does not take into account the mid-level features and observe that the loss in predictive performance of going through the mid-level features is surprisingly low, on average. The design of our network allows us to visualize the effects of perceptual features on individual emotion predictions, and we argue that the small loss in performance in going through the mid-level features is justified by the gain in explainability of the predictions.
Deep neural networks are preferable for many tasks due to their high performance but can be considered black boxes due to their non-linear and nested structure. While in some fields such as healthcare or criminal justice the use of predictive analytics can have life-affecting consequences @cite_9 , the decisions of MIR models are generally not as severe. Nevertheless, also in MIR it would be desirable to be able to obtain explanations for the decisions of a music recommendation or search system, for various reasons (see also Section ). Many current methods for obtaining insights into deep network-based audio classification systems do not explain the predictions in a human understandable way but rather design special filters that can be visualized @cite_16 , or analyze neuron activations @cite_3 . To the best of our knowledge, @cite_13 is the only attempt to build an interpretable model for MER. They performed the task of feature extraction and selection and built models from different model classes on top of them. The only interpretation offered is the reporting of coefficients from their logistic regression models, without further explanation.
@cite_9 @cite_16 @cite_13 @cite_3
2894881080 2964052309 2414752589 2918524527
The authors developed and implemented transparent machine-learning models that call into question the use of black-box machine-learning models in healthcare and criminal justice applications. Deep learning is progressively gaining popularity as a viable alternative to i-vectors for speaker recognition. Promising results have been recently obtained with Convolutional Neural Networks (CNNs) when fed by raw speech samples directly. Rather than employing standard hand-crafted features, the latter CNNs learn low-level speech representations from waveforms, potentially allowing the network to better capture important narrow-band speaker characteristics such as pitch and formants. Proper design of the neural network is crucial to achieve this goal.This paper proposes a novel CNN architecture, called SincNet, that encourages the first convolutional layer to discover more meaningful filters. SincNet is based on parametrized sinc functions, which implement band-pass filters. In contrast to standard CNNs, that learn all elements of each filter, only low and high cutoff frequencies are directly learned from data with the proposed method. This offers a very compact and efficient way to derive a customized filter bank specifically tuned for the desired application.Our experiments, conducted on both speaker identification and speaker verification tasks, show that the proposed architecture converges faster and performs better than a standard CNN on raw waveforms. Music emotion recognition (MER) is an important topic in music understanding, recommendation, retrieval and human computer interaction. Great success has been achieved by machine learning methods in estimating human emotional response to music. However, few of them pay much attention in semantic interpret for emotion response. In our work, we first train an interpretable model between acoustic audio and emotion. Filter, wrapper and shrinkage methods are applied to select important features. We then apply statistical models to build and explain the emotion model. Extensive experimental results reveal that the shrinkage methods outperform the wrapper methods and the filter methods in arousal emotion. In addition, we observed that only a small set of the extracted features have the key effects to arousal. While, most of our extracted features have small contribution to valence music perception. Ultimately, we obtain a higher average accuracy rate in arousal, compared to that in valence.