Melody Extraction on Vocal Segments Using Multi Column Deep Neural Networks
Singing melody extraction is a task that tracks pitch contour of singing voice in polyphonic music. While the majority of melody extraction algorithms are based on computing a saliency function of pitch candidates or separating the melody source from the mixture, data driven approaches based on classification have been rarely explored. In this paper, we present a classification based approach for melody extraction on vocal segments using multi column deep neural networks. In the proposed model, each of neural networks is trained to predict a pitch label of singing voice from spectrogram, but their outputs have different pitch resolutions. The final melody contour is inferred by combining the outputs of the networks and post processing it with a hidden Markov model. In order to take advantage of the data driven approach, we also augment training data by pitch shifting the audio content and modifying the pitch label accordingly. We use the RWC dataset and vocal tracks of the MedleyDB dataset for training the model and evaluate it on the ADC 2004, MIREX 2005 and MIR 1k datasets. Through several settings of experiments, we show incremental improvements of the melody prediction. Lastly, we compare our best result to those of previous state of the arts.
ISMIR