In this paper, we present a novel state of the art system for automatic downbeat tracking from music signals. The audio signal is first segmented in frames which are synchronized at the tatum level of the music. We then extract different kind of features based on harmony, melody, rhythm and bass content to feed convolutional neural networks that are adapted to take advantage of each feature characteristics. This ensemble of neural networks is combined to obtain one downbeat likelihood per tatum. The downbeat sequence is finally decoded with a flexible and efficient temporal model which takes advantage of the metrical continuity of a song. We then perform an evaluation of our system on a large base of 9 datasets, compare its performance to 4 other published algorithms and obtain a significant increase of 16.8 percent points compared to the second best system, for altogether a moderate cost in test and training. The influence of each step of the method is studied to show its strengths and shortcomings.
Past approaches use domain-specific knowledge of music for feature design, often from a single attribute only. For example, chromas @cite_35 @cite_6 @cite_38 @cite_21 and spectral component histograms @cite_22 have been utilized as harmonic features, having in mind that the harmonic content is more likely to change around the downbeat positions. Other features, more generally considered as rhythm markers, are based on the onset detection function (ODF) @cite_22 @cite_16 @cite_13 @cite_38 @cite_21 . It is worth noting here that, even if isolated onsets are not always present at downbeat positions, onset patterns will often be synchronous with the bar changes. ODF are often extracted across frequency bands @cite_30 @cite_24 in order, for instance, to separate the events corresponding to kick drums or bass from those of the snare. A third category concerns timbre inspired features @cite_5 @cite_38 @cite_11 @cite_50 . Alterations of the timbre content occur more likely at the start of a new section and near a downbeat position. This feature extraction step can be done in conjunction with an onset @cite_11 , tatum @cite_40 or beat @cite_0 segmentation.
@cite_30 @cite_35 @cite_38 @cite_22 @cite_21 @cite_6 @cite_24 @cite_0 @cite_40 @cite_50 @cite_5 @cite_16 @cite_13 @cite_11
2155764689 326350596 2146516687 2113332804
A method is described which analyzes the basic pattern of beats in a piece of music, the musical meter. The analysis is performed jointly at three different time scales: at the temporally atomic tatum pulse level, at the tactus pulse level which corresponds to the tempo of a piece, and at the musical measure level. Acoustic signals from arbitrary musical genres are considered. For the initial time-frequency analysis, a new technique is proposed which measures the degree of musical accent as a function of time at four different frequency ranges. This is followed by a bank of comb filter resonators which extracts features for estimating the periods and phases of the three pulses. The features are processed by a probabilistic model which represents primitive musical knowledge and uses the low-level observations to perform joint estimation of the tatum, tactus, and measure pulses. The model takes into account the temporal dependencies between successive estimates and enables both causal and noncausal analysis. The method is validated using a manually annotated database of 474 music signals from various genres. The method works robustly for different types of music and improves over two state-of-the-art reference methods in simulations. This dissertation covers machine listening techniques for the automated real-time analysis of live drum performances. Onset detection, drum detection, beat tracking, and drum pattern analysis are combined into a system that provides rhythmic information useful in performance analysis, synchronization, and retrieval. The techniques are designed with real-time use in mind but can easily be adapted for offline batch use for large scale rhythm analysis. At the front end of the system, onset and drum detection provide the locations, types, and amplitudes of percussive events. The onset detector uses an adaptive, causal threshold in order to remain robust to large dynamic swings. For drum detection, a gamma mixture model is used to compute multiple spectral templates per drum onto which onset events can be decomposed using a technique based on non-negative matrix factorization. Unlike classification-based approaches to drum detection, this approach provides amplitude information which is invaluable in the analysis of rhythm. In addition, the decay of drum events are modeled using "tail" templates , which when used with multiple spectral templates per drum, reduce detection errors by 42 . The beat tracking component uses multiple period hypotheses and an ambiguity measure in order to choose a reliable pulse estimate. Results show that using multiple hypotheses significantly improves tracking accuracy compared to a single period model. The drum pattern analysis component uses the amplitudes of the detected drum onsets and the metric grid defined by the beat tracker as inputs to a generatively pre-trained deep neural network in order to estimate high-level rhythmic information. The network is tested with beat alignment tasks, including downbeat detection, and reduces alignment errors compared to a simple template correlation approach by up to 59 . This paper deals with the simultaneous estimation of beat and downbeat location in an audio-file. We propose a probabilistic framework in which the time of the beats and their associated beat-position-inside-a-bar roles; hence, the downbeats, are considered as hidden states and are estimated simultaneously using signal observations. For this, we propose a “reverse” Viterbi algorithm which decodes hidden states over beat-numbers. A beat-template is used to derive the beat observation probabilities. For this task, we propose the use of a machine-learning method, the Linear Discriminant Analysis, to estimate the most discriminative beat-templates. We propose two functions to derive the beat-position-inside-a-bar observation probability: the variation over time of chroma vectors and the spectral balance. We then perform a large-scale evaluation of beat and downbeat-tracking using six test-sets. In this, we study the influence of the various parameters of our method, compare this method to our previous beat and downbeat-tracking algorithms, and compare our results to state-of-the-art results on two test-sets for which results have been published. We finally discuss the results obtained by our system in the MIREX-09 and MIREX-10 contests for which our system ranked among the first for the “McKinney Collection” test-set. Up to now, there has only been little work on using features from temporal approximations of signals for audio recognition. Time-frequency tradeoffs are an important issue in signal processing; sparse representations using overcomplete dictionaries may (or may not, depending on the dictionary) have more time-frequency flexibility than standard short-time Fourier transform. Also, the precise temporal structure of signals cannot be captured by spectral-based feature methods. Here, we present a biologically inspired three-step process for audio classification: 1) Efficient atomic functions are learned in an unsupervised manner on mixtures of percussion sounds (drum phrases), optimizing the length as well as the shape of the atoms. 2) An analog spike model is used to sparsely approximate percussion sound signals (bass drum, snare drum, hi-hat). The spike model consists of temporally shifted versions of the learned atomic functions, each having a precise temporal position and amplitude. To obtain the decomposition given a set of atomic functions, matching pursuit is used. 3) Features are extracted from the resulting spike representation of the signal. The classification accuracy of our method using a support vector machine (SVM) in a 3-class database transfer task is 87.8 . Using gammatone functions instead of the learned sparse functions yields an even better classification rate of 97.6 . Testing the features on sounds containing additive white Gaussian noise reveals that sparse approximation features are far more robust to such distortions than our benchmark feature set of timbre descriptor (TD) features.