What is more interesting is its generalizability.
All the deep learning based systemsRNN, tanhB, ReLUts, lstmpB, GRUts - RNN-based systems present deteriorating performance in the transfer learning scenario (eval-cross) compared to the dataset split scenario (eval-subset).See Figure 10 (b) of [1]}. Note that most of the reported scores in papers also follow eval-subset setup. However, less data-driven approachesSANMF, NMF, PFNMF, AM1, AM2 - NMF-based systems present similar or even increased performances in eval-cross. This implies that the distributions within datasets are fairly different and biased to certain types of drum tracks
and therefore, a transcription system trained with those datasets will be also biased accordingly. This limitation may be attributed to the small sizes of those datasets. Theoretically, supervised deep learning systems may generalize better if trained on a very large dataset, which lacks practicality due to the high annotation cost. In contrast, it is relatively easy to unbias DrummerNet. One only needs to control the distribution of drum tracks by their style/genre/sounds without annotating every note.
<FIGURE><FIGURE>