Discovering and exploring the underlying structure of multi-instrumental music using learning-based approaches remains an open problem. We extend the recent MusicVAE model to represent multitrack polyphonic measures as vectors in a latent space. Our approach enables several useful operations such as generating plausible measures from scratch, interpolating between measures in a musically meaningful way, and manipulating specific musical attributes. We also introduce chord conditioning, which allows all of these operations to be performed while keeping harmony fixed, and allows chords to be changed while maintaining musical "style". By generating a sequence of measures over a predefined chord progression, our model can produce music with convincing long-term structure. We demonstrate that our latent space model makes it possible to intuitively control and generate musical sequences with rich instrumentation (see this https URL for generated audio).
The MusicVAE architecture introduced by @cite_39 is able to learn a latent space of musical sequences using a novel hierarchical decoder that allows it to model long-term structure and multi-instrument sequences. However, this work applies strict constraints to the sequences to reach its goals. In order to guarantee a constant number of events per measure, non-drum tracks are limited to monophonic sequences, and all tracks are represented with a single velocity and quantized at the level of 16 th notes. This reduces the challenge of modeling longer-term structure, but at the expense of expressiveness. Furthermore, while the trio'' MusicVAE is capable of modeling three broad instrument classes--melody, bass, and drums--it is limited to exactly three instruments, arbitrarily excluding potentially pivotal voices and disregarding the specific identity of each instrument (e.g. eletric guitar and piano are both considered melody'' instruments). Further, do not consider modeling fine-grained timing and velocity, nor do they develop a method for chord conditioning. Nevertheless, the MusicVAE architecture and its implementation provide a powerful basis for exploring a more expressive and complete multitrack latent space, and thus we position our work as an extension of it.
@cite_39
2792210438
Discovering and exploring the underlying structure of multi-instrumental music using learning-based approaches remains an open problem. We extend the recent MusicVAE model to represent multitrack polyphonic measures as vectors in a latent space. Our approach enables several useful operations such as generating plausible measures from scratch, interpolating between measures in a musically meaningful way, and manipulating specific musical attributes. We also introduce chord conditioning, which allows all of these operations to be performed while keeping harmony fixed, and allows chords to be changed while maintaining musical "style". By generating a sequence of measures over a predefined chord progression, our model can produce music with convincing long-term structure. We demonstrate that our latent space model makes it possible to intuitively control and generate musical sequences with rich instrumentation (see this https URL for generated audio).