Data Representation: We use the MAESTRO v2.0.0 dataset [1]} which consists of 1,282 performances with aligned audio-MIDI pairs,
and we split the data
as annotated.
We train on 20-second random crops from audio clips,
which are converted into a log-scale Mel-spectrogram with 80 Mel-filters (\(\textbf {X}^{\scriptsize {\textrm {}}}\) ).
The MIDI note sequences are represented as piano rolls,
from which the onset roll (\(\textbf {Y}^{\scriptsize {\textrm {onset}}}\) ) is extracted.
We represent both articulation (staccato, legato) and dynamics (loud, soft) as binary sequences \(\textbf {c}^{\scriptsize {\textrm {art}}}_{1...T}\) , \(\textbf {c}^{\scriptsize {\textrm {dyn}}}_{1...T} \in \lbrace 0, 1\rbrace \) , each of which corresponds to two components of a Gaussian mixture.
\(c^{\scriptsize {\textrm {art}}}_{t} = 1\)  if at least one note is held at \(t\) ;
and \(c^{\scriptsize {\textrm {dyn}}}_{t} = 1\)  if the average velocity across all notes is greater than 70 at \(t\) ,
which the threshold is determined by our preliminary data analysis.
