The network is implemented in Tensorflow, a machine learning library for
rapid prototyping and production training [1]}. It was trained for four days on an
Nvidia Tesla K80 GPU.
We used Gated Recurrent Units [2]} to build the bidirectional recurrent layer
and Exponential Linear Units [3]} as activation functions.
These significantly accelerate training
while simplifying the network [4]}, [3]}.
fig:trainerror shows the training error (the sum of model reproduction errors)
and the difference of the latent distribution from a unit Gaussian distribution,
as measured by Kullback-Leibler divergence [6]}.
The network's input data (available at https://goo.gl/VezNNA)
is a set of MIDI songs from various online sources.
Our harmonic analysis converted this to
\(1.9\times 10^6\)  measures of melodies and corresponding chords.
We implemented KL warm up, because that is crucial to learning for
a variational autoencoder [7]}. But instead of linearly scaling
the KL term for this, we found that a sigmoid reduced the network's reproduction loss.
<FIGURE>