
Condition the NN on the chord progression.
The chord progression is provided to the NN at every level,
so when reproducing a melody, the decoder has access to both
the representation and the chord progression.
This is useful because a melody has rhythmic information,
intervallic content, and contour.
The decoder can ignore the separately provided harmonic information,
and use only the melody's other aspects.
This also lets the representation remain constant while altering the chord progression,
so the NN can adapt a melody to a changed chord progression,
such as what happens when a key changes from minor to major.


Add a stochastic layer.
Autoencoders which learn a stochastic representation are called variational
autoencoders, and perform well in generative modelling of images [1]}.
The representation is not deterministic. We assume a particular
(Gaussian) distribution in the representation space, and then train
the NN to transform this distribution to match the distribution of
input melodies in their high dimensional space. This ensures that we can
take a random sampling of the representation space following its associated
probability distribution, then feed it through the decoder and expect
a melody similar to the set of musically sensible melodies.


Use recurrent connections.
Pop music has many time-invariant elements, especially at
time scales below a few beats. A recurrent NN shares
the same processing infrastructure for note sequences starting at
different times, and thereby accelerates learning.


Normalize all other notes relative to the tonic.
Pop music is also largely pitch invariant, insofar as a song
transposed by a few semitones still sounds perceptually similar.
The NN ignores the song's key and considers the tonic pitch to be abstract,
as far as pitches in melody and chords are concerned.


