Automatic song writing usually covers several tasks including lyric generation [1]}, melody generation [2]}, lyric-to-melody generation (L2M) [3]}, [4]} and melody-to-lyric generation (M2L) [5]}, [6]}. In this work, we focus on L2M and M2L. [3]}, [4]} generated melody conditioned on the lyrics with RNN-based language model. [9]}, [5]} used sequence to sequence model for L2M and M2L. However, these works on L2M and M2L usually only used limited paired data, without leveraging large-scale unpaired data. On the other hand, some works only focused on L2M and M2L on the sentence level, assuming there are strict one-to-one mapping in the training data, which cannot compose a complete song. Some other works [5]}, [12]} explicitly predicted the alignment flag (e.g., whether switches to next word/syllable when predicting notes or not) in the model, with a greedy decision in the word/syllable or note level, which is not flexible and fail to capture the global alignment in the whole sentence. In this paper, we propose SongMASS, which uses sequence to sequence pre-training method to leverage the unpaired lyric and melody data, and attention-based alignment constraints for global and precise lyric-melody alignment.
