Automatic song writing is an interesting and challenging task in both research and industry. Two most important tasks in automatic song writing are lyric-to-melody generation (L2M) [1]}, [2]}, [3]} and melody-to-lyric generation (M2L) [4]}, [5]}, [3]}. L2M and M2L can be regarded as sequence to sequence learning tasks and can be modeled by the techniques in natural language processing since both melody and lyric can be represented as discrete token sequence. However, L2M and M2L have distinctive characteristics that differ them from other sequence to sequence learning tasks: 1) lyric and melody are weakly correlated in L2M and M2L while in other tasks [7]}, [8]}, source and target sequences are strongly correlated in semantics; 2) one word or syllable in lyric always strictly aligns with one or more notes in melody, while other tasks do not require strict alignments. An example of aligned lyric and melody piece is shown in Figure REF .
