Poem Generation using Transformers and Doc2Vec Embeddings
Poems are sequences of words that express ideas and emotions in an imaginative style, some following strict literary syntax or form. They are artistic expressions, and their generation requires in depth knowledge and mastery of language. As such, poem generation is considered a very challenging task in Natural Language Processing and has been attracting research interest in the recent decade. We propose a method of generating poems using transformers, coupled with doc2vec embeddings in order to assess the automatically generated poems. In this method, we first train a transformer and a doc2vec model using a poem dataset. Then the trained transformer takes an input text and produces several generated poems. To have an objective basis for assessing the generated poems, we present a preliminary attempt at measuring the quality of a machine generated poem by computing the cosine similarity score, referenced to the trained doc2vec model. This score is used as a basis for choosing the final output poem. The results show that this method ensures good cohesion between the machine generated poem and the given input text. We then also explore the implication of the transformer training to the doc2vec embeddings of its output poems, which are shown to be more similar to poems (documents) in the train set as training progresses. Finally, we demonstrate how transformers can learn some poetry styles by exposing them to poems of specific poets.
2020 International Joint Conference on Neural Networks (IJCNN)