When fine-tuning is combined with multi-task learning in a two-stage pipeline, we improve the model's CSR and outperform state-of-the-art perplexity on the WritingPrompts<cite> (Fan et al., 2018)</cite> Our primary task is to perform language modeling (Elman, 1990; Bengio et al., 2003; Dai and Le, 2015) on the WritingPrompts dataset.