**Downstream Task Accuracy.** To evaluate downstream performance of our trained models, we report average 0-shot accuracy across the 16 NLP tasks from Zhang et al. [59], and use a prompting methodology consistent with Zhang et al. [59]. These set of 16 NLP tasks include Arc Challenge and ArcEasy [12], HellaSwag [58], OpenBookQA [33], PIQA [7], StoryCloze [36], Winograd [28], Winogrande [42], as well as tasks from SuperGLUE [52]. We refer the reader to Zhang et al. [59] for more information about this evaluation setup.