C4 in Figure 12, and even lower than 1.3B models trained only on C4 in Figure 1c. We also show an increase of 1.5% across general understanding language tasks, and roughly 3% on specialized knowledge tasks in Tables 8-9 when adding QA rephrases. We also experimented with medium rephrases at this smaller scale. Our findings indicate that the high quality provided by medium rephrases improves performance over only C4, however matching the style as indicated by QA rephrase performance further improves performance.