Footnote 2: CC-RecentNews consists of 221,779 articles (\(\sim\)168M tokens), which is estimated to be about 750 times smaller than C4, a cleansed version of the April 2019 Common Crawl dataset ([https://commoncrawl.org/](https://commoncrawl.org/)) that was used to initially pretrain the T5 LM (Raffel et al., 2019).