CC-RecentNewsWe first construct CC-RecentNews, a novel text corpus containing relatively _new_ knowledge as \(D_{1}\). We use news-please (Hamborg et al., 2017), similar to the CC-NEWS (Liu et al., 2019) and REALNEWS dataset (Zellers et al., 2019), to crawl 221,779 news articles published from May 2020 to April 2021. LMs initially pretrained on \(D_{0}\) constructed before May 2020 can be continually pretrained on CC-RecentNews to gain relatively _recent_ world knowledge.