Language models are commonly evaluated on the One Billion Word Benchmark (lm1b) [1]}, reporting performance on perplexity [2]}. lm1b is derived from the WMT 2011 News Crawl datasethttp://www.statmt.org/wmt11/translation-task.html released ten years ago. Though the number of citations per year is decreasing, it is still widely used by researchers in recent years, as shown in Figure REF . Examples in the dataset were constructed by extracting single sentences from news articles, such as:
