* MMLU (Hendrycks et al., 2020), Massive Multitask Language Understanding, is a benchmark designed to measure capability of language models. It covers 57 subjects across STEM, the humanities, the social sciences, and more areas. We only use the test split for reporting the test results, and report the average score across all tasks. * BBH (Suzgun et al., 2022), BIG-Bench-Hard, includes 23 challenging BIG-Bench tasks that prior language models did not outperform average human-raters.