* **Natural Questions**[179] is a QA dataset that consists of real anonymized, aggregated queries submitted to the Google search engine as questions. An annotator is presented with a question along with a Wikipedia page from the top \(5\) search results, and annotates a long answer (typically a paragraph) and a short answer (one or more entities) if present on the page, or marks null if no long/short answer is present. * **MMLU**[180] is intended to evaluate the knowledge gained in zero-shot and few-shot scenarios. That means that MMLU assesses both the general knowledge and problem-solving ability of a model. It covers 57 subjects in STEM, humanities, social sciences, and other areas. The benchmark varies in complexity, ranging from elementary to advanced professional. It is worth mentioning that the main contribution of this dataset is for multi-task language understanding, question answering, and arithmetic reasoning. * **MBPP**[181] stands for "Mostly Basic Python Problems" and provides a benchmark for evaluating the performance of models designed for code generation. The benchmark encompasses \(974\) short Python programs including a wide range of topics, including fundamental programming concepts and standard library usage, and more. Each challenge comprises a task description, a code solution, and three automated test cases. * **HumanEval**[182] is a dataset for code generation task. This dataset consists of \(164\) hand-crafted programming challenges. Each challenge is accompanied by a function signature, docstring, code body, and multiple unit tests. The main intuition behind developing this dataset is to guarantee the exclusion of its contents from training datasets for code generation models. * **APPS**[183] is designed for code generation task focusing on the Python programming language. The APPS dataset contains a collection of \(232,444\) Python programs. Each program in the dataset has an average of \(18\) lines of Python code. Additionally, APPS offers access to a repository of \(10,000\) unique programming exercises, each with text-based problem descriptions. The final aspect to highlight is that the it includes test cases. * **WikiSQL**[184] is crafted for code generation task and it has 87,726 carefully labeled pairs of SQL queries and corresponding natural language questions from Wikipedia tables. The SQL queries comprise three subsets: test sets (\(17,284\) examples), development (\(9,145\) examples), and training (\(61,297\) examples). * **TriviaQA**[185] is designed for QA task. This dataset comprises more than \(650,000\) question-answer-evidence triples. There are \(95,000\) question-answer pairs in this dataset, each authored by trivia enthusiasts and supported by an average of six independently sourced evidence documents. These documents are automatically acquired from Wikipedia or broader web search results. The dataset is categorized into two segments, including those with authentic answers from Wikipedia and web domains, and verified sets embody the accurately answered questions along with their associated documents from both Wikipedia and online.