1. A principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items (see Section 2). 2. A taxonomy of 13 hazard categories with definitions and subcategories (see Section 3). 3. Tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts (see Section 4). There are 43,090 test items in total, which we created with templates. 4. A grading system for AI systems against the benchmark that is open, explainable, and can be adjusted for a range of use cases (see Section 5). 5. An openly available platform, and downloadable tool, called **ModelBench** that can be used to evaluate the safety of AI systems on the benchmark.2 Footnote 2: [https://github.com/mlcommons/modelbench](https://github.com/mlcommons/modelbench)
6. An example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models. All models have been anonymized (see Section 6).