The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users).1 We created a new taxonomy of 13 hazard categories, of which seven have tests in the v0.5 benchmark. We plan to release v1.0 of the AI Safety Benchmark by the end of 2024.